Overview

Nodes running JVMs can have a large amount of physical memory—16GB, 32GB, and more—but the long-standing problem of Java garbage collection (GC) limits the ability of all Java applications, including Terracotta software, to use that memory effectively. This drawback has limited Terracotta servers, for example, to using a small Java object heap as an in-memory store, backed by a limitless but slower disk store.

How BigMemory Improves Performance

The performance of Terracotta clients and server instances is affected by the amount of faulting required to make data available. In-memory data elements are fetched speedily because memory is very fast. Data elements that are not found in memory must be faulted in from disk, and sometimes from an even slower system of record, such as a database.

While disk-based storage slows applications down, it has the advantage of being limitless in size. In-memory storage is limited in size by system and hardware constraints, yet even this limit is difficult for Java heaps to reach due to the heavy cost imposed by GC. Full GC operations can slow a system to a crawl, and the larger the heap, the more often these operations are likely to occur. In most cases, heaps have been limited to about 2GB in size.

BigMemory allows Terracotta servers to expand memory storage in a way that bypasses the limitations resulting from Java GC. Using this off-heap memory gives the Terracotta cluster a number of important advantages:

• Larger in-memory stores without the pauses of GC.
• More locality of reference as more data is stored at the client.
• Overall reduction in faulting from disk or database.
• Low latencies as a result of more data available to applications at memory speed.
• Fewer Terracotta server stripes required to efficiently handle the same amount of data.

Data stored in off-heap memory is stored in a cache, and therefore all data elements (keys and values) must be serializable. However, the costs imposed by serialization and deserialization are far outweighed by the performance gains noted above.

The following diagram illustrates how BigMemory adds a layer of off-heap memory storage that reduces faulting from the Terracotta server's disk yet remains outside of GC's domain.