Call us: +1-415-738-4000

Terracotta Configuration Reference

Introduction

This document is a reference to all of the Terracotta configuration elements found in the Terracotta configuration file. The Terracotta configuration file is named tc-config.xml by default.

Configuration Schema

The documentation for the Terracotta configuration XML document can be found here.

Reference and Sample Configuration Files

The reference configuration is a sample configuration file with inline comments describing each configuration element. The Terracotta kit contains a reference configuration file under the /config-samples directory.

While this file serves as a good reference for the Terracotta configuration, do not use it as the basis for your Terracotta config. For that purpose, you should start with one of the sample configurations provided in the Terracotta kit. Typically the sample configurations are named tc-config.xml and can be found under the various samples provided with the Terracotta kit.

Configuration Structure

The Terracotta configuration is an XML document that has these main sections: system, servers, clients. Each of these sections provides a number of configuration options relevant to its particular configuration topic.

Configuration Variables

There are a few variables that will be interpolated by the configuration subsystem:

VariableInterpolated Value
%hThe hostname
%iThe ip address
%DTime stamp (yyyyMMddHHmmssSSS)
%(_system property_)The value of the given system property

These variables will be interpolated in the following places:

  • the "name", "host" and "bind" attributes of the <server> element
  • the password file location for JMX authentication
  • client logs location
  • server logs location
  • server data location
NOTE: Value of %i
The variable %i is expanded into a value determined by the host's networking setup. In many cases that setup is in a hosts file containing mappings that may influence the value of %i.

Overriding tc.properties

Every Terracotta installation has a default tc.properties file containing system properties. Normally, the settings in tc.properties are pre-tuned and should not be edited.

If tuning is required, you can override certain properties in tc.properties using tc-config.xml. This can make a production environment more efficient by allowing system properties to be pushed out to clients with tc-config.xml. Those system properties would normally have to be configured separately on each client.

Setting System Properties in tc-config

To set a system property with the same value for all clients, you can add it to the Terracotta server's tc-config.xml file using a configuration element with the following format:

<property name="<tc_system_property>" value="<new_value>" />

All <property /> tags must be wrapped in a <tc-properties> section placed at the beginning of tc-config.xml.

For example, to override the values of the system properties l1.cachemanager.enabled and l1.cachemanager.leastCount, add the following to the beginning of tc-config.xml:

<tc-properties>
  <property name="l1.cachemanager.enabled" value="false" />
  <property name="l1.cachemanager.leastCount" value="4" />
</tc-properties> 

Override Priority

System properties configured in tc-config.xml override the system properties in the default tc.properties file provided with the Terracotta kit. The default tc.properties file should not be edited or moved.

If you create a local tc.properties file in the Terracotta lib directory, system properties set in that file are used by Terracotta and will override system properties in the default tc.properties file. System properties in the local tc.properties file are not overridden by system properties configured in tc-config.xml.

System property values passed to Java using -D override all other configured values for that system property. In the example above, if -Dcom.tc.l1.cachemanager.leastcount=5 was passed at the command line or through a script, it would override the value in tc-config.xml and tc.properties. The order of precedence is shown in the following list, with highest precedence shown last:

  1. default tc.properties
  2. tc-config.xml
  3. local, or user-created tc.properties in Terracotta lib directory
  4. Java system properties set with -D

Failure to Override

If system properties set in tc-config.xml fail to override default system properties, a warning is logged to the Terracotta logs. The warning has the following format:

The property <system_property_name> was set by local settings to <value>. 
This value will not be overridden to <value> from the tc-config.xml file.

System properties used early in the Terracotta initialization process may fail to be overridden. If this happens, a warning is logged to the Terracotta logs. The warning has the following format:

The property <system_property_name> was read before initialization completed. 

The warning is followed by the value assigned to <system_property_name>.

The property tc.management.mbeans.enabled is known to load before initialization completes and cannot be overridden.

System Configuration Section

/tc:tc-config/system/configuration-model

The configuration-model element is for informational purposes. The two configuration-model options are 'development' and 'production'. These values have no effect on the functioning of Terracotta servers or clients, but instead allow you to designate the intended use of a configuration file using one of two recommended modes.

In development, you may want each client might have its own configuration, independent of the server or any other client. This approach is useful for development, but should not be used in production as it can result in unpredictable behavior should conflicts arise. To note that a configuration file is intended for direct use by a client, set the configuration-model to 'development'.

In production, each client should obtain its configuration from a Terracotta server instance. To note that a configuration file is intended be be fetched from a server, set the configuration-model to 'production'.

In general, a client can specify that its configuration come from a server by setting the tc.config system propery:

-Dtc.config=serverHost:dsoPort

Servers Configuration Section

/tc:tc-config/servers

This section defines the Terracotta server instances present in your cluster. One or more entries can be defined. If this section is omitted, Terracotta configuration behaves as if there's a single server instance with default values.

The attribute secure is a global control for enabling ("true") or disabling ("false" DEFAULT) SSL-based security for the entire cluster.

Each Terracotta server instance needs to know which configuration it should use as it starts up. If the server's configured name is the same as the hostname of the host it runs on and no host contains more than one server instance, then configuration is found automatically.

For more information on how to use the Terracotta configuration file with servers, see the Terracotta configuration guide.

/tc:tc-config/servers/server

A server stanza encapsulates the configuration for a Terracotta server instance. The server element takes three optional attributes (see table below).

AttributeDefinitionValueDefault Value
host The ID of the machine hosting the Terracotta server Host machine's IP address or resolvable hostname Host machine's IP address
name The ID of the Terracotta server; can be passed to Terracotta scripts such as start-tc-server using n <name> user-defined string :
bind The network interface on which the Terracotta server listens for Terracotta clients; 0.0.0.0 specifies all interfacesinterface's IP address 0.0.0.0

Sample configuration snippet:

<server>
  <!-- my host is '%i', my name is '%i:dso-port', my bind is 0.0.0.0 -->
  ...
</server>
<server host="myhostname">
  <!-- my host is 'myhostname', my name is 'myhostname:dso-port', my bind is 0.0.0.0 -->
  ...
</server>
<server host="myotherhostname" name="server1" bind="192.168.1.27">
  <!-- my host is 'myotherhostname', my name is 'server1', my bind is 192.168.1.27 -->
  ...
</server>

/tc:tc-config/servers/server/data

This section lets you declare where the server should store its data.

Sample configuration snippet:

  <!-- Where should the server store its persistent data? (This includes
     stored object data for DSO.) This value undergoes parameter substitution
     before being used; this allows you to use placeholders like '%h' (for the hostname)
     or '%(com.mycompany.propname)' (to substitute in the value of
     Java system property 'com.mycompany.propname'). Thus, a value
     of 'server-data-%h' would expand to 'server-data-artichoke' if 
     running on host 'artichoke'.

     If this is a relative path, then it is interpreted relative to
     the location of this file. It is thus recommended that you specify
     an absolute path here.

     Default: 'data'; this places the 'data' directory in the same
     directory as this config file.
  -->
  <data>/opt/terracotta/server-data</data>

/tc:tc-config/servers/server/logs

This section lets you declare where the server should write its logs.

Sample configuration snippet:

  <!-- In which directory should the server store its log files? Again,
     this value undergoes parameter substitution before being used;
     thus, a value like 'server-logs-%h' would expand to
     'server-logs-artichoke' if running on host 'artichoke'. See the
     Product Guide for more details.

     If this is a relative path, then it is interpreted relative to
     the location of this file. It is thus recommended that you specify
     an absolute path here.

     Default: 'logs'; this places the 'logs' directory in the same
     directory as this config file.
  -->
  <logs>/opt/terracotta/server-logs</logs>

You can also specify stderr: or stdout: as the output destination for log messages. For example:

<logs>stdout:</logs>

To set the logging level, see this FAQ entry.

/tc:tc-config/servers/server/statistics

This section lets you declare where the server should store buffered statistics data.

Sample configuration snippet:

<!-- In which directory should the server store statistics
     data that is being buffered? Again, this value undergoes
     parameter substitution before being used; thus, a value
     like 'statistics-data-%h' would expand to 'statistics-data'
     if running on host 'artichoke'. See the Product Guide for
     more details.

     If this is a relative path, then it is interpreted relative to the
     current working directory of the server (that is, the directory
     you were in when you started server). It is thus recommended
     that you specify an absolute path here.

     Default: 'statistics'; this places the 'statistics' directory in the
     directory you were in when you invoked 'start-tc-server'.
-->
<statistics>/opt/terracotta/server-statistics</statistics>

/tc:tc-config/servers/server/dso-port

This section lets you set the port that the Terracotta server listens to. It is called 'dso-port' for historical reasons—it is not related to Distributed Shared Objects.

The default value of 'dso-port' is 9510.

Sample configuration snippet:

<dso-port>9510</dso-port>

/tc:tc-config/servers/server/jmx-port

This section lets you set the port that the Terracotta server's JMX Connector listens to.

The default value of 'jmx-port' is 9520.

Sample configuration snippet:

<jmx-port>9520</jmx-port>

/tc:tc-config/servers/server/l2-group-port

This section lets you set the port that the Terracotta server uses to communicate with other Terracotta servers when the servers are run in network-based active-passive mode.

The default value of 'l2-group-port' is 9530.

Sample configuration snippet:

<l2-group-port>9530</l2-group-port>

/tc:tc-config/servers/server/security

This section contains the data necessary for running a secure cluster based on SSL, digital certificates, and node authentication and authorization.

See the advanced security page for a configuration example.

/tc:tc-config/servers/server/security/ssl/certificate

The element specifying certificate entry and location of the certificate store. The format is:

<store-type>:<certificate-alias>@</path/to/keystore.file>

The Java Keystore (JKS) type is supported by Terracotta 3.7 and higher.

/tc:tc-config/servers/server/security/keychain

This element contains the following subelements:

  • <class> – Element specifying the class defining the keychain file. If a class is not specified, com.terracotta.management.keychain.FileStoreKeyChain is used.
  • <url> – The URI for the keychain file. It is passed to the keychain class to specify the keychain file.

/tc:tc-config/servers/server/security/auth

This element contains the following subelements:

  • <realm> – Element specifying the class defining the security realm. If a class is not specified, com.tc.net.core.security.ShiroIniRealm is used.
  • <url> – The URI for the Realm configuration (.ini) file. It is passed to the realm class to specify authentication file.
  • <user> – The username that represents the server and is authenticated by other servers. This name is part of the credentials stored in the .ini file.

/tc:tc-config/servers/server/authentication

In this section, you can configure security using the Lightweight Directory Access Protocol (LDAP) or JMX authentication. Enabling one of these methods causes a Terracotta server to require credentials before allowing a JMX connection to proceed.

For more information on how to configure authentication, including secure SSL-based connections, see Terracotta Cluster Security.

/tc:tc-config/servers/server/http-authentication/user-realm-file

Turn on authentication for the embedded Terracotta HTTP Server. This requires a properties file that contains the users and passwords that have access to the HTTP server.

The format of the properties file is:

username: password [,rolename ...]

The supported roles and protected sections are: * statistics (for the statistics gatherer at /statistics-gatherer.)

Passwords may be clear text, obfuscated or checksummed. The class com.mortbay.Util.Password should be used to generate obfuscated passwords or password checksums.

By default, HTTP authentication is turned off.

Sample configuration snippet:

<http-authentication>
  <user-realm-file>/opt/terracotta/realm.properties</user-realm-file>
</http-authentication>

/tc:tc-config/servers/server/dso

This section contains configuration specific to the functioning of the Terracotta server. It is called 'dso' for historical reasons—it is not related to Distributed Shared Objects.

/tc:tc-config/servers/server/dso/client-reconnect-window

This section lets you declare the client reconnect-time window. The value is specified in seconds and the default is 120 seconds. If adjusting the value in <client-reconnect-window>, note that a too-short reconnection window can lead to unsuccessful reconnections during failure recovery, and a too-long window lowers the efficiency of the network.

Sample configuration snippet:

    <client-reconnect-window>120</client-reconnect-window>

Further reading: For more information on client and server reconnection is executed in a Terracotta cluster, and on tuning reconnection properties in a high-availability environment, see Configuring and Testing Terracotta For High Availability.

/tc:tc-config/servers/server/dso/persistence

This section lets you configure whether Terracotta operates in persistent (permanent-store) or non-persistent mode (temporary-swap-only).

Temporary-swap-only mode is the default mode. This mode uses the disk as a temporary backing store. Data is not preserved across server restarts and cluster failures, although shared in-memory data may survive if a backup Terracotta server is set up. This mode maximizes performance and should be used where restoring data after failures is not dependent on the server.

Permanent-store mode backs up all shared in-memory data to disk. This backed-up data survives server restarts and cluster failures, allowing all application state to be restored. This mode is recommended for ensuring a more robust failover architecture, but will not perform as well as temporary-swap-mode.

Further reading: For more information on Terracotta cluster persistence and restarting servers, see the architecture guide.

Sample configuration snippet:

<persistence>
  <!--  
     Default: 'temporary-swap-only'
  -->
  <mode>permanent-store</mode>
</persistence>

/tc:tc-config/servers/server/dso/garbage-collection

This section lets you configure the periodic distributed garbage collector (DGC) that runs in the Terracotta server.

Further reading:

Configuration snippet:

<garbage-collection>


  <!-- If 'true', distributed garbage collection is enabled.
     You should only set this to 'false' if you are
     absolutely certain that none of the data underneath
     your roots will ever become garbage; certain 
     applications, such as those that read a large amount
     of data into a Map and never remove it (merely look up
     values in it) can safely do this.

     Default: true
  -->
  <enabled>true</enabled>

  <!-- If 'true', the DSO server will emit extra information when
     it performs distributed garbage collection; this can
     be useful when trying to performance-tune your
     application.

     Default: false
  -->
  <verbose>false</verbose>

  <!-- How often should the DSO server perform distributed 
     garbage collection, in seconds?

     Default: 3600 (60 minutes)
  -->
  <interval>3600</interval>
</garbage-collection>

/tc:tc-config/servers/ha

Use this section to indicate properties associated with running your servers in active-passive mode. The properties apply to all servers defined. You can omit this section, in which case your servers, running in persistent mode, will run in networked active-passive mode.

To allow for at most 1 <ha> section to be defined along with multiple <server> sections, define multiple <server> sections followed by one <ha> section (or no <ha> section).

Sample configuration snippet:

<servers>
  <server host="%i" name="sample1">
  </server>
  <server host="%i" name="sample2">
  </server>
  <ha>
    <mode>networked-active-passive</mode>
  </ha>
</servers>

/tc:tc-config/servers/ha/mode

This section allows you configure whether servers run in network-based (default) or disk-based active-passive High Availability (HA) mode. Network-based servers have separate databases and sync over the network. Disk-based servers share the database, which is locked by the active server. Disk-based HA is appropriate only in special use-cases.

There are two corresponding value options: 'networked-active-passive' and 'disk-based-active-passive'.

When using networked-active-passive mode, Terracotta server instances must not share data directories. Each server's <data> element should point to a different and preferably local data directory.

Sample configuration snippet:

<mode>networked-active-passive</mode>

/tc:tc-config/servers/ha/networked-active-passive

This section allows you to declare the election time window, which is used when servers run in network-based active-passive mode. An active server is elected from the servers that cast a vote within this window. The value is specified in seconds and the default is 5 seconds. Network latency and work load of the servers should be taken into consideration when choosing an appropriate window.

Sample configuration snippet:

<networked-active-passive>
    <election-time>5</election-time>
</networked-active-passive>

/tc:tc-config/servers/mirror-groups

Use the <mirror-groups> section to bind groups of Terracotta server instances into a server array. The server array is built from sets of Terracotta server instances with one active server instance and one or more mirrors ("hot standbys" or back-ups).

The following table summarizes the elements contained in a <mirror-groups> section.

Element Definition Attributes
<mirror-groups> Encapsulates any number of <mirror-group> sections. Only one <mirror-groups> section can exist in a Terracotta configuration file. None
<mirror-group> Encapsulates one <members> element and one <ha> element. <group-name> can be assigned a unique non-empty string value. If not set, Terracotta automatically creates a unique name for the mirror group.
<members> Encapsulates <member> elements. Only one <members> section can exist in a <mirror-group> section. None
<ha> Applies high-availability settings to the encapsulating mirror group only. Settings are the same as those available in the main high-availability section. Mirror groups without an <ha> section take their high-availability settings from the main high-availability section. None
<member> Contains the value of the server-instance name attribute configured in a <server> element. Each <member> element represents a server instance assigned to the mirror group. None

For examples and more information, see the Terracotta Configuration Guide.

Clients Configuration Section

The clients section contains configuration about how clients should behave.

/tc:tc-config/clients/logs

This section lets you configure where the Terracotta client writes its logs.

Sample configuration snippet:

<!--     
   This value undergoes parameter substitution before being used;
   thus, a value like 'client-logs-%h' would expand to
   'client-logs-banana' if running on host 'banana'. See the
   Product Guide for more details.

   If this is a relative path, then it is interpreted relative to
   the current working directory of the client (that is, the directory
   you were in when you started the program that uses Terracotta
   services). It is thus recommended that you specify an absolute
   path here.


   Default: 'logs-%i'; this places the logs in a directory relative
   to the directory you were in when you invoked the program that uses
   Terracotta services (your client), and calls that directory, for example,
   'logs-10.0.0.57' if the machine that the client is on has assigned IP
   address 10.0.0.57.
-->
<logs>logs-%i</logs>

You can also specify stderr: or stdout: as the output destination for log messages. For example:

<logs>stdout:</logs>

To set the logging level, see this FAQ entry.

/tc:tc-config/clients/dso/fault-count

Sets the object fault count. Fault count is the maximum number of reachable objects that are pre-fetched from the Terracotta server to the Terracotta client when an object is faulted from that server to that client.

Pre-fetching an object does not fault the entire object, only the minimum amount of metadata needed to construct the object on the client if necessary. Unused pre-fetched objects are eventually cleared from the client heap.

Objects qualify for pre-fetching either by being referenced by the original object (being part of original object's object graph) or because they are peers. An example of peers are primitives found in the same map.

In most applications, pre-fetching improves locality of reference, which benefits performance. But in some applications, for example a queue fed by producer nodes that shouldn't pre-fetch objects, it may be advantageous to set the fault count to 0.

Sample configuration snippet:

<fault-count>500</fault-count>

Default: 500

/tc:tc-config/clients/dso/debugging

The client debugging options give you control over various logging output options to gain more visibility into what Terracotta is doing at runtime.

/tc:tc-config/clients/dso/debugging/runtime-logging/new-object-debug

When set to true, the addition of every new clustered object is logged.

Default: false

/tc:tc-config/clients/dso/debugging/runtime-output-options

The runtime output options are modifiers to the runtime logging options

/tc:tc-config/clients/dso/debugging/runtime-output-options/caller

When set to true, this option logs the call site in addition to the runtime output.

Default: false

/tc:tc-config/clients/dso/debugging/runtime-output-options/full-stack

When set to true, this option provides a full stack trace of the call site in addition to the runtime output.

Default: false