Call us: +1-415-738-4000

Working with Terracotta Configuration Files

Introduction

Terracotta XML configuration files set the characteristics and behavior of Terracotta server instances and Terracotta clients. The easiest way to create your own Terracotta configuration file is by editing a copy of one of the sample configuration files available with the Terracotta kit.

Where you locate the Terracotta configuration file, or how your Terracotta server and client configurations are loaded, depends on the stage your project is at and on its architecture. This document covers the following cases:

  • Development stage, 1 Terracotta server
  • Development stage, 2 Terracotta servers
  • Deployment stage

This document discusses cluster configuration in the Terracotta Server Array. To learn more about the Terracotta server instances, see Terracotta Server Arrays.

For a comprehensive and fully annotated configuration file, see config-samples/tc-config-express-reference.xml in the Terracotta kit.

How Terracotta Servers Get Configured

At startup, Terracotta servers load their configuration from one of the following sources:

  • A default configuration included with the Terracotta kit
  • A local or remote XML file

These sources are explored below.

Default Configuration

If no configuration file is specified and no tc-config.xml exists in the directory in which the Terracotta instance is started, then default configuration values are used.

Local XML File (Default)

The file tc-config.xml is used by default if it is located in the directory in which a Terracotta instance is started and no configuration file is explicitly specified.

Local or Remote Configuration File

You can explicitly specify a configuration file by passing the -f option to the script used to start a Terracotta server. For example, to start a Terracotta server on UNIX/Linux using the provided script, enter:

start-tc-server.sh -f <path_to_configuration_file>

where <path_to_configuration_file> can be a URL or a relative directory path. In Microsoft Windows, use start-tc-server.bat.

How Terracotta Clients Get Configured

At startup, Terracotta clients load their configuration from one of the following sources:

  • Local or Remote XML File
  • Terracotta Server
  • An Ehcache configuration file (using the <terracottaConfig> element) used with Enterprise Ehcache and Enterprise Ehcache for Hibernate.
  • A Quartz properties file (using the org.quartz.jobStore.tcConfigUrl property) used with Quartz Scheduler.
  • A Filter (in web.xml) or Valve (in context.xml) elements used with containers and Terracotta Sessions.
  • The client constructor (TerracottaClient()) used when a client is instantiated programmatically using the Terracotta Toolkit.

Terracotta clients can load customized configuration files to specify <client> and <application> configuration. However, the <servers> block of every client in a cluster must match the <servers> block of the servers in the cluster. If there is a mismatch, the client will emit an error and fail to complete its startup.

NOTE: Error with Matching Configuration Files
On startup, a Terracotta client may emit a configuration-mismatch error if its <servers> block does not match that of the server it connects to. However, under certain circumstances, this error may occur even if the <servers> blocks appear to match. The following suggestions may help prevent this error: - Use `-Djava.net.preferIPv4Stack` consistently. If it is explicitly set on the client, be sure to explicitly set it on the server. - Ensure `etc/hosts` file does not contain multiple entries for hosts running Terracotta servers. - Ensure that DNS always returns the same address for hosts running Terracotta servers.

Local or Remote XML File

See the discussion for local XML file (default) in How Terracotta Servers Get Configured.

To specify a configuration file for a Terracotta client, see Clients in Development.

NOTE: Fetching Configuration from the Server
On startup, Terracotta clients must fetch certain configuration properties from a Terracotta server. A client loading its own configuration will attempt to connect to the Terracotta servers named in that configuration. If none of the servers named in that configuration are available, the client cannot complete its startup.

Terracotta Server

Terracotta clients can load configuration from an active Terracotta server by specifying its hostname and DSO port (see Clients in Production).

Configuration in a Development Environment

In a development environment, using a different configuration file for each Terracotta client facilitates the testing and tuning of configuration options. This is an efficient and effective way to gain valuable insight on best practices for clustering your application with Terracotta DSO.

One-Server Setup in Development

For one Terracotta server, the default configuration is adequate.

Terracotta development cluster with one server.

To use the default configuration settings, start your Terracotta server using the start-tc-server.sh (or start-tc-server.bat) script in a directory that does not contain the file tc-config.xml:

[PROMPT] ${TERRACOTTA_HOME}\bin\start-tc-server.sh

To specify a configuration file, use one of the approaches discussed in How Terracotta Servers Get Configured.

Two-Server Setup in Development

Terracotta development cluster with two servers.

A two-server setup, sometimes referred to as an active-passive setup, has one active server instance and one "hot standby" (the passive, or backup) that should load the same configuration file.

The configuration file loaded by the Terracotta servers must define each server separately using <server> elements. For example:

<tc:tc-config xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-5.xsd"
xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
...
<!-- Use an IP address or a resolvable host name for the host attribute. -->
  <server host="123.456.7.890" name="Server1">
...
  <server host="myResolvableHostName" name="Server2">
...
</tc:tc-config>


Assuming Server1 is the active server, using the same configuration allows Server2 to be the hot standby and maintain the environment in case of failover. If you are running both Terracotta servers on the same host, the only port that has to be specified in configuration is the <dso-port>; the values for <jmx-port> and <l2-group-port> are filled in automatically.

NOTE: Running Two Servers on the Same Host
If you are running the servers on the same machine, some elements in the <server> section, such as <dso-port> and <server-logs>, must have different values for each server.

Server Names for Startup

With multiple <server> elements, the name attribute may be required to avoid ambiguity when starting a server:

start-tc-server.sh -n Server1 -f <path_to_configuration_file>

In Microsoft Windows, use start-tc-server.bat.

For example, if you are running Terracotta server instances on the same host, you must specify the name attribute to set an unambiguous target for the script.

However, if you are starting Terracotta server instances in an unambiguous setup, specifying the server name is optional. For example, if the Terracotta configuration file specifies different IP addresses for each server, the script assumes that the server with the IP address corresponding to the local IP address is the target.

Clients in Development

You can explicitly specify a client's Terracotta configuration file by passing -Dtc.config=path/to/my-tc-config.xml when you start your application with the Terracotta client.

DSO users can use the dso-java.sh script (or dso-java.bat for Windows):

dso-java.sh -Dtc.config=path/to/my-tc-config.xml -cp classes myApp.class.Main

where myApp.class.Main is the class used to launch the application you want to cluster with Terracotta. In Microsoft Windows, use dso-java.bat.

If tc-config.xml exists in the directory in which you run dso-java, it can be loaded without -Dtc.config.

TIP: Avoiding the dso-java Script
If you do not require DSO, do not use the dso-java script. Terracotta products provide simple configuration setups. If you are using DSO, you may want to avoid using the dso-java script and start your application with the Terracotta client using your own scripts. See Setting Up the Terracotta Environment.

Configuration in a Production Environment

Terracotta production cluster with two servers.

For an efficient production environment, it's recommended that you maintain one Terracotta configuration file. That file can be loaded by the Terracotta server (or servers) and pushed out to clients. While this is an optional approach, it's an effective way to centralize and decrease maintenance.

If your Terracotta configuration file uses "%i" for the hostname attribute in its server element, change it to the actual hostname in production. For example, if in development you used the following:

<server host="%i" name="Server1">

and the production host's hostname is myHostName, then change the host attribute to the myHostName:

<server host="myHostName" name="Server1">

Clients in Production

For clients in production, you can set up the Terracotta environment before launching your application. DSO users can use the dso-java script.

Setting Up the Terracotta Environment

To start your application with the Terracotta client using your own scripts, first set the following environment variables:

TC_INSTALL_DIR=<path_to_local_Terracotta_home>
TC_CONFIG_PATH=<path/to/tc-config.xml>

or

TC_CONFIG_PATH=<server_host>:<dso-port>

where <server_host>:<dso-port> points to the running Terracotta server. The specified Terracotta server will push its configuration to the Terracotta client.

If more than one Terracotta server is available, enter them in a comma-separated list:

TC_CONFIG_PATH=<server_host1>:<dso-port>,<server_host2>:<dso-port>

If <server_host1> is unavailable, <server_host2> is used.

If using DSO, complete setting up the Terracotta client's environment by running the following:

UNIX/Linux

[PROMPT] ${TC_INSTALL_DIR}/platform/bin/dso-env.sh -q
[PROMPT] export JAVA_OPTS="$TC_JAVA_OPTS $JAVA_OPTS"

Microsoft Windows

[PROMPT] %TC_INSTALL_DIR%\bin\dso-env.bat -q
[PROMPT] set JAVA_OPTS=%TC_JAVA_OPTS%;%JAVA_OPTS%

Before starting up your application, confirm that the value of JAVA_OPTS is correct.

Terracotta Products

Terracotta products without DSO (also called the "express" installation) can set a configuration path using their own configuration files.

For Enterprise Ehcache and Enterprise Ehcache for Hibernate, use the <terracottaConfig> element in the Ehcache configuration file (ehcache.xml by default):

<terracottaConfig url="localhost:9510" />

For Quartz, use the org.quartz.jobStore.tcConfigUrl property in the Quartz properties file (quartz.properties by default):

org.quartz.jobStore.tcConfigUrl = /myPath/to/tc-config.xml

For Terracotta Web Sessions, use the appropriate elements in web.xml or context.xml (see Web Sessions Installation).

Binding Ports to Interfaces

Normally, the ports you specify for a server in the Terracotta configuration are bound to the interface associated with the host specified for that server. For example, if the server is configured with the IP address "12.345.678.8" (or a hostname with that address), the server's ports are bound to that same interface:

<server host="12.345.678.8" name="Server1">
 ...
 <dso-port>9510</dso-port>
 <jmx-port>9520</jmx-port>
 <l2-group-port>9530</l2-group-port>
</server>

However, in certain situations it may be necessary to specify a different interface for one or more of a server's ports. This is done using the bind attribute, which allows you bind a port to a different interface. For example, a JMX client may only be able connect to a certain interface on a host. The following configuration shows a JMX port bound to an interface different than the host's:

<server host="12.345.678.8" name="Server1">
 ...
 <dso-port>9510</dso-port>
 <jmx-port bind="12.345.678.9">9520</jmx-port>
 <l2-group-port>9530</l2-group-port>
</server>

terracotta.xml (DSO only)

The file terracotta.xml, which contains a fragment of Terracotta configuration, can be part of a Terracotta integration module (TIM). When the TIM is loaded at runtime, terracotta.xml is inserted into the client configuration. For more information on TIMs and terracotta.xml, see the Terracotta Integration Modules Manual.

NOTE: Element Hierarchy
If the use of terracotta.xml introduces duplicate elements into the client configuration, the value of the last element parsed is used. The last element parsed appears last in order in the configuration file.

Which Configuration?

Each server and client must maintain separate log directories. By default, server logs are written to %(user.home)/terracotta/server-logs and client logs to %(user.home)/terracotta/client-logs.

To find out which configuration a server or client is using, search its logs for an INFO message containing the text "Configuration loaded from".