Setting up a Tomcat Web Cluster

Introduction

This guide is intended to step you through the process of installing and deploying your web application in a Terracotta Sessions cluster.

The architecture of a Terracotta-enabled, high-availability web application looks like this:

The load balancer parcels out HTTP requests from the Internet to each application server. Ideally, the load balancer would use HTTP session affinity to route all requests that corresponded to the same HTTP session to the same application server to maximize the locality of reference of the clustered HTTP session data, although with a Terracotta-enabled web application, any application server can process any request.

The application servers run your web application and the Terracotta client software. More application servers may be deployed as needed to handle additional request load, but, for simplicity, this guide will assume there are three active application servers.

The Terracotta server acts as the data store for HTTP session data and coordinates access by the application servers to that session data.

Basic Steps

Process of getting this system installed, configured, and running is as follows:

download relevant software, including Terracotta, Java, and the application server
setup and install software on the cluster
install software
prepare a load balancer
Install Terracotta configuration on the server
Configure the application server to run with Terracotta enabled
Start the Terracotta server
Start the application server instances
Start the load balancer
Test

Download

Install

Install Java on all machines
Install Terracotta on all machines
Install Tomcat on all of the application server machines
Install the web application on all of the application server machines

Generating a Boot Jar

You may need to generate a boot jar on the application server machines for the version of the JVM that you are using on your application servers. That can be done using the make-boot-jar.sh script in the Terracotta kit. It will also be created for you automatically as necessary if you use the dso-env.sh script as discussed in the next section.

For more information on what the boot jar is and how to create it, see the following:

XXX: There's no documentation on the make-dso-bootjar script. Need to add it and xref it.
Concept and Architecture Guide

Configuring Terracotta to Work With Your Application

If you already have a working Terracotta configuration for your application, copy the configuration file to the Terracotta server machine.

If you don't yet have a Terracotta configuration for your application, you can build one by hand or you can use the Terracotta Sessions Configurator to help you build one.

See the following for more information on configuring Terracotta for your application:

Sessions Quick Start
Sessions Configurator Reference Guide
Configuration Guide and Reference

Start the Terracotta Server

Once the Terracotta configuration file is on the Terracotta server machine, start the Terracotta server on the Terracotta server machine.

To start the Terracotta server, use the start-tc-server.sh script. You should see output similar to this:

[orion@tcserver ~]$ terracotta/bin/start-tc-server.sh -f tc-config.xml &
[1] 13545
[orion@tcserver ~]$ 2007-06-07 18:25:57,850 INFO - Terracotta version 2.3.0,
 as of 20070427-110443 (Revision 2739 by cruise@rh4mo0 from 2.3)
2007-06-07 18:25:59,875 INFO - Configuration loaded from the file at '/home/orion/tc-config.xml'.
2007-06-07 18:26:00,387 INFO - Log file: '/home/orion/logs/server-logs/terracotta-server.log'.
2007-06-07 18:26:01,962 INFO - JMX Server started. Authentication OFF -
 Available at URL[service:jmx:rmi:///jndi/rmi://localhost:9520/jmxrmi]
2007-06-07 18:26:03,706 INFO - Terracotta Server has started up as
 ACTIVE node on port 9510 successfully, and is now ready for work.

[orion@tcserver ~]$

See the Tools Guide for more information on the start-tc-server script.

Configure Tomcat

Configure Tomcat to run with Terracotta enabled. This is as simple as adding some elements to the JAVA_OPTS environment variable. The easiest way to do this is to execute the dso-env script and add the output to the JAVA_OPTS environment variable prior to calling any of the Tomcat startup scripts. Here's a shell script snippet that you can either put directly into catalina.sh or write as a standalone environment script:

TC_INSTALL_DIR=<path to terracotta installation>
TC_CONFIG_PATH="<terracotta server hostname>:<terracotta port (default: 9510)"
. ${TC_INSTALL_DIR}/bin/dso-env.sh -q
export JAVA_OPTS="$TC_JAVA_OPTS $JAVA_OPTS"

Here's a sample script with actual values plugged in:

[orion@domU-12-31-35-00-31-63 ~]$ cat set-tc-env.sh 
TC_INSTALL_DIR=/home/orion/terracotta
TC_CONFIG_PATH="tcserver:9510"
. ${TC_INSTALL_DIR}/bin/dso-env.sh -q
export JAVA_OPTS="$TC_JAVA_OPTS"

Here's a sample of how to use it:

[orion@domU-12-31-35-00-31-63 ~]$ . set-tc-env.sh

Here's what the JAVA_OPTS environment variable looks like after the script is run:

[orion@domU-12-31-35-00-31-63 ~]$ echo $JAVA_OPTS
-Xbootclasspath/p:/home/orion/terracotta/lib/dso-boot/dso-boot-hotspot_linux_150_12.jar
 -Dtc.install-root=/home/orion/terracotta -Dtc.config=tcserver:9510

Start the Tomcat Servers

Once the JAVA_OPTS environment variable is set properly, you can use either the startup.sh script or the catalina.sh script to start Tomcat.

Here's an example of the entire process, using the sample environment script described above.

[orion@domU-12-31-35-00-31-63 ~]$ . set-tc-env.sh 
[orion@domU-12-31-35-00-31-63 ~]$ echo $JAVA_OPTS
-Xbootclasspath/p:/home/orion/terracotta/lib/dso-boot/dso-boot-hotspot_linux_150_12.jar
 -Dtc.install-root=/home/orion/terracotta -Dtc.config=tcserver:9510
[orion@domU-12-31-35-00-31-63 ~]$ tomcat/bin/catalina.sh start
Using CATALINA_BASE:   /home/orion/tomcat
Using CATALINA_HOME:   /home/orion/tomcat
Using CATALINA_TMPDIR: /home/orion/tomcat/temp
Using JRE_HOME:       /home/orion/java
[orion@domU-12-31-35-00-31-63 ~]$

If Terracotta is set up correctly, you should see something like this in the catalina.out log file:

2007-06-07 18:29:23,208 INFO - Terracotta version 2.3.0, as of 20070427-110443 (Revision 2739 by cruise
@rh4mo0 from 2.3)
2007-06-07 18:29:24,451 INFO - Configuration loaded from the server at 'tcserver:9510'.

If you don't see the terracotta INFO lines, you probably don't have the JAVA_OPTS environment variable set or passed in to Tomcat propertly.

Start Tomcat in this way on all of the application server machines.

Configure and Start the Load Balancer

In a production environment, you should use a production quality hardware or software load balancer. For testing purposes, Terracotta comes with a simple TCP proxy that may be used like a load balancer. You should not use this TCP proxy in a production environment.

Here's a sample script to start the TCP proxy:

TC_INSTALL_DIR=<path to terracotta installation>

exec "${JAVA_HOME}/bin/java" \
  -Dtc.install-root="${TC_INSTALL_DIR}" -Ddaemon=true \
  ${JAVA_OPTS} \\
  -cp "${TC_INSTALL_DIR}/lib/tc.jar" \
   com.tc.net.proxy.TCPProxy <port to listen to> <web host 1>:<port>,<web host 2>:<port>,<web host 3>:<port>

Here's a version of the script with the terracotta installation directory and the web hosts filled in:

[orion@loadbalancer ~]$ cat bin/start-loadbalancer.sh 
TC_INSTALL_DIR=/home/orion/terracotta

exec "${JAVA_HOME}/bin/java" \
  -Dtc.install-root="${TC_INSTALL_DIR}" -Ddaemon=true \
  ${JAVA_OPTS} \
  -cp "${TC_INSTALL_DIR}/lib/tc.jar" \
   com.tc.net.proxy.TCPProxy 8080 web01:8080,web02:8080,web03:8080

To run the TCP proxy, execute the script on the load balancer machine:

[orion@loadbalancer ~]$ bin/start-loadbalancer.sh  
Thu Jun 07 18:56:02 EDT 2007: Starting listener on port 8080, proxying to [web01/10.255.54.145:8080],
 [web02/10.255.54.97:8080], [web03/10.255.54.146:8080] with 0ms delay

Functional Testing

Load Testing