Skip navigation

Release: 2.4 Previous Releases
Publish Date: October, 2007
Article Rating?

Terracotta for Spring Clustering Tutorial

Purpose

This tutorial shows how to configure a Spring-based Web application for clustering using Terracotta for Spring. By the end of the tutorial, you should be ready to use Terracotta for Spring on your own Spring-based Web applications.

What You'll Need

Required:

  1. Terracotta
  2. Apache Ant

What You Will Learn

At the end of this tutorial, you will have learned three things:

  1. How to use Terracotta for Spring to cluster singleton Spring beans.
  2. How to use Terracotta for Spring to cluster Spring AOP interceptors/advice.
  3. How to make the clustered state available through JMX (using Spring's JMX exporter).

Overview

The basic steps of this tutorial are:

  1. Download and install Terracotta.
  2. The sample problem.
  3. Understanding the sample code.
  4. Build the sample code.
  5. Cluster the application using Terracotta for Spring.
  6. Run the clustered sample application.
  7. Inspect the clustered Spring ApplicationContext object graph. Terracotta enables runtime visibility into live clustered objects.
  8. Start up JConsole, monitor and manage the JMX exported Spring beans.

Getting Started

Steps:

  1. Download and install Terracotta
  2. Download and install Apache Ant if you don't have it installed already. Detailed instructions for installing Ant are in the Apache Ant Manual

The sample application is bundled in the Terracotta distribution. You can find it in the samples/spring/jmx directory. The distribution is also bundling Spring 2.0 and Tomcat 5.5, so the sample Spring Web application will work right out of the box.

The sample problem

Spring's support for AOP and JMX allows for the capture of application runtime information which can then be made available to management tools such as JConsole, the standard JMX browser in the JDK. But in a clustered application you do not get an aggregated view of the application state. You have to manage or monitor each node individually, and either write some home-grown code to do the aggregation or do it manually.

In this tutorial we will see how Terracotta for Spring creates a single access point for monitoring and management of the services in the cluster. 

Understanding the sample code

Counter bean

The application has two singleton Spring beans, named localCounter and clusteredCounter. Each of these beans implement a counter service that has an integer that can be incremented. The beans are also exposed in the Web UI, using Spring MVC, as a link labeled Increment. Both of these Spring beans are instances of the class demo.jmx.Counter.

Here is the code for the Counter bean:

demo.jmx.Counter
public class Counter implements ICounter, BeanNameAware {
  private volatile int counter = 0;
  private String name;

  public synchronized int next() {
    return counter++;
  }

  public synchronized int getCurrent() {
    return counter;
  }

  public String getName() {
    return name;
  }

  public void setBeanName(String name) {
    this.name = name;
  }
}

Monitoring AOP Interceptor

Additionally, these Spring beans are advised using an AOP interceptor that is tracking the number of times that the Counter bean has been incremented per 30 seconds. E.g. implements a simple monitoring service. This interceptor also has a method for resetting the history statistics, something that we will make use of later. The interceptor stores the state in the demo.jmx.HistoryQueue bean, which is simply wrapping a java.util.LinkedList with demo.jmx.HistoryData instances.

Here is the code for the interceptor:

public class CounterHistoryAdvice implements MethodInterceptor {
  private Map queues = new HashMap();

  public void setQueues(Map queues) {
    this.queues = queues;
  }

  /**
   * Advice method for collecting metrics.
   *
   * @see org.aopalliance.intercept.MethodInterceptor#invoke(MethodInvocation invocation)
   */
  public Object invoke(MethodInvocation invocation) throws Throwable {
    String error = null;
    try {
      return invocation.proceed();
    } catch(Throwable t) {
      error = t.toString();
      throw t;
    } finally {
      String name = ((ICounter) invocation.getThis()).getName();
      HistoryQueue historyQueue = (HistoryQueue) queues.get(name);
      if (historyQueue != null) {
        historyQueue.updateHistory(0, error);
      }
    }
  }
}

HistoryQueue bean

Here is the code for the demo.jmx.HistoryQueue bean:

public class HistoryQueue implements IHistory {
  private static final int DEFAULT_INTERVAL = 30;

  private final long granularity;
  private final int capacity;
  private final List buffer = new LinkedList();

  private boolean enabled = true;

  public HistoryQueue() {
    this(DEFAULT_INTERVAL, 24 * 60 * 60 / DEFAULT_INTERVAL);  // default 24h
  }

  public HistoryQueue(int granularity, int capacity) {
    this.granularity = granularity * 1000;
    this.capacity = capacity;
  }

  // JMX access
  public String[] getHistory() {
    synchronized(this.buffer) {
      String[] history = new String[this.buffer.size()];
      for(int i = 0; i < history.length; i++) {
        HistoryData data = (HistoryData) this.buffer.get(i);
        history[i] = data.toString();
      }
      return history;
    }
  }

  ... // rest of code is omitted
}

Spring bean configuration file

Here is an example of the Spring bean config file that is used to define the different services:

<bean id="localCounter"     class="demo.jmx.Counter"/>
<bean id="clusteredCounter" class="demo.jmx.Counter"/>
<bean id="localHistory"     class="demo.jmx.HistoryQueue"/>
<bean id="clusteredHistory" class="demo.jmx.HistoryQueue"/>

All of these components are published to JMX via the Spring framework's JMX exporter.

Architecture overview

Here is a picture of the architecture for the sample application (click on the image to enlarge it):

Build the sample code

This sample application ships built ready to run. If you modify the application code, use the provided build.xml file to re-build the application with Apache Ant (ant build).

Clustering the application using Terracotta for Spring

So far we have only written a regular non-clustered, single-JVM application. But the interesting thing is that in order to turn this implementation into a distributed, multi-JVM application, we do not need to write any code at all. All we need to do is to drop in Terracotta for Spring along with its XML configuration file, in which we simply define which Spring beans we want to cluster.

Understanding the Terracotta for Spring configuration file

Here we can see the Terracotta for Spring config file on the left hand side and the Spring bean config file on the right hand side. As you can see we can declaratively choose which Spring beans to share across the cluster, even if the services are just different instances of the same class.

As you can see, we choose to cluster the clusteredCounter and the clusteredHistory Spring beans. While the localCounter and the localHistory beans are kept local to the each node:

Apart from defining which beans should be clustered, we need to define two more things:

  1. We have to define the path, or the pattern, to the Spring bean definition file, in this example the pattern */applicationContext.xml.
  2. We have to define the name of the Web application, in this example is named tc-jmx (this is usually the name of the war file).

Run the clustered sample application

Running the sample application is a very simple, everything is preconfigured for you:

  1. First you have to start a Terracotta Server from the samples directory.
    • Unix: ./start-demo-server.sh
    • Windows: start-demo-server.bat
  2. Then you have to start the two different Tomcat server nodes from the samples/spring/jmx directory.
    • Unix: ./start-tomcat1.sh& ./start-tomcat2.sh&
    • Windows: start-tomcat1.bat start-tomcat2.bat

Now take your favorite web browser and open the URL http://localhost:8081/jmx/ to see the application running on the first Tomcat node. If you now click on the two different Increment links (one for the local service and one for the clustered), you will see something like this (click on the image to enlarge it):

As you can see in the screen shot above, we have now updated both the clustered and the local counter for the application running on port 8081. If you now open up the URL http://localhost:8082/jmx/ you will see the second Tomcat instance running the same application on port 8082 (click on the image to enlarge it):

As you can see in this screen shot, the state for the clustered counter is consistent across the different JVM and Tomcat instances, while the local counter is still zero. If you now update the clustered and local counter on this node (8082), then when you switch back to the first node (8081) and click Refresh you will see that the clustered counter still remains consistent while the local counters are incremented independently.

Configure it yourself

If you want to configure everything yourself, or enable Terracotta for Spring for your custom application, you would have to modified the Tomcat startup script. There are two ways you can do this:

Use Terracotta wrapper script

Terracotta for Spring ships with a startup script that should be used as a drop-in replacement to the regular java command. The name of the script is dso-java and can be found in the bin directory in the distribution. To use the recommended Terracotta startup script:

  1. Find the (dso-java) script in the bin directory.
  2. Replace the regular invocation to java with the invocation to dso-java.

Use JVM options

You can also use additional JVM options for applications inside the web container, perform the following:

  1. Prepend the Terracotta Boot Jar to the Java bootclasspath.
  2. Define the path to the Terracotta configuration file.

Here is an example (assuming you are running Windows):

java -Xbootclasspath/p:<path to terracotta boot jar> \
     -Dtc.config=path/to/your/tc-config.xml \
     -Dtc.install-root=<path to terracotta install dir> \
     ... <your regular options> ...

Runtime visibility: Inspect the clustered Spring ApplicationContext object graph

Terracotta for Spring enables runtime visibility into live clustered objects (as well as transaction rates, cache activity etc.). In order to make use that, you first have to start up the Terracotta Administrator Console, this is done by executing the bin/tc-admin.bat script. If you do that now then you will see a window with a tree structure similar to the one below (click on the image to enlarge it):

Now, expand the Roots tree and select the tc:spring_context element. This is a real-time view of the clustered Spring ApplicationContext's object graphs (click on the image to enlarge it):

If you now expand the tree then you will be able to see all the Spring beans that are clustered, both its name and its data (click on the image to enlarge it):

To refresh the object graph view, click on the element or select it and press F5 on your keyboard. You can also right-click on an element to see a context menu.

Try changing the state of the Spring beans by clicking on the Increment button in the Web UI, see how the data changes in the Spring beans

Start up JConsole, monitor and manage the JMX exported Spring beans

From the JDK home directory, launch JConsole and connect to one of the local JVM instance(s) - it does not matter which one since the state is clustered and therefore available on all nodes (JVMs).

Launch JConsole:

  • Unix: bin/jconsole
  • Windows: bin\jconsole.exe

The first window that you will see is a dialog that asks you to connect to a JMX Agent. Here it does not matter which agent (JVM) you choose. The reason for this is that the state is clustered, so you will get the same view of the state regardless of which node you are looking at. The same thing holds for operations; a JMX exported operation made on a shared Spring bean will have a cluster-wide effect (click on the image to enlarge it):

In the main window, select the MBeans tab and then expand the jmx node in the tree on the left hand side. Now you will be able to see the exported Spring beans, the two clustered ones (ClusteredCounter and ClusteredHistory) as well as the two local ones (LocalCounter and LocalHistory). If you now choose the ClusteredCounter, you will see its current value (which is the same value as we have seen in both the Web UI as well as in the Terracotta Administrator Console) (click on the image to enlarge it):

Now you have seen how you can browse monitoring data in your cluster, but what about managing the services (Spring beans)? Well, it turns out that Spring JMXcan export operations on the beans as well, and if they are invoked on clustered beans, then they will have a cluster-wide effect. For example, if you select the ClusteredHistory bean and then click on the Operations tab, then you will see all the operations (methods on the bean that are exposed through JMX). Now, if you click on the reset operation then you will reset the whole clustered history to zero (click on the image to enlarge it):

If you now go back to the Web browser and click on Refresh, you will see that the operation had the expected effect.
Adaptavist Theme Builder Powered by Atlassian Confluence