Project name

Open Data Grid

Overview (one or two sentences)

The Open Data Grid is a POJO-based Data Grid implementation that allows you to utilize the Data Grid with nothing but plain POJO-based Java (JDK 1.5).

Description (detailed)

The Open Data Grid basically implements the Master/Worker pattern wrapped up in the CommonJ WorkManager interfaces. But it adds much more functionality in terms of reliability and stability. For example there are implementations that handles routing, work or a worker failure, high volumes of data, ordering etc.

The implementation is completely based on JDK 1.5 and POJOs. This allows the user to utilize the Data Grid with nothing but plain POJO-based Java (JDK 1.5) and also has the benefits of keeping the core implementation open and simple, e.g. easily customizable. It is implemented as a single-JVM, multithreaded implementation (that is usable as is) that optimized for turning into a distributed multi-JVM implementation using Terracotta.

The implementation is based on the java.util.concurrent abstractions (ExecutorService, LinkedBlockingQueue etc.) and the user API mainly consists of the CommonJ WorkManager. Here is an example of what the user would have to do in order to use the data grid:

// Create the work queue manager with 'String' as routing ID type. The DefaultWorkQueueManager 
// is usually sufficient, but you can easily provide your own implementation of the 
// WorkQueueManager interface
WorkQueueManager workQueueManager = new DefaultWorkQueueManager();

// create the router - there are a bunch of default routers in the Router interface, 
// such as SingleQueueRouter, RoundRobinRouter and LoadBalancingRouter, but
// it is probably here that you need to plug in your custom implementaion 
Router router = new Router.LoadBalancingRouter(workQueueManager);   
    
// create the work manager - this implementation is very generic and most likely enough
// for you needs
WorkManager workManager = new RoutingAwareWorkManager(router);

// optionally (null is ok) create a work listener that will get a call back each time 
// the status of a WorkEvent (Work) has changed - allows you to for example take proper 
// action upon failure and retry etc. 
WorkListener workListener = null;

Set workSet = ... // create a set with the work to be done

// loop over the work set
for (Work work: workSet) {
  RoutableWorkItem workItem; // the work item, wrapping the work (result) and holds the status 
  try {
    // schedule the work, e.g. add it to the work queue and get the work item in return that can
    // be used to keep track of the pending work
    workItem = workManager.schedule(work, workListener); 
  } catch (WorkException e) {
    // handle failure
  }
}

The project will also implement to some common use cases, such as for example a distributed web spider.

License

Project license: BSD

Committers (name and email)

Project Lead:

Jonas Bonér - jonas@terracottatech.com

Team members:

Taylor Gautier - tgautier@terracottatech.com

Infrastructure requirements

• SVN Repository: yes
• Wiki space: yes
• Mailing lists
  • User: yes
  • Dev: yes
  • SCM: yes
• FishEye: yes
• Issue Tracking: yes
• Downloads: yes
• Backup/recovery: yes