Sunday, December 7, 2008

JPPF and GridGain. Two Java computational grids.

It took me a month to get back to my blog and today I'd like to talk about JPPF (Java Parallel Processing Framework). As usually I will compare it with GridGain just because GridGain to my personal understanding is the best one in some aspects.
Last couple of years JPPF grew up very intensively and brought some new features that made it flexible and robust.

Differences in architecture.

In general we can say that GridGain has only one "layer". Every node ("master" and "worker") can execute tasks either from another node or from itself. Whenever you start GridGain from your code or as standalone application or as a service integrated into the application server you should know that you start new node and to avoid any calculation started on this node one should make some configuration changes. Normally it should be node attribute that says to topology SPI not to include this node into the calculation.
On one hand this is very flexible because you don't need to change your code or start additional services to involve this node into calculation, but on the other hand it messes concept up for the newcomers who usually expect some clients and servers (just because of the common multi-tier approach).
Another consequence of this approach is a peer-to-peer architecture. All nodes are connected to each other. Obviously this leads to the nodes number limitation (because of the network traffic). I know that GridGain jumped on this issue and is going to solve it quickly.


Unlike the GridGain, JPPF divides the framework into the client, driver and executor parts. Client layer provides an API and communication tools to use the framework to submit tasks, to execute in parallel. Service layer (driver) is responsible for the communication between with the clients and the nodes, along with the management of the execution queue, the load-balancing and recovery features, and the dynamic loading of both framework and application classes onto the appropriate nodes. And execution layer is the node - it executes individual tasks, return the execution results, and dynamically request, from the JPPF driver, the code they need to execute the client tasks.
This approach simplifies the initial understanding and in some cases makes "master-worker" implementation less complicated.
Another benefit of such division is that it overcomes the limitation of the maximum number of nodes. Only servers are connected to each other, not nodes and this segmentation allows having a lot of nodes with a few connected servers. On the other hand if you need simple 5-10 nodes grid, server could be potentially a single-point-of-failure.


Features and features.

Both products are very featured and cover a lot of edge cases. Let me mention some of them:
  1. On demand class loading. To my understanding (just because I suggested this feature and was responsible for the implementation ;) ) GridGain was the first one who supported transparent class loading between nodes. JPPF supports it as well.
  2. Load balancing. Both frameworks support it, but GridGain gives you about 5-7 strategies out-of-the-box.
  3. J2EE integrations. Both products can be integrated into the application servers, but the integration ways are different. GridGain starts either as a service (JBoss, WebLogic, WebSphere) and uses application server resources (executor service, logs, etc) or as a servlet (Tomcat, GlassFish). JPPF registers in JNDI tree and provides its functionality as a standard J2EE component (JCA). As far as I know GridGain has a ticket to support JNDI lookup as well.
  4. Both frameworks are task/job based.
  5. Both frameworks have annotation based execution (@Gridify in GridGain and @JPPFRunnable in JPPF).
  6. JFFP has DataProvider to exchange data between tasks/nodes. GridGain gives distributed TaskSession for that.

Despite of some common features they have a lot of differences:

  1. Communication. JPPF is a TCP/Multicast based approach. GridGain supports various protocols (TCP, JGroups, Mule, JMS, Mail, JBoss and so on).
  2. Extension. JPPF is much more closed then GridGain. Last gives you SPI interfaces and one can extend or write new functionality and integrate it into the GridGain.
  3. Monitoring. While GridGain is still writing their cool monitoring console, JPPF already provided one and as far as I see it's pretty good.
  4. Node information. I did not find any node attributes in JPPF which is not very good because it's very useful when you send your tasks/jobs into the grid. Very often you need to control, which node should execute this particular task. Simple example is executing task on Linux nodes just because it loads some native libraries or uses node specific resources. GridGain supports node attributes and even custom (user defined) ones.
  5. Tasks rescheduling in case of overload. JPPF keeps tasks on server side (note that execution layer is not the same as server one) and all nodes execute tasks as soon as they have free resources for that. This is great but what if one server got overloaded and another one has nothing to do? Nothing in case of JPPF. GridGain will redistribute work (taking into account user wishes) if job stealing is on.

Coding.

Coding approach is more or less similar with some differences.

In JPPF one should start client and submit set of tasks into the grid like this:

JPPF Code

JPPFClient client = new JPPFClient();

List<JPPFTask> tasks = new ArrayList<JPPFTask>();

tasks.add(new HelloTask());

try {
// execute the tasks
List<JPPFTask> results = client.submit(tasks, null);
} catch (Exception e) {
e.printStackTrace();
}
JPPFJob interface can be used as a tasks container and provide some additional functionality. But anyway it's quite far away from Map/Reduce way.

GridGain requires a kind of Map/Reduce implementation and force you to understand this concept. Your task should implement method "map" where you split it into small jobs and "reduce" to collect results from all jobs and reduce it into one task result. but from execution standpoint they look alike.

GridGain Code

Grid grid = GridFactory.start();

try {
// Execute Hello World task.
GridTaskFuture<Integer> future = grid.execute(GridHelloWorldTask.class,
task_param);
}
finally {
GridFactory.stop(true);
}

Conclusion.

Both frameworks are very friendly and reliable. They are simple and flexible. But in this particular case I would say that GridGain wins just because of simple common Map/Reduce approach and its SPIs that provide incredible flexibility.

2 comments:

Laurent Cohen said...

Hi Dennis,

Great article! Thank you for taking the time to write it.

I would just like to bring one correction. You mentioned: "... I did not find any node attributes in JPPF ...".
Actually those exist since JPPF 1.3, along with the possibility to submit a job with an execution policy. It is documented in details in the JPPF Tasks and Execution Policy section of the documentation.

Thanks again,
-Laurent

dkharlamov said...

Hi Laurent,
Thanks for the correction. Yes, you are right. I was looking at the nodes API instead of policy.
Thanks
Dennis.