Working Distributed Features

STATUS

The content on this page is outdated. The page is archived for reference only. For more information about current work, please contact the Distributed Execution Interest Group.

Overview

This page is an attempt at rough documentation for the currently implemented distribution functionality in Kepler. Use these methods with care as they have only been tested by the developers (Lucas and Chad) and we can't vouch for anything working with whatever current version of Kepler you're using. For best results, check out the newest version of Kepler from CVS. None of these features will work in the last release of Kepler. This documentation is a work-in-progress as of 2007.10.15.

Installation

If this is your first time using the distributed workflows, take a look at Installing a Distributed Workflow.

Building a distributed workflow

Building a distributed workflow in Kepler is done much like building a regular workflow. The main difference is that any part of the workflow that you would like to be distributed over multiple CPUs must be created inside a DistributedCompositeActor (DCA). You can find the DCA by searching for it in the actor library. Drag a DCA out onto the canvas, then right click on it and choose "Open Actor" to add components. Examples can be found in the kepler/workflows/distributed directory. At this time, only SDF directors have been tested within a DCA. At this time the DCA must contain an SDF director.

A note about terminology: In this document, we refer to "Slave" and "Master" controllers, or more simply "the slave" and "the master". The master is always the machine from which you will start the execution of the workflow. The slave is any remote machine (1 or more) that will accept incoming connections from the master and execute all or part of your workflow. You should have shell access to both the master and any slaves that you wish to run and you'll need to make sure the RMI port 1099 is open if you have a firewall.

Distributed Kepler on a cluster

If you have access to the NCEAS ROCKS computing cluster (or another cluster), Kepler will run pretty easily on the cluster nodes. Take a look at Distributed Computing with Kepler on the NCEAS ROCKS Cluster.

Work in progress

This is all work in progress and any bug reports, corrections, critiques, etc are welcome. Please either open bugs in bugzilla or send email to kepler-dev@ecoinformatics.org. Thanks.