Steps for Kepler conversion to OSGi

there is no particular ordering to these tasks

Separate Jars

          External Jars used in Kepler and Ptolemy need to be rebundled with new or updated manifest files that include properly formatted import-package and export-package headers.  This has been done for all of the jars that Kepler currently uses in the HEAD nightly build (see SVN "external" directory).  However, package versioning was not included in this first pass, only bundles were versioned.  Versioning the packages should be straight forward and could be scripted so that exported package versions reflect the bundle version that has already been assigned.  We may also want to bundle all of the other jars in SVN that we aren’t currently using just to have them on hand.

Create Launchers

            Determine the best and most framework independent way to launch the Kepler OSGi application.  Up to this point I have been running the development Kepler OSGi version from Ganymede and the bundled Kepler OSGi version with the Eclipse executable and Equinox launcher provided for Windows (for explanation of how this zip works see the Equinox QuickStart Guide).  I'm certainly at fault for being stubborn at sticking with Eclipse/windows similarly as David and Sean are stubborn about sticking with Command Line/linux.  So I'm going to try to change that around a bit (but it's just so easy with eclipse!  ;)  So we need to figure out the launching mechanism to use for linux and macosx and create similar zip or tar.gz files for those platforms that work.  There are known issues and workarounds for using the eclipse executable on macosx and linux to launch swing based apps.  Also, we need to figure out how to launch the application using command line java and what the proper configuration file setup is in that environment.  I have only used command line for simple tutorials and not for launching the kepler OSGi version.

Handle Resources

            Any files that are read or written from kepler, ptolemy, or any modules, actors, etc need to be dealt with properly as if they were in a jar file.  To this point developers have sometimes assumed that resources could be found on disk.  My current thoughts on this:

For resources that are read/write add a function in the start method that does the following:

·         checks to see if a directory exists in the plugins directory

o   if it does exist, do nothing

o   if it does not exist, create it and write out the resources to the directory

In some cases the bundle may contain a default file that gets extracted to the bundle’s directory on disk the first time the bundle is started in a particular distribution.  We then use that file for read/write operations.

            OSGi provides functions for searching and reading resources in a bundle.  We could update Kepler code to use these functions for read-only files.

            A lot of the Kepler/Ptolemy code base already accesses files by getting a stream to the file from the classpath.  This seems to be why the current Kepler OSGi version is so functional.  For most of our read-only resources we may only need to make sure that files are in the bundle-classpath for things to work.

            The Kepler/Ptolemy environment variables are also used to access files.  Code that is  doing this needs to be updated to either find the file on disk (if it is read/write) or to find the file in the bundle using the framework (if it is read-only).

Note:  So I believe this is going to be the most difficult activity in converting to OSGi and the most painful for core developers.  I am still a bit foggy on exactly how to accomplish this task and how difficult a task this will be.  It is certainly something we need to look into more.

Native Libraries

            I have not experimented with Native libraries yet.  It seems the way to deal with them is to just put them in what I’ll call an “open bundle”.  This is a bundle that is not jared but just a directory in the plugins directory with the manifest.  Native libraries for different platforms should be organized into separate bundles so they can be easily included or excluded when building distributions.

ObjectManager Update

            The existing objectmanager reads and caches kars and loads classes with a custom classloader.  Currently, actors loaded with the objectmanager cannot access jar files that are loaded with the OSGi framework.  This is just a classpath issue and objectmanager could likely be updated to query the framework for bundles and classes that it needs.  However, this would not solve the dependency issue for actors.  The OSGi framework performs much of the function that objectmanager performs now.  It detects, loads, and caches classes from jar files on startup while supporting lazy loading.  In addition, OSGi configures the classpaths based on the dependency information in the bundle manifest.  So doing a minor upgrade to objectmanager would not gain us much.  We would need to significantly overhaul objectmanager (perhaps out of existence) and modify existing code to use the OSGi framework.  Currently, many of the actors depend on the objectmanager.  Code changes to actors would need to be made to resolve these dependencies.  This should be fairly straightforward.  Since the actors loaded in OSGi have immediate access to other classes, there is no need to make any calls to the framework to access those classes, they are already present and can be imported and used. So this task would be to remove the calls to objectmanager methods, and properly set up the dependencies in the bundle for that actor.  Other functions performed by ObjectManager would have other solutions.  For example LSIDs could be handled by adding an extension point to the kernel.  Each actor would then extend that extension point with their LSID supplied as a property of the extension point (i.e. in the xml, no class implementation is needed).  And other functions of the ObjectManager would likely have similar solutions.

Bundleize Kepler/Ptolemy/Actors

            Two approaches to the physical relocation of packages and resources into bundles.

The first is to slowly break up the monolithic Kepler bundle into smaller and smaller bundles over time.  The advantage here is the more gradual disturbance to the existing codebase/build system.  The disadvantage is that we end up making non-optimal choices while trying to coexist with the current system – for example, the first thing I did when updating the head was add an OSGi system property so I could switch on whether to Initialize the classpath or not (InitClasspath fails in the OSGi framework so right now it gets skipped).  In the branch I just deleted that line of code (and the classes that went with it).  Also, how do you deal with the existing system when you start moving everything around or changing major components like ObjectManager?  It’s going to break pretty quickly unless we put in the effort to figure out how both systems can coexist.  Another issue with the big approach is that we have to redefine dependencies each time a bundle is broken up.  So say we take the one Kepler/Ptolemy/Actor bundle and create three bundles: Kepler bundle, Actor bundle and Ptolemy bundle, we now have to reanalyze the dependencies - removing dependencies from the Kepler/Ptolemy/Actor bundle and adding them to the Ptolemy and actor bundles.  It actually isn’t a huge pain to do this, but it might be a little tedious if we break down the 300+ actors into libraries and then smaller libraries and then single bundle per actor.

Another way to approach this is to design the final bundle architecture up front.  This might look something like: kepler.kernel, ptolemy.kernel, and several bundles for  actors/directors at a TBD granularity (the extreme being one bundle per actor/director but more practically we would group actors by the existing package layout).  Then we start copying things from the current system into the new system.  Get kepler.kernel and ptolemy.kernel working with no actors and then add actors one bundle at a time until we’ve built them all back up.  This is more akin to Tristan’s suggestion a while ago that we start with small bundles and build things up from there.  However, at the time, we were thinking this would actually break the existing system for a prolonged period.  Instead, we could leave the old system in place and build the new system as if it were a branch (but we would never merge the two codesets as a whole but could merge smaller sets of code if they change).  This approach may also make any modifications to resources go quicker.  We would deal with the resources for each bundle as we go.  This would act as a mechanism for keeping track of which parts of the code have already been checked for accessing resources and could help us with making sure pieces of the code have proper tests in place.  We also would not have to rework dependencies as we separate larger bundles into smaller ones.  Dealing with dependencies from smaller to bigger bundles (grouping actors into an actor library) does not require new analysis, you just add the dependencies from each of the smaller bundles into the larger bundle.  And finally, there are no compromises to make for coexisting with the old system.

Other Thoughts…

            There are no doubt a few other things we’ll come across that require attention.  One I can think of right now is how to best handle composite actors and modified actors in the new system.  I don’t know the answer to that but there are no doubt several different ways to deal with it that we’ll want to look into. 

Once we have actors, resources, native libraries, launchers, and versioned jar bundles taken care of we can begin working with Ptolemy to separate out GUIs, directors, etc.  We can create an actor developer kit to help users with generating actors and manifests that contain the required dependencies for those actors.