the problem we're solving with multiple class loaders

I think it is worthwhile trying to be very specific about what we are trying to achieve when proposing a framework for loading Kepler modules in distinct Java class loaders.  

First, a problem we are not trying to solve.  We are not trying to make it possible for two modules to load two different versions of the same class and exchange instances of these classes, while pretending that they are in fact the same.  As far as I know this simply cannot be done in Java.  If two modules need different versions of a class then these modules are islands with respect to each other as regards this class.  They cannot use instances of the class created in the other module as if they were instances of the locally loaded class.

What problem then are we trying to solve?  We are trying to make it possible for two modules to use different versions of a class when instances of these classes are *not* going to be shared between the two modules.  A common situation would be one where the Kepler kernel uses one version of a jar, and a Kepler extension uses another version of the jar.  Another situation is when two extensions need to use different versions of a jar.

A second problem we are trying to solve is that of being explicit about what libraries a module depends on, and easily distributing the required libraries with the module.  An easy way to do this is to have each module include in their respective lib directories the jars they require.  But what we don't want is a situation in which two modules include the same version of a jar and yet, because the two modules loaded the jar using different class loaders, cannot exchange instances of classes defined in the jar.  Instead, one wants each jar to be loaded only once regardless of how many modules depend on it and include it in their lib directories. 

Thus what we want is to be able to load modules into distinct class loaders, while allowing them to exchange objects when the underlying classes are the same version.

If this is indeed the problem we are trying to solve, then we currently have a solution in hand.  David Welker last week extended the class loader approach Sean Riddle was working on earlier in the year to work in conjunction with the new extension-build system.  Any module can be configured to run in its own class loader (by default all modules load in a single class loader), each version of a class or jar is loaded only once, and all modules employing the same version of a class can exchange instances of that class.

This approach could be readily enhanced to work as a run-time Kepler Loader on user machines as well.

Please correct any misconceptions I may have about these problems and solutions!

I think your description of the problem we are trying to solve is reasonable.  Its also good to hear that David was successful with his quick prototype.  You didn't provide enough details for me to evaluate the solution -- this is similar to the proposed new build system.  There has been some work, and I think something is working, but there haven't been any announcements to kepler-dev or the forums or a README describing what was done, or how to use it, so its very hard to evaluate. Maybe David could chime in and describe what he actually did, and describe what someone else should do to use the system, either through a post here or as a link to a readme in SVN somewhere.  It also sounds like there would be more work to get to a runtime system, right?

We'll start to put up documentation of all this on the Plone site today.  David is preparing an overview of how the class loader solution works in the context of the build system and a description of what would need to be done to use the approach in a deployed system. 

For the build system we'll also be posting an updated description of how to create and develop extensions, including how to request that a particular module be loaded in its own class loader.  General instructions for using the new build system for development on the Kepler trunk will come later because it does not yet support all the capabilities of current system.

Please make sure to put this documentation in the Build and Release Team folder.  Looking forward to seeing what David and Sean have been up to.  They seem to be quite far down the development path.

Thanks!

Previously Timothy McPhillips wrote:

We'll start to put up documentation of all this on the Plone site today.  David is preparing an overview of how the class loader solution works in the context of the build system and a description of what would need to be done to use the approach in a deployed system. 

For the build system we'll also be posting an updated description of how to create and develop extensions, including how to request that a particular module be loaded in its own class loader.  General instructions for using the new build system for development on the Kepler trunk will come later because it does not yet support all the capabilities of current system.