build system usability issues

I was wondering how to update modules in the new build system. In particular, lets say that I am using "vanilla-trunk", and notice that a few of the modules used in vanilla-trunk have been updated. Is there a way using a single svn or ant command to easily update these modules? Or, does each module have to be updated manually (by cd'ing into the module dir and running 'svn update', e.g.)?  It might be nice to have an ant command for this that updates all the modules in the active module list/suite.

Hi Shawn,

I just added a new ant target to do this. Now typing in "ant update" will run svn update on all the modules listed in modules.txt.

David, I just tried the ant update task and it seemed to work well.  Doing a refresh in subversive recognized that the update had happened.  I notice that the reported revision numbers from the ant task are the latest revision number for the entire repository and not for the individual modules.  Not a big deal but it threw me off for a sec because the latest revision reported in subversive is specific to the module (for example the loader module is only at revision 16235 at the moment).  Also, Shawn, if you are using a graphical SVN client you can select all the projects and do an update that way as well.

pdate:
[update-modules] Using command line interface
[update-modules] Svn : Updating
[update-modules] up -r HEAD /Users/aaron/kepler.modules/loader --non-interactive
[update-modules] At revision 16300.
[update-modules] Svn : Updating

Another, build system usability issue I'd like to bring up is the usage of svn:ignore for all build generated files.  It is somewhat difficult to determine exactly what has changed locally with extra build generated files laying around.  However, it looks like a lot of care was taken to keep it to a minimum (target directory, .classpath, .project for each module)  Perhaps this could be automated somehow.  I have already added target, .classpath, .project to svn:ignore for the util module by hand.

This brings to mind another question, where do application generated files go?  Does each module have a folder in the .kepler directory?  Is there a specific folder in the module directory (marked with svn:ignore) that is meant for writing by the module?  How is this being handled?  Is it documented somewhere that I missed?

Thanks!

Hi Aaron,

The reason that this has not been addressed is because with svn, any files that are not added using svn add will be ignored when you do a check-in. You don't have to explicitly use svn:ignore, you simply don't add them and they will be ignored. That is the reason you don't see a bunch of .project and .classpath files and for that matter .iml files (IDEA equivalents) in the repository in the absence of support for svn:ignore.

Ah, OK, in subversive when I do a commit at the top level, loader for example, I get loader/target, loader/.classpath, and loader/.project all with check boxes next to them for adding to version control and then I have to uncheck them so they don't get added.

The mysql jar is no longer in the repository. How did this happen? Have other jars been removed?

Hi David,

Has there been any decision on how to reintegrate the initializations that we deleted earlier this week? They are important since they provide backwards-compatibility for several actors and are required to run some REAP workflows.

What about splitting Kepler.java into build system and non-build system components?

Hi Daniel,

I think the easiest thing to do would probably be to copy whatever classes are needed by these initializations into loader.

The issue with splitting Kepler.java into two parts is that the part that is split off (containing your initiazations) would still need to be invoked somehow. One approach would to provide some sort of class that contains a no-op function, but then overwrite that class in another module which is contained in whatever distribution needs those initializations.

David

I was curious as to what suites were available for changing to from the build system and I found them listed in build-area/build-settings/module-locations.txt

Easy enough, but my first reaction was to look for an ant target that listed all of the available suites.  Could we add a target for that?

Hi David,

I don't think copying these classes into the loader module is a good policy:

1. In this case, we'd have to copy DateToken and all of its dependencies. The loader module is not the appropriate location for these classes since they are unrelated to the new build system. In the future, there may be other classes that require this kind of initialization and this policy leads to lots of unrelated classes in the loader. Additionally, suppose the next version of DateToken needs to be initialized; how will this handle multiple versions?

2. Kepler 1.0 does not have DateToken. If it is copied into the loader module, then the new build system can longer provide a vanilla 1.0.

An interesting related question is: can the new build system currently provide a vanilla 1.0? If the loader module is used, then I don't see how it can...

Anyway, here's how I would split Kepler.java:

1. Everything except the build system initializations would go into the core module.

2. The build system initializations stay and after they execute, call org.kepler.loader.Kepler.main(). Both core and 1.0 have this method.

What do you think?

Previously David Welker wrote:

Hi Daniel,

I think the easiest thing to do would probably be to copy whatever classes are needed by these initializations into loader.

The issue with splitting Kepler.java into two parts is that the part that is split off (containing your initiazations) would still need to be invoked somehow. One approach would to provide some sort of class that contains a no-op function, but then overwrite that class in another module which is contained in whatever distribution needs those initializations.

David

Hi Daniel,

I think your proposal is a good one. You can invoke your initializations from org.kepler.loader.Kepler.main().

I think the general problem you have identified through your additions to Kepler might still be useful to discuss. I was thinking of calling a build team meeting. At this point though, I am going to hold off because I think your solution solves this problem for the time being and it may not pop up again.

David

OK, I've split Kepler.java. I've tested it with both kepler-trunk and kepler-1.0. Let me know if you have problems.

I noticed that 'ant update' does not update build-area/, so be sure to update that directory to get all my changes.

Previously David Welker wrote:

Hi Daniel,

I think your proposal is a good one. You can invoke your initializations from org.kepler.loader.Kepler.main().

I think the general problem you have identified through your additions to Kepler might still be useful to discuss. I was thinking of calling a build team meeting. At this point though, I am going to hold off because I think your solution solves this problem for the time being and it may not pop up again.

David

Hi David,

Could you check in ReportDependencies.java? I'm getting the following error when running ant:

build-area/build.xml:5: The following error occurred while executing this line:
build-area/build-settings/taskdefs.xml:114: typedef class org.kepler.core.ant.ReportDependencies cannot be found

Also, could we svn:ignore build-area/modules.txt? It changes frequently and usually is a different suite.

thanks

Hi Dan,

I have in fact checked in that file. Also, I have fixed the modules.txt problem by deleting it. Now, the value of modules.default will be used if modules.txt does not exist.

David

Great, thanks.

By the way, the build system is not setting java.lib.path so none of the actors using JNI will run. Are you working on this? If not, I'll fix it.

Previously David Welker wrote:

Hi Dan,

I have in fact checked in that file. Also, I have fixed the modules.txt problem by deleting it. Now, the value of modules.default will be used if modules.txt does not exist.

David

I would like to propose a new ant target for the build system.

Currently Kepler excludes many packages and folders from Ptolemy.  However, when someone makes changes in Ptolemy and those changes introduce dependencies on classes that are excluded in Kepler then our excludes list needs to be updated (as is the case at the moment). Updating the Ptolemy excludes list is difficult to manually maintain especially since there are 4 copies of it in the build system, one for the ant build and three for the supported IDEs.  A simple algorithm could be implemented in an ant target that would greatly simplify this process.

Here are the steps of the proposed algorithm:

1. Read in the existing master ant build exclusion list
   
2. Open and read in every java file in Ptolemy that is not excluded in the master exclusion list
   
3. Check each import statement to see if there are any dependencies on excluded packages or classes, if there is an exclusion dependency add the current class to the exclusion list
   
4. Recurse steps 2 and 3 until there are no more dependencies on excluded packages or classes
5. print out the new exclusions to the screen so they can be manually examined and pasted into the master exclusions list

.

Hi Dan,

If you don't mind fixing it, feel free to do so.

David

Great, thanks.

By the way, the build system is not setting java.lib.path so none of the actors using JNI will run. Are you working on this? If not, I'll fix it.

 

Previously David Welker wrote:

Hi Dan,

I have in fact checked in that file. Also, I have fixed the modules.txt problem by deleting it. Now, the value of modules.default will be used if modules.txt does not exist.

David

 

Hi, I was wondering if there was a way to specify a revision for the ant update target in the build system?

Maybe something like:

ant update -Drevision=18179

Aaron

So it would update all of your active modules to the same revision?  That sounds great!

Tim