Modified items

All recently modified items, latest first.

Re: Interfaces to Provenance for REAP by Shawn Bowers, last updated: Nov 14, 2008 10:04 AM: That sounds great, thanks Derik. I've noticed a log of "bugzilla" activity concerning provenance. The idea of having consistent terminology I think is very important (brought up in one of the bugzilla "threads"). There has already been a good deal of provenance-related activities within Kepler as well as within other workflow systems, and various terms (including the term "provenance" itself) are often used in different ways. As part of the provenance interest group, it might also be useful to create a page of basic provenance-related terms and how they are, or should be, used within Kepler. Does this sound useful? Should we create this page? --shawn

Interfaces to Provenance for REAP by Derik Barseghian, last updated: Nov 12, 2008 06:56 PM: As requested during a REAP call, I've started a page about the Provenance related work that's underway. I've also checked in some bugs for some of the tasks.

Re: The Repository Structure by Timothy McPhillips, last updated: Nov 05, 2008 07:36 PM: I have a proposal that I hope will make all this a bit simpler.   First, I'd like to point out that I think we all may be talking past each other because we have different ideas about what a 'module' represents.  To me, a module is a unit of compilation.  Everything in a particular version of a module is meant to be compatible with everything else in that module.  However, the latest (trunk) version of one module generally cannot be assumed to be compatible with the trunk of some other module.  The trunk of the ppod-actors modules might be based on the 1.2 tag of the comad module (rather than the trunk of comad), for example, and for very good reasons.  Modules of this sort are thus very loosely coupled, and it is not necessarily that useful to download all the trunks of every module (in this sense of the word), because they generally won't be guaranteed to work with each other, and many of their dependencies will not satisfied.    The above is the concept of modules that the new build system is based upon. And I think we very much need this kind of module and the support for developing such modules that the new build system provides.  It is critical for the projects I represent.   An alternative concept of 'modules' is that they represent conceptual chunks of software that are developed in concert such that the latest version (trunk) of each module is guaranteed to work with (or depend on) the trunk of all other modules.  Checking out all of the trunks at one time in this case can be very useful (as can tagging all of these modules with the same tag).  This, I think, is an appropriate way of managing the core of Kepler in the repository, the part of Kepler that is shipped in every standard release.  The core is also the base system that developers adding new capabilities either to the core, or via a non-core module, would start with.    Fortunately, I think we can easily accomodate both kinds of modules in our repository and build system.  Let's say we use the word 'module' (and a directory named 'modules' in the repository) to stand for the first concept:  a unit of compilation and internal consistency.  We then put the entire core of Kepler, or base system, inside just *one* of those modules.  Inside that module we are of course free to be as modular as we please (I'd prefer using the Java package scheme rather than multiple source trees for this kind of modularity, but I could be persuaded).  Checking out the trunk of the 'core' module would thus grab all of the code that comprises the Kepler base system.  This would take only a single svn command.  Checking out a tagged version of the core module would similarly check out all of the code for that version of the Kepler base system.   Does this distinction between these two concepts of modules make sense?  If so, does the current approach of putting all modules (in the first sense) at the same level of the repository (where they are now), with their own trunk, tags, and branches directory in each, also make sense?   The core module would have a single trunk, and this would be all that is needed when building the Kepler base system.    Now, what if another module at some point needs to be added to the base system?  Simple--we just do an svn copy of the appropriate version of this module's source code and resources into the right places in the core module.  I'm hoping that the comad module can be made a part of the Kepler base system someday, and why shouldn't its source files be merged into the source tree for the base system at that point?  The comad classes are all in the org.kepler package hierarchy anyway.  Once comad becomes a part of the base system it *will* need to be kept in sync with rest of the base system, so it really does belongs with the files in the core module (again, in the first sense of a 'module').     I hope this doesn't count as another novella.  If there is any confusion at all about what I'm saying, let's have a telecon soon to talk this through.

Re: The Repository Structure by Matthew Jones, last updated: Nov 05, 2008 06:46 PM: I want to pipe in here as well.  I had a hard time digesting David's initial post in this thread -- it was an incredibly long rationalization, and therefore I didn't really even know how to start.  I have no desire to write my own novella here, so instead I'll just make two points: 1) Having trunk/branches/tags at the root of the repository allows us to check out the head with no further tool support, and yet also allows the build system to pick specific modules or branches/tags of modules based on configuration information.  This is the approach advocated in the SVN book, and I think it will be familiar to most SVN users.  I see no downside to this approach, and several simplicity advantages.  Having a large collection of branches and tags in one directory is not an issue, as a good naming system will work as well or better than additional directory hierarchy without the annoyances that the additional directories introduce. 2) The current problems in understanding the repository structure are due to lack of information about project history, not due to the inherent organization.  I can make similar arguments that David did about how confusing the current modules list is because of poor naming choices (e.g., what is the difference between vanilla-1.0, and vanilla-trunk, and why does vanilla-1.0/trunk exist, and what is an ustan-master module, ad infinitum).  The problem is that we don't have metadata about what these modules are in the source code system.  This is the same as the previous tags in CVS.  I have no problem with the hundreds of v_### tags simply because I know they were generated by Cruise Control as part of our previous nightly build system.  People with a shorter time horizon on Kepler might have a harder time understanding these artifacts.  I think the solution in the new system is to have a consistent module naming system that is a) predictable to be able to classify various types of tags and branches, and b) associated with some more descriptive metadata that explains the content and purpose of a particular module or branch or tag.  For this reason, I think the branch and tag naming scheme Chad proposed is definitely a major improvement over our current system or the current modules proposal. Looking forward to movement on these items, Matt

Re: The Repository Structure by Chad Berkley, last updated: Nov 05, 2008 01:40 PM: Hi David, Sorry it took so long for me to respond to this post.  It got lost in my RSS feeds.   I agree with much of what you say and disagree with parts.  I'll try to summarize my thoughts on your post here.   -multiple module structures I agree that modules should all follow the same structure.  I don't see the utility in having multiple organization strategies.  That having been said, I think there are advantages of the various nesting techniques and maybe we should list the benefits and downsides of each method.   First method would put the branches/tags/trunk directories under the module directory: -kepler --build ---branches ---tags ---trunk --modules ---core ----branches ----tags ----trunk   Benefits:  -centralized storage of a given modules files.   -All parts of a single module can be checked out simultaneously   Downsides: -can't check out all modules with a single svn command -requires a scripted build system to checkout modules   The other method is to put the branches/tags/trunk directories higher up in the tree. -kepler --trunk ---build.xml ---build-files ----build-common-targets.xml ----build.properties ---modules ----core ----ppod --branches ---build-1.0 ---modules ----core-1.0 ----ppod-1.0 ---tags ----core-release-1.0 ----build-release-1.0   Benefits: -can check out all modules with one svn co command -the head of each module is centrally located -does not require the build system to checkout modules   Downsides: -branches/tags directory could have lots of sub-dirs (but a good naming convention will eliminate any problems here)   We're going to have lots of modules anyway, so we're going to have to figure out a naming convention to make it less confusing.  This naming convention should also apply to branches and tags so that we don't end up with the similarly-named, confusing structure in the current repository.  I propose that each module in the trunk have a root name that is agreed upon when the module is created, and that all branches use a hyphen and version number, like this: trunk/modules/core tags/modules/core-release-1.0.0 tags/modules/core-release-1.0.1 tags/modules/core-checkpoint-1 tags/modules/core-checkpoint-chadexp branches/modules/core-1.0.x branches/modules/core-1.1.x branches/modules/core-semtools   I'd like to mention that the old repository structure evolved organically and made more sense in CVS (from which it was imported) where you didn't actually see the tags and branches as directories.   I agree that the naming in the branches and tags directories is confusing at best, but this is more an issue of developers being pretty loose with their naming conventions -- the current 'modules' directory already suffers from this problem too.  I don't think, however, that this is the way the new repository will be if we allow tags/branches of different modules to exist in a single directory -- we just need to enforce naming conventions.  I should point out that many of the confusing tags/branches in the current repository were auto generated by automated build systems of the past.   I think the fact that the 2nd repository structure allows one to check out all of the code with one checkout is a major advantage.  It will make nightly builds much easier with the NMI system and allow those of us who need it the ability to have all of the kepler code checked out in one place.  I don't see any disadvantages to doing it this way if we enforce a strict naming convention.

review and revise the new web content by Matthew Jones, last updated: Nov 05, 2008 12:03 PM: Hi, As we discussed yesterday, here are the web pages for editing on the website, divided among those who volunteered to help. It'll be great to get a clean version of the site released so that we can remove the JSPwiki site.  Let's discuss these edits and changes in the User Rep forum. Thanks, Matt Mark ------- https://dev.kepler-project.org/users/the-kepler-project https://dev.kepler-project.org/users/downloads https://dev.kepler-project.org/users/projects-using-kepler https://dev.kepler-project.org/developers/reference/using-commons-logging https://dev.kepler-project.org/developers/reference/making-changes-to-the-kepler-wiki-account-and-password-information (probably can be deleted) https://dev.kepler-project.org/developers/reference/what-happens-when-kepler-starts-up Matt ------ https://dev.kepler-project.org/users/about-us https://dev.kepler-project.org/users/contact-us https://dev.kepler-project.org/developers/get-involved https://dev.kepler-project.org/developers/get-kepler-code https://dev.kepler-project.org/developers/reference/downloading-and-building-the-kepler-source-code https://dev.kepler-project.org/developers/reference/software-development-guidelines Tim ------ https://dev.kepler-project.org/users/features https://dev.kepler-project.org/users/sample-workflows https://dev.kepler-project.org/users/faq https://dev.kepler-project.org/developers/faq https://dev.kepler-project.org/developers/reference/adding-a-new-java-actor-to-kepler-a-quick-tutorial https://dev.kepler-project.org/developers/reference/adding-new-classes-jars-to-kepler https://dev.kepler-project.org/developers/reference/ptolemy-package-dependencies Ilkay ------ https://dev.kepler-project.org/users/documentation https://dev.kepler-project.org/users/publications https://dev.kepler-project.org/users/support https://dev.kepler-project.org/developers/reference/kepler-and-eclipse https://dev.kepler-project.org/developers/reference/executing-kepler-from-the-command-line https://dev.kepler-project.org/developers/reference/accessing-hsql https://dev.kepler-project.org/developers/reference/kepler-installer-checklist

Kepler web site change requests by Timothy McPhillips, last updated: Nov 04, 2008 04:49 PM: Suggestions for new content for the Users are of the Kepler web site, requests for changes to current content, and ideas for news items.

Provenance Interest Group by Shawn Bowers, last updated: Nov 04, 2008 12:35 PM: All things related to recording, storing, managing, querying, and use provenance information associated to scientific workflows

Developer Forum by Development Account, last updated: Nov 04, 2008 12:34 PM: A forum for the various contributers to Kepler teams, interest groups, and task groups to discuss issues in Kepler design and development.

Build System Ideas -- Rough Draft by Chad Berkley, last updated: Nov 03, 2008 12:01 PM

Re: The Repository Structure by Timothy McPhillips, last updated: Oct 30, 2008 12:21 PM: Interesting.  One thing we're running into here is the limitation intrinsic in trying to represent information in nested folder structure when the information doesn't cleanly nest.  As you point out, the experimental/stable/standard classification is in some cases nearly orthogonal to the project-specific/community/core classification. It is difficult to represent both using a single set of nested folders.  However, I don't think that means we should reject representing one classification or the other this way.  The question may be, who has more use for each classification scheme?  Or perhaps one can be nested in the other?  Maybe only the Kepler base system developers need the first classification?  The term "Experimental" makes sense from the point of view of deciding what should be included in the next release of the Kepler base system, and what needs more work before distributing it as a standard component.  At the same time, I agree that many modules will be very stable, useful, and well-supported even if they are never "standard" and never shipped as part of the Kepler base system.  So classifying such modules as "Experimental" indeed would be misleading. I think I proposed the idea of organizing branches and tags for different kinds of modules differently, and I agree with the limitations and disadvantages you point out.

The Repository Structure by David Welker, last updated: Oct 30, 2008 12:17 PM: I wanted to get this discussion started so we can begin to explore the pros and cons of various means of storing different sorts of modules in the repository. This starting point will be rather long, so I am going to number the sections according to the topic discussed.    1. Naming of Module Categories First, there has been some discussion of physically separating modules based on their classification. The three initial classifications that have been proposed are, "standard," "stable" and "experimental." As an initial matter, I do not think these are the best names for these categories. In particular, the term "experimental" perhaps implies that the code is not reliable and is not suitable to be used in real world projects. Yet, this is the area where we will be typically developing code for our end users who will then make use of that code in real world projects. Also, it should be kept in mind that there is no reason to think that in some cases, the code in the "experimental" section is not in some way equal to or even superior to code in other classifications. I think that we should use a more neutral term that does not have any tendency to disparage the value of code developed in that category. So, here is my proposal. I think we should have three or four categories. Instead of "standard" we should have "core" or alternatively we should have both "core" and "standard." For now, since we do not have anything approximating the idea of a "standard extension" I think we should just have "core" and explore adding a "standard" category later, if and when we decide the it is useful to designate modules as such. Instead of "stable" I think we should call it "community." Code here is not necessarily more stable than code in the "experimental" area, which may in fact be quite mature and have a limited set of real world users dependent on it. I think the main idea is not just that the code in this area is stable, the idea is that the maintainers of code in this area are able and willing to back that code up and provide wider support for it. Clearly, in order to provide this sort of support, it would be necessary for a developer or development team to have a stable tag or branch. However, that same development group might have other branches and tags that are in development, and are unstable. It seems strange to me to have any code whatsoever in the "stable" area that is not in fact stable. But that is precisely what you will have, as actual development is likely to occur here as well, perhaps in various "experimental" branches. I think that overall, we already have directory structures that are intended to indicate the stability of code. Code in the "trunk" or in a "branch" can be expected to change. Code in a "tag" should be stable and probably should never change. I do not think we need any more indicators than this, although we could conceivably want to add the concept of a "stable branch" to differentiate branches where active development occurs from branches were changes are made more conservatively. Regardless, the indication of stability should be in the trunk, tags, branches area, not at a higher level of the tree where it is sure to be misleading. The term "community" in contrast is not misleading. This is code that is intended to be shared. There should be some stable code here, preferably in a tag. But, there might also be some unstable code that you should not be using without close collaboration with the developer. Coming up with an alternative name to "experimental" is more challenging, but I would propose as an initial matter the term "project-specific." Here, we have modules that are developed with a particular individual or team in mind, and which is not ready (and may never be ready) to be shared with the wider community. There might be very stable code here, and there also might be more experimental code here. Overall, I think using the terms "stable" and "experimental" would be both misleading (because there would be stable code in the experimental area, and experimental code in the stable area) and redundant (we already have the concept of trunk, tags, and branches to indicate stability). I think it would be preferable to use the abstract terms "community" and "project-specific" because they more accurately convey what is intended to be stored in each category and also because the term "experimental" is simply too disparaging for code that will in fact be depended on by specific individuals and teams for their important, reliable, and serious scientific investigations.   2. Migrating Modules There is an issue regarding migration of modules. It may be desirable from time-to-time to change the classification of a module. A "project-specific" module may have much functionality that a developer thinks could be widely applicable, and they want to share it with the wider community. They are also able and willing to commit resources to supporting the community in the use of that module. As such, they would like to change the classification of the module from "project-specific" to "community." Alternatively, one can imagine a case of a module that was classified as "community" but which simply did not take off. The maintainer of that module may no longer have the resources to provide support for the wider community. They thus may wish to reclassify the module from "community" to "project-specific." What are some of the technical challenges involved in changing the classification of a module? Well, because the configuration of which modules are meant to be used together is distributed in order to allow for decentralized development, moving an entire module from one category to another is not a good idea. When a module is reclassified, each and every suite or module group that uses that module will have to be updated to reflect that change. Given that a module group could even exist outside of our own repository, such changes have the potential to be very disruptive. Since not all suites or module groups are necessarily stored in our repository, there is no way for an individual who decides to reclassify a module by physically moving it to know exactly who will be impacted. There are various hacks that we might use to get around this. Instead of modules referring to specific locations, we could have them refer to a centralized module registry which tracks the location. This would be a huge headache and would add unnecessary bugs and complication to the system. It would compromise the more decentralized structure that we have now, as any new modules would be required to register themselves in our centralized module registry. Overall, in my view, this or any other solution that relies on some sort of centralized registry of some sort is a bad idea. I think we need to ask ourselves, exactly what are we buying by having "community" and "project-specific" (or "stable" and "experimental") modules stored in different areas in the repository? My answer is that we are not buying ourselves much of anything. This is nothing more than a particularly expensive means of documentation. Only, when you decide to update the documentation (i.e. reclassify a module), it actually has the potential to physically break the entire system and cause no end of headaches. A simple alternative is to simply have a listing of non-core modules that we consider to be "community" modules. Or better yet, I propose that we do away with the distinction altogether! The line between a project-specific module and a community module is rather fuzzy. Probably most people who develop modules would not mind contributing to the community, but they also probably have limited and varying resources regarding the extent to which they are able and willing to support that module's use by others. What we have in essence is a continuous variable (i.e. intention, willingness, and ability to provide support for a module) classified in a stark and simplistic binary scheme (either "community" versus "project-specific" or "stable" versus "experimental") that distorts as much as it enlightens. Not only that, we are to develop either a complex centralized registry or break peoples' code in order to provide this sort of documentation? I would classify this firmly in the bad idea category. (I don't mean to be binary here. I am not saying that there are only bad ideas and good idea, or that all bad ideas are equally bad!) I don't think we should necessarily have this classification of modules at all. But, if we do, may I suggest that we keep it to pen and paper rather than in the repository structure???   3. The Use of Multiple Strategies in the Structuring of Modules It has been proposed that different modules be structured according to multiple standards in the repository. In particular, it has been proposed that all modules of a certain category be stored in only three folders. As a simple example, consider the case where you only have 4 modules, A, B, C and D. The proposal is to store code associated with those modules in the following manner: trunk/moduleA trunk/moduleB trunk/moduleC trunk/moduleD tags/moduleA-tag1 tags/foo-moduleA-tag2 tags/moduleA-tag3 tags/moduleA-tag4 tags/moduleB-tag1 tags/beta-moduleB-tag2 tags/moduleB-tag3 tags/moduleB-tag4 tags/moduleC-tag1 tags/alpha-moduleC-tag2 tags/moduleC-tag3 tags/moduleC-tag4 tags/moduleD-tag1 tags/bar-moduleD-tag2 tags/moduleD-tag3 tags/moduleD-tag4 branches/moduleA-branch1 branches/moduleA-branch2 branches/moduleA-branch3 branches/baz-moduleA-branch4 branches/moduleB-branch1 branches/moduleB-branch2 branches/foo-moduleB-branch3 branches/moduleB-branch4 branches/moduleC-branch1 branches/moduleC-branch2 and so on... Now, you may notice in this simplified example, there are only 4 modules, A, B, C, and D. Yet, these three directories are already getting rather large. And even a bit confusing. It sort of reminds me of the somewhat confusing (to me at least, and I know I am not the only one) structure that already exists in tags and branches under kepler at the following locations: https://code.kepler-project.org/code/kepler/branches https://code.kepler-project.org/code/kepler/tags Go ahead, have a look... I will be waiting for you when you get back. Are you confused yet? Lets see, lets look at a couple of the branches: KEPLER_1_0_0_BRANCH/ RELEASE-BRANCH-1-0-0/ Which branch do you think is associated with Kepler 1.0 that was released? Well, I suppose after thinking about it, you say, well, it was a release, so, I bet that RELEASE-BRANCH-1-0-0 is the one! Besides, I much prefer hyphens rather than underscores to separate numbers and words, so it must be that one! And you would be right. Okay, you may have guessed that one (I had a much harder time than you). But how about this. Consider the following tags: RELEASE-1-0-0-beforeJarRemoval/ RELEASE-BRANCH-1-0-0-beforeJarRemoval/ RELEASE-TAG-1-0-0/ KEPLER_1_0_0_BRANCH_MERGE1/ POST_MERGE1_KEPLER_1_0_0_BRANCH/ v1_0_129/ v1_0_170/ v1_0_174/ Now, if you are anything like me, you are thinking. What the heck are all these tags named BRANCH? Is it a tag, or is it a branch? Well, which of these tags do you think is associated with the 1.0 release of Kepler? You probably guessed RELEASE-TAG-1-0-0. And you would be right! However, since I am not as smart as you, I will admit that I wasn't able to figure it out before I asked around. Oh, one other really interesting thing. The tag RELEASE-TAG-1-0-0 is exactly the same as the branch RELEASE-BRANCH-1-0-0. Why is the exact same thing stored as both a tag and a branch? It is a mystery to me to. Why in the tag does release come first, but in the branch, it comes second? Another mystery. Oh, and what the heck does v1_0_129 and v1_0_170 and all the rest of the 100 or so similar tags refer to? I think I knew once, but I don't remember. I do know they were supposed to be deleted, but never were. I consider them eye candy! It is nice to live in a world with so many mysteries. Anyway, why spend all this time reviewing the old repository structure. We are supposed to be forward thinking here. Its off to a new (and hopefully better) repository structure in the future! Well, not so fast. This mess in the old repository is the result of branching and tagging only one piece of software. Namely, Kepler. But, when we break-up Kepler into little parts called modules, and each of those modules has its own branches, tags, and trunks, when we store all those branches, tags, and trunks in the same three folders, this old repository structure will seem like a pleasant fantasy of wondrous simplicity! This is, after all, nothing more than the branches and tags from one code base, not several as has been proposed. Well, I have a few more criticisms for this proposal. First and foremost, we really do not need to have two different ways of storing modules. This is just going to cause unnecessary confusion for new developers. Why are some modules stored this way, and others stored that way, they will ask. Second, this deviates from the SVN standard. You are supposed to have the name of the software, then trunk/  then straight to the code. You are not supposed to have anything between trunk and the code. It is not supposed to be kepler/trunk/foo-module/<code> its supposed to be kepler/trunk/<code>. Moving away from the standard will add unnecessary confusion for new developers who are already familiar with the standard. What about when I want to rename a module? That use case is going to be a huge mess too. What is all this buying us exactly? Oh, right. It is allowing us to check out the trunk of all the core modules with one svn command. That way, we don't have to depend on the build to do it. I have a couple of issue with that. First, the build is simply superior at doing checkout than a single SVN command from the command-line. Imagine this scenario. Your internet connection fails during your checkout of all the core modules. With the svn command-line, you have to start all over (I believe). With the build system, one or more modules have probably been downloaded and added to the build system module registry. The build system is smart enough not to try to retrieve this code again. Second, like it or not, we need to make a commitment here! We don't trust the build system or any other system to do checkout, but its okay for the vast majority of non-core modules to depend on that "unreliable" (actually, more reliable) system? I don't think so! We need to eat our own dog food, as it were, and use the same build system for the core that we promote for other non-core developers. Third, you check out the core using SVN instead of the build system. Then what? You are going to be working with a bunch of rogue modules that aren't registered in the build system module registry. Not a good idea. Fourth, while the build system and other systems do not currently explicitly depend on the repository structure, it would be nice if, in the future, we keep open at least the option of depending on the repository structure. When you decide to go with a multiplicity of standards for storing modules in the same system, it adds a whole level of new complexity to any code that would like to make use of the standard used to store modules in the repository. Why don't we commit to a single way of storing modules? Overall, I am again going to have to classify this idea that we should have multiple standards for storing modules as a very bad idea. What are we buying here? SVN checkouts of the core with a single command? Write a darn script for goodness sake if you want that for your own use. Don't corrupt the repository and add unnecessary complexity to our development efforts which we will have to live with for a very long time because you don't trust the build system. If we don't trust the build system for the core, perhaps we shouldn't use it at all! We are going to promote this build system to the community, but it is not good enough for us? I don't buy it, especially since the build system is actually a better way to do checkouts anyway, because it is more robust in cases of network failure. Regardless, we shouldn't be adding huge amounts of unnecessary complexity, both from a mental perspective (new developers saying, what is going on with multiple ways of storing modules in the repository and what the heck is going on with all of these branches in the same directory and why aren't we following the SVN standard) and technically (we have to deal with the complexity of storing modules in two different ways in the repository and renaming a module - when for example, you decide to break one module into two - is a nightmare).   4. My Proposal for a Repository Structure This is my proposal: modules/                 core/                           core-module1/                                                   trunk/                                                   tags/                                                   branches/                           core-module2/                                                    trunk/                                                    tags/                                                    branches/                            etc...                 ppod/                              ppod-suite/                                                    trunk/                                                    tags/                                                    branches/                               ppod-actors/                                                    trunk/                                                    tags/                                                    branches/                                                                                prov-gui/                 comad/                               comad-suite/                                                      trunk/                                                      tags/                                                      branches/                               comad/                   First, let me describe some basics about this proposal. We have a core area, so that core modules can be stored together. For each suite, (previously known as a master module), we have folder. For example, ppod/. In that folder there is a folder called ppod-suite/ which contains the meta-data specifying which modules are meant to work together. Finally, there are additional folders containing all the modules that are unique to that suite. Just by looking at the repository, you can get an idea of who is responsible for developing a module. You know that ppod-actors is maintained by whatever people are in charge of ppod. Whereas the comad module, which ppod also depends on, is probably maintained by the people in charge of developing comad-suite. In this particular case, those people happen to be one and the same. But they might not be. Let me describe the characteristics of this proposal that address the objections I raised to alternatives above. Here, there is no distinction between "stable" and "experimental" modules or between "community" and "project-specific" modules. Instead, modules might consist of stable and experimental parts. There is no added complication arising from having to change the classification of a module according to some artificial scheme. Any such artificial scheme that exists is stored in documentation elsewhere (including perhaps in a standard location in the modules themselves, so anyone curious about the classification of a particular module could click on a standard file and see.) There is no multiplicity of conventions utilized for storing a module. Once a developer understands how one module is stored, that developer understand how all modules are stored. It follows the SVN standard, so developers familiar with that standard are not confused. Tools are also able to, when and if necessary, easily rely on one standard convention for how modules are stored. You do not mix the branches of different modules into the same folder, which is nothing more than a recipe for chaos and confusion. Instead, this is a very simple and straightforward structure that allows modules to be grouped so as to manage the complexity of the modules directory, but does not sacrifice simplicity and understandability in doing so. I look forward to hearing the thoughts and opinions of others!

Build System Requirements by Chad Berkley, last updated: Oct 29, 2008 02:38 PM

Re: Build System Requirements by Timothy McPhillips, last updated: Oct 29, 2008 02:38 PM: I don't think it is necessary to standardize directory structures across Kepler and Ptolemy such that Ptolemy looks like a Kepler module.  The Kepler repository and build system are meant for development of Kepler and extensions to it, and I wouldn't expect developers to develop new features in Ptolemy using this system.  So loose coupling to Ptolemy should be fine, I think. What are your thoughts? Previously Daniel Crawl wrote: Does #2 apply to Ptolemy?

Re: Build System Requirements by Daniel Crawl, last updated: Oct 29, 2008 01:49 PM: Does #2 apply to Ptolemy?

Re: Build System Requirements by Timothy McPhillips, last updated: Oct 29, 2008 12:04 PM: Yesterday, a few of us chatted on the phone about some of the ideas mentioned above.  Here's a summary of some of the things we discussed: 1.  Chad is experimenting with splitting the source code and resources at the trunk of Kepler into two modules, a core module and a "non-core" module.  (Chad, I assume one thing you want to demonstrate is that the core module has no dependencies at compile time on the non-core module?) 2.  We would like the internal directory structure of modules in the repository to be standardized across the core module and all extensions, standard and otherwise.  The build system will then be able to treat all modules uniformly (and will not need to restructure directories on the developer's machine in order to achieve this uniformity).  David is going to make the next revision to the directory structure proposed at https://dev.kepler-project.org/developers/teams/build/systems/build-system/build-system-ideas . 3. We agreed that the directory structure within modules should be flatter than that currently used by the modules in https://code.kepler-project.org/code/kepler/modules/ .  (We no longer see a significant advantage in adopting the Maven standard, for example.) There should be clear places to put Java source code, third-party jars, native libraries and executables, source code for native libraries and executables, unit tests, workflow tests, and resources including icons for actors. 4. There could be advantages in making it clear in the source code repository what modules represent the core of Kepler and those standard extension modules that would ship by default with a new release of Kepler.  It could make it easier for developers to know what modules to check out when working with or extending the standard Kepler base system.  And it could simplify applying access control to these modules. 5.  Currently, the modules in the repository can be tagged and branched independently.  In general this is very useful.  We are currently using the svn standard of using directories named 'trunk', 'tags', and 'branches' to identify the trunk for a module, and to group the tags and branches for it; and grouping these three directories at the same level within each module.  An alternative would be to put all of the trunks for modules together in a single directory such that they can be checked out together via a single svn command (without getting any of the tags or branches), and put the branches and tags elsewhere.  (One disadvantage of this would be that tags and branches would have to be named to indicate what module they are tags or branches of, and this could lead to difficulties when modules are renamed.) 6.  The new build system is flexible with respect to the locations of particular modules in the repository.  They do not necessarily need to be organized uniformly with respect to each other--modules managed by the build system do not even need to be stored in the central repository.  There may be good reasons to group modules by topic area or contributing project.  At the same time, more standardization of the locations of things in the repository, where to find branches and tags for modules, etc, would make it easier for everyone to see what is going on and to share their developments with others.

Chartering the web UI interest group by Timothy McPhillips, last updated: Oct 29, 2008 11:26 AM

Re: Chartering the web UI interest group by Timothy McPhillips, last updated: Oct 29, 2008 11:26 AM: I agree.  Let's move it over to the Interest Group area.

Re: Chartering the web UI interest group by Matthew Jones, last updated: Oct 28, 2008 12:59 PM: It seems this group charter is fairly stable and well defined.  I propose it be moved out of the incubation area and into the Interest Group area.  Any objections?

Architecture team roadmap by Timothy McPhillips, last updated: Oct 28, 2008 12:55 PM

« Previous 20 items Next 20 items » 1 … 28 29 30 31 32 33 34 … 46

Sections

Personal tools

Modified items