Kepler Documentation Requirements

STATUS:

The content on this page is outdated. The page is archived for reference only. For more information about current work, please contact the User Representation Team.

Overview

Kepler software documentation requirements, user manual details, and action items.

Kepler Software Documentation Requirements

The system should focus on two types of documentation:
*User documentation
*Developer documentation
The user documentation should ideally consist of the following material:
1. Getting Started with Kepler Guide: A tutorial-style manual that explains
  *what scientific workflows are,
  *a brief (short) overview of Kepler entities (a.k.a. Vergil, actors, directors, utilities),
  *how to search and run existing Kepler workflows,
  *how to build workflows in Kepler,
  * and, a series of tutorial workflows that are well-documented.
2. Documentation on the Existing Kepler Workflows: These are 1-2 page technical notes on the major Kepler workflows. They should contain 1-2 paragraphs on the specific science problem that this workflow solves, and then how this problem was solved using the Kepler system. It finally should mention how to run this workflow, give some example inputs and generated results, and explain the future work on this workflow if it is now finalized.
  1. This documentation needs to be developed together with a scientist who uses the workflow.
  2. This detailed documentation on using the workflow should be part of the workflow itself so that it can be viewed in the workflow or by clicking a button in the workflow. Perhaps this is just a link back to the technical notes referred to abov
3. User Manual:This is a detailed description of the Kepler GUI, actors, directors and utilities. It needs to explain the existing actor and domains, also needs to give tips on their usage.
4. Cookbook on Workflow Patterns: This is a "HOW TO Manual" on different technical problems. Examples to topics are iterations, conditionals, user Interactions, some workflow patterns like the 'stage->execute->fetch' pattern for high-throughput computing.
5. Actor Manual Pages: User-centric descriptions of the actors, auto-generated from an XML documentation file. See the [Actor Manual Pages Section].
The developers documentation should contain the following material:
1. Javadocs: Automatically generated documentation for the code base. Needs to be extended using custom doclet and taglets.
2. Design Documentation
3. Coding Style Document
4. Tutorials and Documentation on How To Develop New Kepler Components
The documentation system should support exposing appropriate documentation in a context sensitive way, for example from the tree of actors in the left pane one should be able to see a tooltip explaining the actor in a brief sentence and get a niceley rendered full printout of the actor manual page.
The usage of the Documentation Enhancer (http://www.alphaworks.ibm.com/tech/docenhancer) (for dev. docs) will be assesed in design. It offers some nice features including semantically analysing your code.
The usage of a text-based file format is required for the efficiency and easy maintanance of the documentation.
Possible formats are Latex, HTML, XML, Docbook. These formats can partially be automatically generated by an application. This will be discussed in detail below.
An application that goes through the updated code-base and checks if the related documentation is updated is required.
Kepler collaboration needs an automated documentation checklist for contributing members. Everytime somebody check in new code they need to update the checklist online on what is there for the documentation.
*This can be implemented as a simple CGI application with a set of checkboxes, or a more complicated software engineering framework can be adapted.
*Possible solutions to this should be discussed in the framework design below.
Kepler needs a checklist for how the installers should behave. Please refere to the Installer Checklist Page for the most recent checklist.
Automatically Generating Test Cases: We need an interface (maybe workflow ;) that will instantiate an actor, that will ask example input values for each input port and parameter, and create a workflow instance that will just test that actor and add this actor into the test workflows list.
A logging utility is required. Please refer to the black-box logger design page to contribute to the design of the black box logger.
A description for how the example workflows should look like is required. This will be a simple text description like the title should be on the top of the page and the color and font for it.

User Manual Details

Below is a draft outline for a detailed user manual. The sections and subsections given are subject to change. Please send comments to Kepler-Dev or edit them below this documents as comments under the contributors section..

KEPLER USER's GUIDE

Getting Started with Kepler
1. Getting and Installing Kepler:
  This section should give information on how to download Kepler, the system requirements, the installation instructions for all platforms.
2. Running the Examples:
  Info on quick links and what they do. More info on how to reach the rest of the Kepler workflows. Choose one or two workflow and explain in detail.
3. Basic Kepler Entities:
  Very basic information on actors, directors and Vergil interface to make the user undersatnd what is happening in the examples.
4. HelloWorld in Kepler:
  Instructions to make the user generate HelloWorld workflow using Kepler.

[Actor Manual Pages]

Kepler needs a system to fully document all user-oriented details about actors so that the actor can be effectively deployed in user-created workflows. This documentation will take the form of "Actor Manual Pages" which are extended MoML docuemnts.

The Actor Manual Pages should be user-centric not developer centric. They should describe, in terms familiar to our target scientists, 1) what the actor does, 2) how it does it (algorithm), and 3) what its inputs and outputs are. The documentation system will support annotating (1) and (3) in semantic terms and will support (3) in structural terms as well. For example, a given input port may be a floating point value (structural), but may represent rainfall in millimeters (semantic). The semantic annotations will draw from one or more ontologies for their terminology. This should be annotated when the actor is registered and be reflected to the documentation automatically.

These XML documents will be used to generate HTML and other output formats that are needed to display the manual pages on the web and within the Kepler application. We originally discussed putting this documentation in the Javadoc code for the actors, but agreed that this would be difficult to maintain because we expect that most of this documentation will be written and maintained by non-developers. Also, modifying the source for every documentation change creates versioning problems.

The manual pages together wil support creation of an actor cheat sheet, to more quickly identify which actor does what. The "brief" description section in the more extended actor manual pages will be automatically extracted to generate the cheat sheet.

Categorization of actors in terms of their functionality/behavior and their I/O signatures will be accomplished by annotating the actors with terms drawn from one or more ontologies. This will replace the category taglet that was previously used in the Javadoc proposal. The ontology of categories needs to be more fully developed and refined, but the basic framework is in place. Semantic categories are referenced based on their LSID, so the translation of these manual pages to user-readable forms will need to resolve those LSIDs into human readable category names.

The proposed XML format for the documentation is provided here:

<?xml version="1.0"?>
<entity name="Constant" class="ptolemy.kernel.ComponentEntity">

  <!-- The unique identifier for the actor.  This is an LSID and must be unique, although it can
       be a revision of a previous version of the actor by changing only the version field of the LSID. -->
  <property name="entityId"  value="urn:lsid:localhost:actor:1:1" class="org.kepler.moml.NamedObjId"/>

  <!-- The Java class and its unique LSID that is requireed to execute this actor. -->
  <property name="class" value="ptolemy.actor.lib.Const" class="ptolemy.kernel.util.StringAttribute">
    <property name="id" value="urn:lsid:localhost:class:877:1" class="ptolemy.kernel.util.StringAttribute"/>
  </property>

  <!-- Documentation regarding the actor's overall functionality. -->
  <property name="documentation" class="org.kepler.moml.DocumentationAttribute">
    <description><brief>Produces a constant value on each fire cycle.</brief>  The constant value is set by the user as
                 a parameter of the actor, or defaults to an integer value of 1 if unset.
    </description>
    <algorithm>Examines the &quot;value&quot; parameter and emits it on the ouput port during each fire event.</algorithm>
    <typicalUsage>Used to parameterize other models that take constant values as inputs.</typicalUsage>
    <author>Yuhong Xiong</author>
    <author>Edward A. Lee</author>
    <project>Ptolemy II</project>
  </property>

  <!-- Component's overall semantic type -->
  <property name="semanticType" value="urn:lsid:localhost:ontology:1:1#Class1" class="org.kepler.sms.SemanticType"/>

  <!-- Each port is listed with its properties and documentation -->
  <property name="output1" class="org.kepler.moml.PortAttribute">
    <property name="direction" value="output" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="dataType" value="unknown" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="isMultiport" value="false" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="documentation" class="org.kepler.moml.DocumentationAttribute">
      <description>The constant value is emitted on this port.  The type of the values are the same
                   as the type of the &quot;value&quot; parameter.
      </description>
    </property>
    <property name="semanticType" value="urn:lsid:localhost:ontology:1:1#Class2" class="org.kepler.sms.SemanticType"/>
  </property>

  <property name="output2" class="org.kepler.moml.PortAttribute">
    <property name="direction" value="output" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="dataType" value="unknown" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="isMultiport" value="false" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="documentation" class="org.kepler.moml.DocumentationAttribute">
      <description>The constant value is emitted on this port.  The type of the values are the same
                   as the type of the &quot;value&quot; parameter.
      </description>
    </property>
    <property name="semanticType" value="urn:lsid:localhost:ontology:1:1#Class3" class="org.kepler.sms.SemanticType"/>
  </property>

  <property name="output3" class="org.kepler.moml.PortAttribute">
    <property name="direction" value="output" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="dataType" value="unknown" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="isMultiport" value="false" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="documentation" class="org.kepler.moml.DocumentationAttribute">
      <description>The constant value is emitted on this port.  The type of the values are the same
                   as the type of the &quot;value&quot; parameter.
      </description>
    </property>
    <property name="semanticType" value="urn:lsid:localhost:ontology:1:1#Class4" class="org.kepler.sms.SemanticType"/>
  </property>

  <property name="trigger" class="org.kepler.moml.PortAttribute">
    <property name="direction" value="input" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="dataType" value="unknown" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="isMultiport" value="true" class="ptolemy.kernel.util.StringAttribute"/>
    <property name="documentation" class="org.kepler.moml.DocumentationAttribute">
      <description>Does this port actually exist for the Constant actor?
      </description>
    </property>
    <property name="semanticType" value="urn:lsid:localhost:ontology:1:1#Class5" class="org.kepler.sms.SemanticType"/>
  </property>

  <!-- Each Parameter is listed too along with its documentation. -->
  <property name="value" class="org.kepler.moml.ParameterAttribute">
    <property name="documentation" class="org.kepler.moml.DocumentationAttribute">
      <description>Contains the constant value which is produced on the output port.
      </description>
      <defaultValue>1</defaultValue>
    </property>
    <property name="semanticType" value="urn:lsid:localhost:ontology:1:1#Class6" class="org.kepler.sms.SemanticType"/>
  </property>

  <!-- Creates a Port Group vport1 = (output1, output2) and assigns a semantic type to it, i.e., Class6(vport1) -->
  <property name=“vport1” class=“org.kepler.sms.KeplerIOPortGeneralization”>
    <property name=“_ref1” value=“output1” class=“org.kepler.sms.KeplerIOPortReference”/>
    <property name=“_ref2” value=“output2” class=“org.kepler.sms.KeplerIOPortReference”/>
    <property name="semanticType" value=“urn:lsid:localhost:ontology:1:1#Class6” class=“org.kepler.sms.SemanticType”/>
  </property>

  <!-- Connects a source port (this case, a virtual one) to a target port and links them via an ontology property -->
  <property name="semlink1" class="org.kepler.sms.KeplerIOPortSemanticLink">
    <property name="domain" value="vport1" class="org.kepler.sms.KeplerVirtualIOPortReference"/>
    <property name="range" value="output3" class="org.kepler.sms.KeplerIOPortReference"/>
    <property name="semanticType" value="urn:lsid:localhost:ontology:1:1#Property1" class="org.kepler.sms.SemanticProperty"/>
  </property>

</entity>

An early template for the actor manual pages in doc format is available in CVS (Kepler Workflow Documentation Template), but we no longer expect to utilize that format. Use the XML template above instead.

Storage locations for documentation

Once the above template format is approved, every actor that is included in the Kepler release will need to be documented in this format. Right now we are listing all of the actors that are present in CVS in the configuration file "keplerActorLibrary.xml". The documentation and semantic annotations for each is to be included in that file. It will in turn be used to generate KSW files as well as documentation for use in the Kepler framework and on the web.

Action Items

SAM:
- Create a list of questions that the getting started document needs to answer.
- Create a draft getting started document. (req#1)

ILKAY:
- Explain the existing capabilities of the pre-developed taglets and doclets.
- Create a worklfow that creates automatic test cases for actors and appends them to the nightly-build test list. (req#10)
- Create a documentation checklist. (req#7 - req#8)

DAN:
- Will finalize the installer checklist (req#9)

CHAD & ILKAY:
- Work on the text-based documentation metadata. (req#6)
  - Get the format approved
  - Write XSLT stylesheets to create HTML cheat sheet and the manual page for each actor
- Write a GUI widget for documenting actors and workflows for users to use
  - Should include all fields in the XML template
  - Might be integrated with the semantic annotation dialogs, which are just more structured documentation

ALL KEPLER ACTOR DEVELOPERS: (until April 10th)
- Update their javadocs for all the actors so that we can have javadoc-based documentation (manual pages) in the upcoming release. (No longer needed except normal javadoc documentation targeted at developers.)

Completed Action Items

Check in the existing automated framework code base into the Kepler CVS and make it work for all the code base. (Ilkay-closed March 28th)
Updated developers tutorial to bring it up to date. It is at KeplerDevelopersTutorial (Ilkay-closed April 5th)
FAQ's generated for general site visitors, users, and developers. (Ilkay)
Added some workflow documentation under $KEPLER/users/ExistingWF directory. (Ilkay-April 10th)
Kepler installation instructions checkin into the CVS. Download.pdf here (Ilkay+Nandita-April 12th)
Document the requirements for creating XML-based metadata descriptions for all the entitites, and using them to generate documentation (Matt-September 9, 2005)

Contributors

Ilkay Altintas, Matt Jones, Dan Higgins, Chad Berkley, Deana Pennigton, Sam Romanello, Xiaowen Xin

Document Actions

Print this

Sections

Personal tools