This document provides comprehensive software development processes tailored for open source projects. It covers processes, conventions, and recommended tools. The guide aims to help developers quickly get up to speed with best practises.

You are not expected to read this guide cover-to-cover. You probably should be familiar with the contents and reference the appropriate section when you need it. Documentation aims to be concise with references elsewhere on the web for details like installation instructions.

This guide is written in a modular fashion so that different projects can easily add, delete, or modify sections. It is hoped that this guide will become the de-facto standard software developers guide for java based open source projects.

For this guide to be useful it needs to be continually added to and improved as tools are developed, processes improved and projects grow. Please consider improving or adding a section if you feel it is required.

Ideally, all recommended tools would be open source, however we have included some no-cost tools where there are gaps in the open source tool set. Where applicable, widely accepted conventions and open standards are used. It has been satisfying to discover the breadth of quality open source tools which support software development. It is hoped that this document will highlight areas where tools can be improved or developed and encourage developers to focus on these areas.

This section should be replaced and explain your project.

Copyright (c) 2003 Generguide Project Management Committee. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being with no Invariant Sections, with the Front-Cover Texts being no Front-Cover Texts, and with the Back-Cover Texts being no Back-Cover Texts.

Copyright (c) 2003 Generguide Project Management Committee.

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

This project contains Java code which requires java version 1.4 or greater. To ensure the classes compile without errors, you need to specify that you are using 1.4 or greater using the following options:

Make sure you update compile options in Netbeans if you use Netbeans.

If you use ant for compiling, or building javadocs or similar, then you will need to include something like:

<javadoc source="1.4">...

This project contains Javascript code which requires XML functions. In order to run these functions you will require generation 6 browsers. Ie, Internet Explorer 6 or Netscape 6 or greater.

Concurrent Versions System (CVS) is a version control system that allows multiple developers to work on the same files at the same time.

Information on how to download and edit this project's source code are provided at: http://sourceforge.net/cvs/?group_id=83647.

Note, before you can edit files, you will need to be granted write access from the Generguide Project Management Committee.

An excellent CVS manual can be found at http://cvsbook.red-bean.com/.

This section describes the terms used when building a release. This section is explained in more detail at https://sourceforge.net/docman/display_doc.php?docid=6445&group_id=1#componentoverview.

Release version numbers are based on 3 main digits and an optional release candidate digit : <major>.<minor>.<patch>-rc<release candidate>. It looks like 2.3.4 or 2.3.4-rc5 .

For projects that use maven, the version number is stored in the <currentVersion> tag in maven's project.xml file.

To do: We need to discuss the meaning of:

Every time the version number is incremented, you need to tag the relevant files in the repository. This ensures that previous releases can be recreated for future debugging.

Tag names are based on the module version and look like <package_name>-<major>_<minor>_<patch>-rc<release candidate>. Eg: generguide-lib-2_3_4-rc5 .

Note that full stops from the version number are replaced with underscores. This is because CVS doesn't allow full stops in tags.

Typically, you would create a release tag with the following statements in CVS:

cd generguide
cvs tag generguide-lib-2_3_4

This section describes the steps to follow when building a release.

Users are the people who use the products of the Project. People in this role aren't contributing code, but they are using the products, reporting bugs, making feature requests, and such. This is by far the most important category of people as, without users, there is no reason for the Project.

When a user starts to contribute code or documentation patches, they become a Contributor.

Contributors are the people who write code or documentation patches or contribute positively to the project in other ways. A volunteer's contribution is always recognized. In source code, all volunteers who contribute to a source file may add their name to the list of authors for that file.

Contributors who give frequent and valuable contributions to a subproject of the Project can have their status promoted to that of a "Committer" for that subproject. A Committer has write access to the source code repository and gains voting rights allowing them to affect the future of the subproject.

In order for a Contributor to become a Committer, another Committer can nominate that Contributor or the Contributor can ask for it.

Once a Contributor is nominated, all existing committers will vote. If there are at least 3 positive votes and no negative votes, the Contributor is converted into a Committer and given write access to the source code repository. This is an example offer letter that should be sent to the volunteer after 3 positive votes have been received:

Committers are asked to coordinate their efforts with the maintainers of the modules they wish to modify/extend.

At times, Committers may go inactive for a variety of reasons. A Committer that has been inactive for 6 months or more may lose their status as a Committer. Getting access back is as simple as re-requesting it on the project's Developer mailing list.

Committers who frequently participate with valuable contributions may have their status promoted to that of a Project Management Committee Member. This committee is the official managing body of the Project and is responsible for setting overall project direction. In order to become a Member, someone on the PMC must nominate the Committer. The individual may then be approved with a 3/4 majority of the PMC.

Internet Chat Relay (IRC) is a protocol which allows a number of users to chat with each other using text messages in real time.

If you are new to IRC, you will need to install one of the many IRC clients. The later versions of Netscape and Mozilla have IRC built in, and you can connect to a IRC meeting with the URL: irc://irc.eu.freenode.net#some_channel.

Email lists provide an effective form of communication. Details about this project's email lists can be found at: http://sourceforge.net/mail/?group_id=83647.

You can place @task markers in the javadoc sections of code to track areas for rework. During the maven build process, the tags are extracted to build a report page. Inline Task Tracking is particularly useful during development. After code has been released, bugs should be tracked using the Bug Tracker.

The following tags are available:

/**
 * This is an example method.
 * @param i This explains the variable.
 * @task TODO: This method should have some code.
 **/
 void example(int i) { }

First off, what is maven? Maven is a Java project management and project comprehension tool, or in other words, yet another build tool.

It is built on top of ant and a number of other open source utilities and brings them together in an easy to use tool.

The key part of maven is the use of project.xml files.

The project.xml file tells you the name of the project, who maintains it, who develops it, what version it has reached and what it depends on. Module project.xml files can extend and inherit from a master project.xml file.

The most important part of the project.xml file is the dependencies section as maven uses this to determine what order to build the modules in and what support jars to download when needed.

cd ~/generguide
maven build

cd ~/generguide
maven build -o

maven java:compile

Error: unable to download core-0.1.jar

maven jar:install

To do: Explain Continuous Build, and CruiseControl (once we get it working).

Javascript files need to be uploaded across potentially slow internet connections. Consequently, it is desirable to keep file sizes to a minimum.

Javascript files can be compressed with The Creativyst CSS and JavaScript Compressor.

To do: Find an open source compression algorithm. The above algorithm removes white spaces and comments, but does not compress file names, which could reduce file size even more.

A design document should provide a brief overview of the design to help developers gain a quick understanding of the system. The design documentation is enhanced by using more diagrams and less text.

Text in the design document should be written in docbook format. Refer to the documentation section for more details about using docbook.

Documentation about procedures and classes should be embedded in the source code and then extracted into Detailed Design documentation using an external parser.

UML is a standard notation used to specify, visualise, construct and document the components of an object-oriented software-intensive system. UML diagrams are used as part of design documentation.

Netbeans is a free java IDE which also provides a framework for adding extra functionality. Sun repackage Netbeans and call it Sun ONE Studio (previously Forte for J).

# change to your home directory
cd ~
# copy the netbeans configuration directory
cp -pr .netbeans/ .netbeans.bak

Emacs is not a Java IDE, however it can be treated like multi-language IDE without a GUI builder. If you are not a GUI designer or developer emacs is well worth considering for a software development environment.

The most important features for Java developers (most of them provided by JDEE) are:

Jalopy is a source code formatter for Java. It lays out any valid Java source code according to configurable rules to meet a coding style without putting a formatting burden on individual developers.

Jalopy can be integrated into a number of IDEs, and a project's coding convensions can be stored centrally and imported by all developers. Installation is straight forward, refer to the Jalopy documentation for details.

Each developer should setup Jalopy in their prefered IDE and then import this project's coding convensions, jalopyconfig.xml.

Code that you don't want formated by Jalopy can be wrapped by //J- and //J+ comments.

//J-
  private specialFormattedClass(){return True} //J+

PMD automatically checks code for poor programming patterns.

To do: Reference PMD configuration file. Explain the patterns that are checked.

The logging package (java.util.logging) is bundled into J2SE 1.4 and above. An overview is available on line in the Sun's SDK documentation.

This project uses one logger per class and is named after the class name.

The logger should be declared in the class's static fields.

// J2SE dependencies
import java.util.logging.Logger;

public class MyClass {
    /**
     * The logger for the GML DataSource module.
     */
    private static final Logger LOGGER =
    Logger.getLogger("net.sourceforge.generguide.mymodule.MyClass");

}

Message can be conveniently logged using one of 7 predefined levels. The levels in descending order are:

By default, level SEVERE, WARNING and INFO are logged to the standard output. Level FINER is commonly used when entering, returning, or throwing an exception. A convenience method exists in Logger for each of those levels.

LOGGER.info("There is a message of interest for ordinary user");

There is three more convenience methods: entering, exiting and throwing when entering and exiting a method, or when we are about to terminate a method with an exception.

public Object myMethod(String myArgument) {
    LOGGER.entering("MyClass", "MyMethod", myArgument);
    // ... do some process here
    LOGGER.exiting("MyClass", "MyMethod", myReturnValue);
    return myReturnValue;
}

When logging a message, the logger will include many informations like date and time, source class and method names, current thread, etc. In order to avoid too many informations to be logged, it may be useful to merge consecutive logging into a single log statement.

// Wasteful use of logging
LOGGER.finer("Value for A is "+A);
LOGGER.finer("Value for B is "+B);
LOGGER.finer("Value for C is "+C);

// Good use of logging
LOGGER.finer("Computed values: A="+A+"; B="+B+"; C="+C);

If the log message is expensive to construct, then consider enclosing it in a "if" statement.

if (LOGGER.isLoggable(Level.FINER)) {
    LOGGER.finer("Current state = "+someVeryExpensiveMethodCall());
}

To change the default logging setting, edit the following file:

$JAVA_HOME/jre/lib/logging.properties

Define the ".level" property to the minimal level of interest for you:

.level= FINER

Define the java.util.logging.ConsoleHandler.level property to the minimal level you want to see on the console. It may be different than the level logged to the XML file.

# Limit the message that are printed on the console to FINE and above.
java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
java.util.logging.ConsoleHandler.encoding = Cp850

Note the "encoding" property. For Windows user, it should be set to the value displayed by "chcp" on the command line. Linux and Unix users may ignore this line since Unix systems do a more intelligent work with page codes.

Finally, a different logging level may be specified for each module. For example you may be interested in fairly detailed logging message from module1, but not from module2. Then you could write:

net.sourceforge.generguide.module1.level = FINE
        net.sourceforge.generguide.module2.level = INFO

To do: Write this. Ideally some large project will offer their exception handling policy so we don't have to re-invent the wheel.

Directory names shall be all lower case with no spaces. Some versions of windows do not distinguish between upper and lower case, and in unix, writing spaces in filenames is painful.

to do: need to fill out this section.

To ease refactoring you can use RefactorIt which provides tools to:

More details can be found from the online help.

RefactorIt is commercial, but provides no-cost licences for Open Source projects. See the RefactorIt web pages for details. It can be plugged into a variety of IDEs, including Netbeans.

Installation into netbeans is fairly strait forward through Tools -> RefactorIt -> Project Options, although setting up soucepath and classpath is a difficult if some of the source files don't compile. You will need to either remove the offending files, get them to compile or remove them from RefactorIt's sourcepath.

Good free profiling tools that we have used are a combination of hprof (provided with JDK) and HPJMeter (which analyzes profiling data from hprof).

To invoke hprof, run java with the following options: -Xrunhprof:cpu=samples,thread=n,depth=40,cutoff=0,format=a

If you are running from Netbeans, select Toos->Options->Debugging and Executing->Execution Types->(right click)->New->External Executor. Rename the new executor HProf External Executor (or whatever you like). Still in the Options dialog box, open HProf External Executor->Properties->External Process and in arguments add at the beginning: -Xrunhprof:cpu=samples,thread=n,depth=40,cutoff=0,format=a {assertEnabled}

hprof will be launched every time you select this kind of executor. Go to HProf External Executor->Expert->Working Directory. Set a working directory, Eg /home/<username>/tmp/hprof. Hprof dumps the resulting text file in this directory. This setting modifies the working directory of the java JVM. Make sure the working directory exists, otherwise the executor won't work.

Close the options dialog and select the file you want to profile, select (right click on file to profile)->Properties->Execution->Executor. Set to the HProf External Profiler. Run. At the end of the run you'll find a java.hprof.txt file that can be opened with HPJMeter.

Junit provides a simple framework to help write repeatable unit tests.

If you are testing net.sourceforge.generguide.module.HelloWorld, you should create a file src/<module>/tests/unit/net/sourceforge/generguide/<module>/HelloWorld.java similar to the following:

package net.sourceforge.generguide.module;

  import net.sourceforge.generguide.module.HelloWorld;

  /**
   * Unit test for HelloWorld.
   *
   * @author Cameron Shorter
   */
  public class HelloWorldTest extends TestCase {

      /** Test suite for this test case */
      private TestSuite suite = null;

      /**
       * Constructor with test name.
       */
      public HelloWorldTest(String testName) {
          super(testName);
      }

      /**
       * Main for test runner.
       */
      public static void main(String[] args) {
          junit.textui.TestRunner.run(suite());
      }

      /**
       * Required suite builder.
       * @return A test suite for this unit test.
       */
      public static Test suite() {
          TestSuite suite = new TestSuite(HelloWorldTest.class);
          return suite;
      }

      /**
       * Initialize variables
       */
      protected void setUp() {
          // initialize variables
      }

      /** Test normal constuctors. */
      public void testHello(){
          assertTrue("HelloWorld should return null is true",HelloWorld.isNull());
      }
  }

Testing with junit is as easy as copying HelloWorldTest.java and adding more tests, and then executing it. More information can be found at the junit site.

Any test data required should be stored in generguide/src/<module>/tests/testdata/.

jcoverage reports the code coverage provided by testing.

quilt looks similar.

To do: Check jcoverage and quilt out, see if they are any good, and write up a section explaining how to use them.

This quick start section will get you editing simple docbook sections quickly and easilly. For more detailed information, refer to the rest of the documentation section.

Documentation for this project is written using Simplified Docbook format. Simplified Docbook is a subset of Docbook XML, a versatile format used by the Linux Documentation Project (among others).

Docbook can be easily converted into numerous output formats, like HTML, PDF, etc.

Using Docbook ensures documentation content is kept separate from presentation so content can written once and published in numerous formats and styles.

There are a few GUI editors for docbook now, with more editors promising to support docbook in the future.

According to The Software Release's Howto, docbook is the open source format of the future, and most of the high profile open source projects have moved or are moving to docbook as their coding standards. So we are saving ourselves pain in the future by embracing Docbook now.

Modular DocBook means content is broken up into smaller file modules that are recombined for publication. The advantages of modular documentation include:

This project uses modular documents for it's documentation.

If you want to document a new module, you need to write a new module.xml section file, and then add an <xi:include> tag into design.xml.

Notice in the above example that <section> tags contain an id attribute. This means the URL of published sections will be human readable, something like http://net.sourceforge.generguide/docs/design.html#licence instead of http://net.sourceforge.generguide/docs/design.html#id2754172.

You can edit docbook with the following free tools:

There are a few references worth knowing: