Quality Assurance Guidelines

Coordination

A QA-master follows commits and checks that reviews are scheduled. It is expected that the QA-master's role will be significant in the beginning and gradually fade, as implementers learn to call for reviews.

At the beginning of each iteration, reviewers are assigned to all implementers. It is the responsibility of the single implementer to call for reviews.

Annotations

Annotations are mandated for the control of reviews and unit tests. The rationale is to closely couple QA to the code, just as documentation is closely coupled to the code with Java Doc. Annotations can be used on many levels on the code, ranging from class-level to single lines. It is expected that the QA-related annotations will be on class- and method-level only.

The annotation-hierarchy goes from class-level to method-level to line-level, meaning that a line-level annotation overrules a class-level. It is recommended that annotations are largely restricted to class-level with some annotations at method-level in order to ensure fairly clean code.

We use the QAInfo annotation provided by sbutil version 0.1.6 and later. sbutil comes with a command line tool qaScan-<version>.jar that is a stand alone application capable of extracting QA reports, possibly with ViewVC integration.

There are two central values for QAInfo annotations:

• level: How important it is that QA is performed. There are four levels defined by the QAInfo.Level enumeration:

  • PEDANTIC: The code is of utmost importance and should be thoroughly reviewed and unit tested. It is expected that all Fault handlers (see the ExceptionGuidelines) will be marked as PEDANTIC.

  • FINE: The code is important or complex and extra care should be taken when reviewing and unit testing.

  • NORMAL: The code is standard and should be reviewed and unit tested normally.

  • NOT_NEEDED: The code does not need reviewing or unit testing.

• state: The state of the code. There are three possible states defined by the QAInfo.State enumeration:

  • IN_DEVELOPMENT: No review should be performed. This is normally used when code is under active development.

  • QA_NEEDED: The code should be reviewed and unit tests performed.

  • QA_OK: Reviews and unit tests has been made and passed for this code. The code is judged to be satisfiable. This annotation should be changed as soon as the code is changed again.

Annotations are added ad-hoc by the responsible programmer and can be changed during the review if the reviewer disagrees on the levels.

• QAInfo accepts a range of other parameters such as deadline, author, comment, revision, and reviewers, which all will be visible in the reports produced by qaScan. Refer to the bundled documentation for the distribution for full details, or check the source code online.

Sample

@QAInfo(level = QAInfo.Level.PEDANTIC,)
        state = QAInfo.State.QA_NEEDED)
public class SuperImportantClass() {
    ....
    @QAInfo(state = QAInfo.State.QA_OK)
    public void storeState() {
        ...
    }
    public void otherImportantStuff() {
        ...
    }
}

Reviews

The practical review process is largely left to the implementers and the reviewers. However, to get things going and ensuring a consistent CodeStyle, every line should be reviewed in the beginning of the implementation state of the DOMS project. It is possible, but not required, to state the designated reviewers in the QAInfo annotation like

@QAInfo(level = QAInfo.Level.NORMAL,
        state = QAInfo.State.QA_NEEDED,
        reveiwers = {"kfc", "te"})
public class Rot13OutputStream() {

The people listed in the reviewers property should be ordered with the last reviewer being last in the list too. This will make it easier to construct a simple audit trail for code going through many revisions. This can be especially handy for code tagged with QA level PEDANTIC.

Remember to review the unit tests along with the code.

A good command for printing code for QA is

a2ps --columns=1 --line-numbers=1 -l 80 -R -s 2 -A sheet -M A4

Unit tests

It is expected that comitted unit tests are able to run without fail. Unit-tests under development should not be comitted or be implemented in a way so that they are not run automatically.

• Unit tests on PEDANTIC-level are expected to be code-covering and of high quality.

• Unit tests on FINE-level are expected to be of high quality.

Unit tests are to be grouped in two suites: light and heavy. Light and heavy refers to approximate run time. Heavy unit tests are expected to run on a build server during night time or similar.

When a unit test fails, the error should be tested for by use of an assertion. (See class Assert at http://junit.sourceforge.net/javadoc/ )