Quality Assurance Guidelines

Coordination

A QA-master follows commits and checks that reviews are scheduled. It is expected that the QA-master's role will be significant in the beginning and gradually fade, as implementers learn to call for reviews.

At the beginning of each iteration, reviewers are assigned to all implementers. It is the responsibility of the single implementer to call for reviews.

Annotations

Annotations are mandated for the control of reviews and unit tests. The rationale is to closely couple QA to the code, just as documentation is closely coupled to the code with Java Doc. Annotations can be used on many levels on the code, ranging from class-level to single lines. It is expected that the QA-related annotations will be on class- and method-level only.

The annotation-hierarchy goes from class-level to method-level to line-level, meaning that a line-level annotation overrules a class-level. It is recommended that annotations are largely restricted to class-level with some annotations at method-level in order to ensure fairly clean code.

We use the QAInfo annotation provided by sbutil version 0.1.6 and later. sbutil comes with a command line tool qaScan-<version>.jar that is a stand alone application capable of extracting QA reports, possibly with ViewVC integration.

There are two central values for QAInfo annotations:

Annotations are added ad-hoc by the responsible programmer and can be changed during the review if the reviewer disagrees on the levels.

Sample

@QAInfo(level = QAInfo.Level.PEDANTIC,)
        state = QAInfo.State.QA_NEEDED)
public class SuperImportantClass() {
    ....
    @QAInfo(state = QAInfo.State.QA_OK)
    public void storeState() {
        ...
    }
    public void otherImportantStuff() {
        ...
    }
}

Reviews

The practical review process is largely left to the implementers and the reviewers. However, to get things going and ensuring a consistent CodeStyle, every line should be reviewed in the beginning of the implementation state of the DOMS project. It is possible, but not required, to state the designated reviewers in the QAInfo annotation like

@QAInfo(level = QAInfo.Level.NORMAL,
        state = QAInfo.State.QA_NEEDED,
        reveiwers = {"kfc", "te"})
public class Rot13OutputStream() {

The people listed in the reviewers property should be ordered with the last reviewer being last in the list too. This will make it easier to construct a simple audit trail for code going through many revisions. This can be especially handy for code tagged with QA level PEDANTIC.

Remember to review the unit tests along with the code.

A good command for printing code for QA is

a2ps --columns=1 --line-numbers=1 -l 80 -R -s 2 -A sheet -M A4

Unit tests

It is expected that comitted unit tests are able to run without fail. Unit-tests under development should not be comitted or be implemented in a way so that they are not run automatically.

Unit tests are to be grouped in two suites: light and heavy. Light and heavy refers to approximate run time. Heavy unit tests are expected to run on a build server during night time or similar.

When a unit test fails, the error should be tested for by use of an assertion. (See class Assert at http://junit.sourceforge.net/javadoc/ )

QA Guidelines (last edited 2010-03-17 13:12:52 by localhost)