Differences between revisions 16 and 17

Release test

The release test consists of the following (TODO: Still needs to be fleshed out some more)

The objects chosen for the first step of the release test are:

Volume Vol. 1
Region København (Contains Town 1)
Region Holstein (Contains Town 200 - 275)
Town 1 (København)
Town 207 (Segeberg)
Town 200 (Kiel)
Paper 1-18
Paper 1-19
Paper 200-1
Paper 200(1771)
Paper 207-1
Page 1_86
Page 1_293
Page 1_300
PNG dda1_86
PNG dda1_293
PNG dda1_300

The release test consists of

Success test
- Test that the system responds as expected when presented with valid data.
Failure tests
- Test of correct behaviour on failures, ensuring that they are "failing gracefully".

Each table below describes the steps needed to complete the release test. Subpages describe the details of what you should do in the step.

On each step, do what the subpage describes. Then mark the status column with "OK" or "Fail" depending on whether the step was succesful or not.

The notes fields can be used for comments about the test.

Success test

Purpose

Test that the system responds as expected when presented with valid data.

Steps

Test step	Status	Notes
Install the miniDOMS-package
Perform pre-ingest of selected objects
Compare pre-ingest output with hand-written objects
Ingest pre-ingest output into the Fedora
Test direct disseminator call
Install Summa On A Stick
Perform OAI harvest from miniDOMS to Summa
Summa-Ingest Harvest Output and Generate Summa Index
Summa Integration Success Test

Failure test - Simple preingester fails on missing pages

Purpose

Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages that has missing pages in the sequence.

Steps

Test step	Status	Notes
Install the miniDOMS-package
Perform preingest of objects with missing pages in the sequence

Note: It might (not) be relevant to test what happens when pre-ingester receives invalid png-files.

Failure test - Simple preingester fails on filename anomalies

Purpose

Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages that includes pages with filename anomalies.

Steps

Test step	Status	Notes
Install the miniDOMS-package
Perform preingest of objects with filename anomalies

Failure test - Simple preingester fails on last page missing

Purpose

Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages where the last page is missing.

Steps

Test step	Status	Notes
Install the miniDOMS-package
Perform preingest of objects with last page missing

Failure test - Simple preingester fails on too many pages

Purpose

Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages with too many pages compared to the number in BIND_x.xml.

Steps

Test step	Status	Notes
Install the miniDOMS-package
Perform preingest of objects with too many pages

Failure test - Full preingester fails on empty xml file

Purpose

Test that teh system fails gracefully (with meaningful error message) when one of the xml files is empty

Steps

Test step	Status	Notes
Install the miniDOMS-package
Perform preingest of xml files with an empty xml file among them

Failure test - Full preingester fails on mismatching pagenumbers

Purpose

Test that teh system fails gracefully (with meaningful error message) when the two occurrences of the pagenumber at imagefile and pagenumber are not matching.

Steps

Test step	Status	Notes
Install the miniDOMS-package
Perform preingest of xml file with internally mismatching pagenumbers

Failure test - Full preingester fails on nonexistent imagefile

Purpose

Test that teh system fails gracefully (with meaningful error message) when the imagefile referred to in the xml is non-existent.

Steps

Test step	Status	Notes
Install the miniDOMS-package
Perform preingest of xml file which refers to a nonexistent imagefile

Failure test - Direct disseminator call fails

Purpose

Ingest an object and its (faulty) disseminator, so that the webservice will crash. Check that in the end, the result in Summa is acceptable.

Steps

Test step	Status	Notes
Install the miniDOMS-package
Perform pre-ingest of object with faulty disseminator
Ingest pre-ingest output into the Fedora
Check Failed Direct Disseminator Call
Install Summa On A Stick
Perform OAI harvest from miniDOMS to Summa
Summa-Ingest Harvest Output and Generate Summa Index
Summa Integration Failed Disseminator Success Test

Failure test - Performing OAI harvest fails

Purpose

Test whether harvester generates the expected output.

Steps

Test step	Status	Notes
Install the miniDOMS-package
Perform pre-ingest of object with faulty disseminator
Ingest pre-ingest output into the Fedora
Purge a few objects
Install the DDA Disseminator in the test Fedora server
Check Direct Disseminator Call
Install Summa On A Stick
Perform OAI harvest from DOMS to Summa
Summa-Ingest Harvest Output and Generate Summa Index
Summa Integration Success Test
Everything should still work, but the chosen paper object will no longer have a link to a page object.

-  ⇤ ← Revision 16 as of 2009-02-02 12:02:34 → 
  Size: 7767
  Editor: jrg
  Comment:
+   ← Revision 17 as of 2010-03-17 13:09:12 → ⇥
  Size: 7801
  Editor: localhost
  Comment: converted to 1.6 markup
-Deletions are marked like this.
+Additions are marked like this.
 Line 42:
-[[BR]]
+<<BR>>
 Line 60:
-[[BR]]
+<<BR>>
 Line 69:
-|| [:DDAReleaseTestInstallMini:Install the miniDOMS-package] ||  ||  ||
|| [:DDAReleaseTestPreingest:Perform pre-ingest of selected objects] ||  ||  ||
|| [:DDAReleaseTestCompareHandWritten:Compare pre-ingest output with hand-written objects] ||  ||  ||
|| [:DDAReleaseTestIngest:Ingest pre-ingest output into the Fedora] ||  ||  ||
|| [:DDAReleaseTestDisseminator:Test direct disseminator call] ||  ||  ||
|| [:DDAReleaseTestInstallSumma:Install Summa On A Stick] ||  ||  ||
|| [:DDAReleaseTestOAIHarvest:Perform OAI harvest from miniDOMS to Summa] ||  ||  ||
|| [:DDAReleaseTestSummaIndex:Summa-Ingest Harvest Output and Generate Summa Index] ||  ||  ||
|| [:DDAReleaseTestSummaIntegration:Summa Integration Success Test] ||  ||  ||

[[BR]]
+|| [[DDAReleaseTestInstallMini|Install the miniDOMS-package]] ||  ||  ||
|| [[DDAReleaseTestPreingest|Perform pre-ingest of selected objects]] ||  ||  ||
|| [[DDAReleaseTestCompareHandWritten|Compare pre-ingest output with hand-written objects]] ||  ||  ||
|| [[DDAReleaseTestIngest|Ingest pre-ingest output into the Fedora]] ||  ||  ||
|| [[DDAReleaseTestDisseminator|Test direct disseminator call]] ||  ||  ||
|| [[DDAReleaseTestInstallSumma|Install Summa On A Stick]] ||  ||  ||
|| [[DDAReleaseTestOAIHarvest|Perform OAI harvest from miniDOMS to Summa]] ||  ||  ||
|| [[DDAReleaseTestSummaIndex|Summa-Ingest Harvest Output and Generate Summa Index]] ||  ||  ||
|| [[DDAReleaseTestSummaIntegration|Summa Integration Success Test]] ||  ||  ||

<<BR>>
 Line 90:
-|| [:DDAReleaseTestInstallMini:Install the miniDOMS-package] ||  ||  ||
|| [:DDAReleaseTestFailPreingestMissing:Perform preingest of objects with missing pages in the sequence] ||  ||  ||
+|| [[DDAReleaseTestInstallMini|Install the miniDOMS-package]] ||  ||  ||
|| [[DDAReleaseTestFailPreingestMissing|Perform preingest of objects with missing pages in the sequence]] ||  ||  ||
 Line 95:
-[[BR]]
+<<BR>>
 Line 107:
-|| [:DDAReleaseTestInstallMini:Install the miniDOMS-package] ||  ||  ||
|| [:DDAReleaseTestFailPreingestFilenameAnomalies:Perform preingest of objects with filename anomalies] ||  ||  ||

[[BR]]
+|| [[DDAReleaseTestInstallMini|Install the miniDOMS-package]] ||  ||  ||
|| [[DDAReleaseTestFailPreingestFilenameAnomalies|Perform preingest of objects with filename anomalies]] ||  ||  ||

<<BR>>
 Line 122:
-|| [:DDAReleaseTestInstallMini:Install the miniDOMS-package] ||  ||  ||
|| [:DDAReleaseTestFailPreingestLastPageMissing:Perform preingest of objects with last page missing] ||  ||  ||

[[BR]]
+|| [[DDAReleaseTestInstallMini|Install the miniDOMS-package]] ||  ||  ||
|| [[DDAReleaseTestFailPreingestLastPageMissing|Perform preingest of objects with last page missing]] ||  ||  ||

<<BR>>
 Line 137:
-|| [:DDAReleaseTestInstallMini:Install the miniDOMS-package] ||  ||  ||
|| [:DDAReleaseTestFailPreingestTooManyPages:Perform preingest of objects with too many pages] ||  ||  ||

[[BR]]
+|| [[DDAReleaseTestInstallMini|Install the miniDOMS-package]] ||  ||  ||
|| [[DDAReleaseTestFailPreingestTooManyPages|Perform preingest of objects with too many pages]] ||  ||  ||

<<BR>>
 Line 151:
-|| [:DDAReleaseTestInstallMini:Install the miniDOMS-package] ||  ||  ||
|| [:DDAReleaseTestFailPreingestEmptyXml:Perform preingest of xml files with an empty xml file among them] ||  ||  ||

[[BR]]
+|| [[DDAReleaseTestInstallMini|Install the miniDOMS-package]] ||  ||  ||
|| [[DDAReleaseTestFailPreingestEmptyXml|Perform preingest of xml files with an empty xml file among them]] ||  ||  ||

<<BR>>
 Line 165:
-|| [:DDAReleaseTestInstallMini:Install the miniDOMS-package] ||  ||  ||
|| [:DDAReleaseTestFailPreingestPagenumberMismatch:Perform preingest of xml file with internally mismatching pagenumbers] ||  ||  ||

[[BR]]
+|| [[DDAReleaseTestInstallMini|Install the miniDOMS-package]] ||  ||  ||
|| [[DDAReleaseTestFailPreingestPagenumberMismatch|Perform preingest of xml file with internally mismatching pagenumbers]] ||  ||  ||

<<BR>>
 Line 179:
-|| [:DDAReleaseTestInstallMini:Install the miniDOMS-package] ||  ||  ||
|| [:DDAReleaseTestFailPreingestImagefileNonexistent:Perform preingest of xml file which refers to a nonexistent imagefile] ||  ||  ||

[[BR]]
+|| [[DDAReleaseTestInstallMini|Install the miniDOMS-package]] ||  ||  ||
|| [[DDAReleaseTestFailPreingestImagefileNonexistent|Perform preingest of xml file which refers to a nonexistent imagefile]] ||  ||  ||

<<BR>>
 Line 193:
-|| [:DDAReleaseTestInstallMini:Install the miniDOMS-package] ||  ||  ||
|| [:DDAReleaseTestFailPreingestFaultyDisseminator:Perform pre-ingest of object with faulty disseminator] ||  ||  ||
|| [:DDAReleaseTestFailIngestFaultyDisseminator:Ingest pre-ingest output into the Fedora] ||  ||  ||
|| [:DDAReleaseTestFailDirectDisseminatorCall:Check Failed Direct Disseminator Call] ||  ||  ||
|| [:DDAReleaseTestInstallSumma:Install Summa On A Stick]||  || ||
|| [:DDAReleaseTestOAIHarvest:Perform OAI harvest from miniDOMS to Summa] ||  ||  ||
|| [:DDAReleaseTestSummaIndex:Summa-Ingest Harvest Output and Generate Summa Index] ||  ||  ||
|| [:DDAReleaseTestFailDisseminatorSuccess:Summa Integration Failed Disseminator Success Test] ||  ||  ||

[[BR]]
+|| [[DDAReleaseTestInstallMini|Install the miniDOMS-package]] ||  ||  ||
|| [[DDAReleaseTestFailPreingestFaultyDisseminator|Perform pre-ingest of object with faulty disseminator]] ||  ||  ||
|| [[DDAReleaseTestFailIngestFaultyDisseminator|Ingest pre-ingest output into the Fedora]] ||  ||  ||
|| [[DDAReleaseTestFailDirectDisseminatorCall|Check Failed Direct Disseminator Call]] ||  ||  ||
|| [[DDAReleaseTestInstallSumma|Install Summa On A Stick]]||  || ||
|| [[DDAReleaseTestOAIHarvest|Perform OAI harvest from miniDOMS to Summa]] ||  ||  ||
|| [[DDAReleaseTestSummaIndex|Summa-Ingest Harvest Output and Generate Summa Index]] ||  ||  ||
|| [[DDAReleaseTestFailDisseminatorSuccess|Summa Integration Failed Disseminator Success Test]] ||  ||  ||

<<BR>>
 Line 213:
-|| [:DDAReleaseTestInstallMini:Install the miniDOMS-package] ||  ||  ||
|| [:DDAReleaseTestFailPreingestFaultyDisseminator:Perform pre-ingest of object with faulty disseminator] ||  ||  ||
|| [:DDAReleaseTestFailIngestFaultyDisseminator:Ingest pre-ingest output into the Fedora] ||  ||  ||
+|| [[DDAReleaseTestInstallMini|Install the miniDOMS-package]] ||  ||  ||
|| [[DDAReleaseTestFailPreingestFaultyDisseminator|Perform pre-ingest of object with faulty disseminator]] ||  ||  ||
|| [[DDAReleaseTestFailIngestFaultyDisseminator|Ingest pre-ingest output into the Fedora]] ||  ||  ||