|
Size: 3949
Comment:
|
Size: 4837
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 42: | Line 42: |
| The steps of the test itself are: | [[BR]] The release test consists of |
| Line 45: | Line 48: |
| 1. Human inspection of preingest FoxML output by comparing the output objects with objects prepared beforehand (available "here"). 2. Ingest into Fedora (seeing if Fedora accepts the FoxML objects). 3. Inspection of the objects in Fedora (by use of admin application) |
Test that the system responds as expected when presented with valid data. |
| Line 52: | Line 52: |
Each table below describes the steps needed to complete the release test. Subpages describe the details of what you should do in the step. On each step, do what the subpage describes. Then mark the status column with "OK" or "Fail" depending on whether the step was succesful or not. The notes fields can be used for comments about the test. [[BR]] == Success test == |
|
| Line 53: | Line 65: |
| Test the Simple Preingest, in particular with the following problems: * missing pages in sequence * invalid png-files |
Test that the system responds as expected when presented with valid data. === Steps === ||'''Test step'''||'''Status'''||'''Notes'''|| || [:DDAReleaseTestInstallMini:Install the miniDOMS-package] || || || || [:DDAReleaseTestPreingest:Perform pre-ingest of selected objects] || || || || [:DDAReleaseTestCompareHandWritten:Compare pre-ingest output with hand-written objects] || || || || [:DDAReleaseTestIngest:Ingest pre-ingest output into the Fedora] || || || || [:DDAReleaseTestDisseminator:Test direct disseminator call] || || || || [:DDAReleaseTestInstallSumma:Install Summa On A Stick] || || || || [:DDAReleaseTestOAIHarvest:Perform OAI harvest from miniDOMS to Summa] || || || || [:DDAReleaseTestSummaIndex:Summa-Ingest Harvest Output and Generate Summa Index] || || || || [:DDAReleaseTestSummaIntegration:Summa Integration Success Test] || || || [[BR]] == Failure test - Simple preingester fails on missing pages == === Purpose === Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages that has missing pages in the sequence. === Steps === ||'''Test step'''||'''Status'''||'''Notes'''|| || [:DDAReleaseTestInstallMini:Install the miniDOMS-package] || || || || [:DDAReleaseTestFailPreingestMissing:Perform preingest of objects with missing pages in the sequence] || || || Note: It might (not) be relevant to test what happens when pre-ingester receives invalid png-files. [[BR]] == Failure test - Simple preingester fails on filename anomalies == === Purpose === Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages that includes pages with filename anomalies. |
| Line 59: | Line 105: |
| This table describes the steps needed to complete the releasetest. Subpages describe the details of what you should do in the step. On each step, do what the subpage describes. Then mark the status column with "OK" or "Fail" depending on whether the step was succesful or not. If the step is unsuccesful, you report the problem to bugzilla, and the QA master must decide whether the bug is important enough that it needs to be fixed before release. In that case the releasetest needs to restart after the bug is fixed. The notes field can be used for comments about the test. |
|
| Line 68: | Line 107: |
| || [:DDAReleaseTestInstallMini:Install the miniDOMS-package] || || || || [:DDAReleaseTestFailPreingestFilenameAnomalies:Perform preingest of objects with filename anomalies] || || || [[BR]] == Failure test - Simple preingester fails on last page missing == === Purpose === Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages where the last page is missing. === Steps === ||'''Test step'''||'''Status'''||'''Notes'''|| || [:DDAReleaseTestInstallMini:Install the miniDOMS-package] || || || || [:DDAReleaseTestFailPreingestLastPageMissing:Perform preingest of objects with last page missing] || || || [[BR]] == Failure test - Simple preingester fails on too many pages == === Purpose === Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages with too many pages compared to the number in BIND_x.xml. === Steps === ||'''Test step'''||'''Status'''||'''Notes'''|| || [:DDAReleaseTestInstallMini:Install the miniDOMS-package] || || || || [:DDAReleaseTestFailPreingestTooManyPages:Perform preingest of objects with too many pages] || || || [[BR]] == Failure test - Full preingester fails == |
|
| Line 72: | Line 148: |
| Line 74: | Line 149: |
| This table describes the steps needed to complete the releasetest. Subpages describe the details of what you should do in the step. On each step, do what the subpage describes. Then mark the status column with "OK" or "Fail" depending on whether the step was succesful or not. If the step is unsuccesful, you report the problem to bugzilla, and the QA master must decide whether the bug is important enough that it needs to be fixed before release. In that case the releasetest needs to restart after the bug is fixed. The notes field can be used for comments about the test. |
|
| Line 83: | Line 151: |
| || || || || [[BR]] == Failure test - Direct disseminator call fails == |
|
| Line 87: | Line 161: |
| Line 89: | Line 162: |
| This table describes the steps needed to complete the releasetest. Subpages describe the details of what you should do in the step. On each step, do what the subpage describes. Then mark the status column with "OK" or "Fail" depending on whether the step was succesful or not. If the step is unsuccesful, you report the problem to bugzilla, and the QA master must decide whether the bug is important enough that it needs to be fixed before release. In that case the releasetest needs to restart after the bug is fixed. The notes field can be used for comments about the test. |
|
| Line 98: | Line 164: |
| || || || || [[BR]] == Failure test - Performing OAI harvest fails == |
|
| Line 102: | Line 174: |
| Line 104: | Line 175: |
| This table describes the steps needed to complete the releasetest. Subpages describe the details of what you should do in the step. On each step, do what the subpage describes. Then mark the status column with "OK" or "Fail" depending on whether the step was succesful or not. If the step is unsuccesful, you report the problem to bugzilla, and the QA master must decide whether the bug is important enough that it needs to be fixed before release. In that case the releasetest needs to restart after the bug is fixed. The notes field can be used for comments about the test. |
|
| Line 113: | Line 177: |
| || || || || |
Release test
The release test consists of the following (TODO: Still needs to be fleshed out some more)
The objects chosen for the first step of the release test are:
- Volume Vol. 1
- Region København (Contains Town 1)
- Region Holstein (Contains Town 200 - 275)
- Town 1 (København)
- Town 207 (Segeberg)
- Town 200 (Kiel)
- Paper 1-18
- Paper 1-19
- Paper 200-1
- Paper 200(1771)
- Paper 207-1
- Page 1_86
- Page 1_293
- Page 1_300
- PNG dda1_86
- PNG dda1_293
- PNG dda1_300
The release test consists of
- Success test
- Test that the system responds as expected when presented with valid data.
- Failure tests
- Test of correct behaviour on failures, ensuring that they are "failing gracefully".
Each table below describes the steps needed to complete the release test. Subpages describe the details of what you should do in the step.
On each step, do what the subpage describes. Then mark the status column with "OK" or "Fail" depending on whether the step was succesful or not.
The notes fields can be used for comments about the test.
Success test
Purpose
Test that the system responds as expected when presented with valid data.
Steps
Test step |
Status |
Notes |
[:DDAReleaseTestInstallMini:Install the miniDOMS-package] |
|
|
[:DDAReleaseTestPreingest:Perform pre-ingest of selected objects] |
|
|
[:DDAReleaseTestCompareHandWritten:Compare pre-ingest output with hand-written objects] |
|
|
[:DDAReleaseTestIngest:Ingest pre-ingest output into the Fedora] |
|
|
[:DDAReleaseTestDisseminator:Test direct disseminator call] |
|
|
[:DDAReleaseTestInstallSumma:Install Summa On A Stick] |
|
|
[:DDAReleaseTestOAIHarvest:Perform OAI harvest from miniDOMS to Summa] |
|
|
[:DDAReleaseTestSummaIndex:Summa-Ingest Harvest Output and Generate Summa Index] |
|
|
[:DDAReleaseTestSummaIntegration:Summa Integration Success Test] |
|
|
Failure test - Simple preingester fails on missing pages
Purpose
Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages that has missing pages in the sequence.
Steps
Test step |
Status |
Notes |
[:DDAReleaseTestInstallMini:Install the miniDOMS-package] |
|
|
[:DDAReleaseTestFailPreingestMissing:Perform preingest of objects with missing pages in the sequence] |
|
|
Note: It might (not) be relevant to test what happens when pre-ingester receives invalid png-files.
Failure test - Simple preingester fails on filename anomalies
Purpose
Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages that includes pages with filename anomalies.
Steps
Test step |
Status |
Notes |
[:DDAReleaseTestInstallMini:Install the miniDOMS-package] |
|
|
[:DDAReleaseTestFailPreingestFilenameAnomalies:Perform preingest of objects with filename anomalies] |
|
|
Failure test - Simple preingester fails on last page missing
Purpose
Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages where the last page is missing.
Steps
Test step |
Status |
Notes |
[:DDAReleaseTestInstallMini:Install the miniDOMS-package] |
|
|
[:DDAReleaseTestFailPreingestLastPageMissing:Perform preingest of objects with last page missing] |
|
|
Failure test - Simple preingester fails on too many pages
Purpose
Test that teh system fails gracefully (with meaningful error message) when receiving a sequence of pages with too many pages compared to the number in BIND_x.xml.
Steps
Test step |
Status |
Notes |
[:DDAReleaseTestInstallMini:Install the miniDOMS-package] |
|
|
[:DDAReleaseTestFailPreingestTooManyPages:Perform preingest of objects with too many pages] |
|
|
Failure test - Full preingester fails
Purpose
Testing the Full Preingest
Steps
Test step |
Status |
Notes |
|
|
|
Failure test - Direct disseminator call fails
Purpose
Check the direct disseminator call
Steps
Test step |
Status |
Notes |
|
|
|
Failure test - Performing OAI harvest fails
Purpose
Test whether harvester generates the expected output.
Steps
Test step |
Status |
Notes |
|
|
|