Differences between revisions 1 and 2
Revision 1 as of 2008-06-26 12:26:08
Size: 1872
Editor: kfc
Comment: Created by the PackagePages action.
Revision 2 as of 2010-03-17 13:12:54
Size: 1872
Editor: localhost
Comment: converted to 1.6 markup
No differences found!

Ingest

The ingest package received Fedora Objects, in the form of FOXML1.0 files, and ingests them into Fedora. The goal for the first iteration of the Ingest is to

  • Provide a command-line controlled mechanism for ingesting a group of objects

Structure

There should be a drop-off folder on the maschine responsible for ingest. The folder should have this structure

  • input/
    • Package1/
    • Package2/
    • ...
  • failure/
    • Package4/
      • Package4.log
      • SomeFOXFile1.xml
      • SomeFOXFile2.xml
      • ...
    • Package8/
      • Package8.log
      • SomeFOXFile1.xml
      • SomeFOXFile2.xml
      • ...
    • ...
  • success/
    • Package4/
      • Package4.log
    • Package10/
      • Package10.log
    • Package11/
      • Package11.log
      • ...

Workflow

  1. Objects are created externally and put in a sub-folder in the input-folder. The name of the sub-folder is project-specific.

  2. The ingest-mechanism is activated with the sub-folder name from step 1 as argument.
    1. The ingester ingests all objects, if possible.
    2. For all successful object ingests, a note is written in a corresponding log in the success-folder and the object is deleted from the input-folder.

    3. For all non-successful object ingests, the object in question is copied to a corresponding sub-folder in the failure-folder and a note is written in the corresponding logfile in the failure-folder.

For iteration 1, this means that an ingest can result in output in both the failure and the success folder. Future iterations with transactions will hopefully only result in output in one of these folders.

Note

Note that the following features are reserved for future iterations

  • Validation of objects
  • Handling of permissions
  • Transaction-based ingest (all objects or no objects)
  • Web-interface

Ingest (last edited 2010-03-17 13:12:54 by localhost)