Salvage data model from pre-analysis project, document, identify unresolved issues

This action is part of Iteration5.

• DONE (1/2 md) Migrate documents from old CVS module to the DOMS wiki. The old documents live in the Ingest module.

• DONE (1/4 md) Migrate illustrations from CVS to Subversion. Put png versions of the xfig illustrations in svn and include those illustrations in the wiki documents

• DONE (2 md) Compile a document describing the data model from the legacy documents and the ones migrated from CVS

• DONE (2 x 1/2 md) Identify unresolved issues and document these. Issues already uncovered include:
  • Machine readable format of type-definitions needs to be chosen and implemented
  • Decide/finalize on a object structure
• DONE (2 md) Review compiled data model document (incl follow up)
• DONE (1/2 md) Review document on unresolved issues (incl follow up)

Total time: 6 1/4 md

Legacy Documents

• DOMS data formats
  22/11-2007: These sub pages describe the format recommendations, they should be incorporated in the new data model.

  • DOMS images (TIFF, PNG, JPEG2000)

  • DOMS audio (Broadcast WAV, WAV)

  • DOMS movies (MPEG1, MPEG2)

  • DOMS text (PDF, XML)
    

  The final recommendations in Danish (as of August 2006) have been imported to SVN:
  

  • DOMS_anbefaling.doc and
    DOMS_anbefaling.pdf.

• DOMS Objects

• DOMS Object Structure

• DOMS Content Model

• Ingest 2 Experiences, Ingest 2 Results, Ingest 2 Extras

Imported Documents

The following documents have been imported from LaTeX documents from old DOMS CVS repository.

• DOMSContentDescription

• DOMSObjectStructureDiscussion

• IngestExampleObjects

• RadioavisManuskripter

Legacy Base and Example Objects

The test objects from earlier in the project can be found in the old DOMS CVS ingest test data directory.

Compiled Document

• Data Model

• Full Data Model

• Review of Compiled Data Model Document

  13/11-2007: TSH, KFC, ABR asked to review Full Data Model.
  14/11-2007: Review meeting with KFC, ABR, MKE, BAM. Follow-up: BAM. And we should have a second review after the follow-up - possibly in connection with review of unresolved issues document.
  19/11-2007: KFC, ABR, JRG asked to review updated Full Data Model.
  20/11-2007: Combined review meeting Gentofte Data Model, Full Data Model Documentation, Data Model Unresolved Issues with KFC, ABR, JRG, MKE, BAM. 22/11-2007: It was not possible to implement the issues raised in review within the iteration 5 timeframe. Comments inlined in the relevant pages instead. -- mke 2007-11-22 10:55:26

Unresolved

• Unresolved Issues

• Review of Unresolved Issues

  20/11-2007: Combined review meeting Gentofte Data Model, Full Data Model Documentation, Data Model Unresolved Issues with KFC, ABR, JRG, MKE, BAM. 22/11-2007: It was not possible to implement the issues raised in review within the iteration 5 timeframe. Comments inlined in the relevant pages instead. -- mke 2007-11-22 10:55:26

TODO

• Update WBS tasks

• Wikify document on recommended file formats and include it in data model wiki pages

• Implement comments from review. This was not possible to do in the timeframe of iteration 5

• Make data model docs more easily discoverable in the wiki

DOMS Contents Description

This document was imported from: contentsDescription.tex

Author: bam

Object Types

All objects to be ingested in the DOMS must implement the SB type, which is summarised in fig. 1. The PID of an object is 'projectname:objectname', i.e. the left side should be a known project; the right side should be 'project' for a project object and an object name for non-project objects. 'Known' projects should be specified in Fedora properties.

We do not have a final DOMS Dublin Core description (Language? Should format and rights be described in DC as well as in RELS-EXT? We talked about letting the Fedora Dublin Core be a simple 'title-holder' and then introduce an SB Dublin Core datastream. Why was that and should we?); a good suggestion can be found on SB DOMS object structure.

Figure 1.

All other types of objects are additions to the SB type and are summarised in figures 2, 3, 4, 5, 6, 7, 8 and 9.

We are still missing descriptions of type, rights and technical datastreams. These are necessary to define for the validation iteration

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Figure 9. DOMS object type hierarchy. Original xfig source.

This page was imported from: discussionPaper.tex by mke

Author: bam

DOMS Objects and Object Structure Discussion

The current (Tue Nov 21) object design and structure choices are presented briefly and a number of questions are raised.

DOMS Objects

On the page DOMS objects three basic objects, 'Group', 'Entity' and 'Atom', are suggested. Are there differences which merit a definition of different DOMS objects? Should such 'top level object types' be defined in the 'contentModel' property (in current example all objects have contentModel 'SBOBJECT')?

There is also a question of 'ordered groups', e.g. pages in a book. There are a number of possible solutions for this...

Example objects

The 'logo1' ingest example data/ingest2Test/ in figure 1 illustrates relations between objects. The image object 'close-up' illustrates the contents of a DOMS object with type image.

Figure 1. The 'logo1' ingest example 'from a distance' and an image object 'close-up'. Original xfig sources here and here.

DOMS Content Model

The content model defines the content of one DOMS object. Different types have different content models. Types are also DOMS objects and all DOMS objects reference a type using the 'isObjectType' relation - except the 'type type object'?

SB Objects

All objects are 'SB objects' and contain (see DOMS Content Model):

• Properties

  • type = "FedoraObject"

  • state = "A"

  • label

  • contentModel = "SBOBJECT"

• Datastreams

  • Dublin Core

  • Relations

    • hasRights

    • isObjectType

  • Administrative

• Disseminators

  • ?VIDEO_PRESENTATION (start, maxMilliSecs) -> MPEG2

  • ?SOUND_PRESENTATION (start, maxMilliSecs) -> MP3

  • ?IMAGE_PRESENTATION (maxWidth, maxHeight) -> PNG

  • ?TEXT_PRESENTATION (start, maxChars) -> UTF-8 TXT

  • Index Representation -> XML

The 'Fedora Object Dublin Core datastream' is specified where? The 'SB Object Dublin Core datastream' should be specified.

The page SB DOMS object structure suggests:

• Additional datastreams

  • ?PRESENTATION_IMAGE_150x150

  • ?PRESENTATION_IMAGE_700x700

  • ?PRESENTATION_IMAGE_FULL

  • ?PRESENTATION_MOVINGIMAGE_PREVIEW

  • ?PRESENTATION_MOVINGIMAGE

  • ?MASTER

Type Objects

Type is a subtype of SB (type type exception)?

• Additional datastream

  • Type

The 'type datastream' is yet to be defined. The type SB (all types subtypes of type SB) should be predefined (missing in example). Which other types should be predefined? Suggestions: type type, type project, type rights, type file, type logical, type image, type audio, type video, type text, type filetype (all filetypes subtypes of type filetype?), certain filetypes?

Would we like all type objects to have 'isObjectType relation' to type type and introduce a 'subtype relation' (self reference issue again)?

Rights Objects

Rights is a subtype of SB (hasRights exception)?

• Additional datastream

  • Rights

    • seeInIndex

    • seeObject

    • accessContent

    • modifyObject

The 'rights datastream' should be defined.

Non-Project Objects

• Datastreams

  • Additional relation

    • isPartOfProject

This is suggested on the page SB DOMS object structure. This suggests an 'SB project' for types and rights objects. Perhaps the 'isPartOfProject relation' should be defined for logical objects?

File Objects

• Datastreams

  • Additional relations

    • isFileType

    • ?hasOriginal

• Additional datastreams

  • Content

  • Technical

Image/Audio/Video/Text Objects

• Datastreams

  • Additional relations

    • ?has...File

Questions

• PID The left side should be a known project; these should be specified in Fedora properties; right side should be? Object renaming?

• contentModel =SBOBJECT for all objects in example; should this be changed to 'top-level types'?

• Dublic Core Language? Should format and rights be described in DC as well as in RELS-EXT? Last Thursday we talked about letting the Fedora Dublin Core be a simple 'title-holder' and then introduce an SB Dublin Core datastream. Why was that and should we?

• RELS-EXT How do we avoid self-references in a nice way? Does 'object type type' not have an 'isObjectType' relation? Do all type objects not have an 'isObjectType' relation? How many 'isObjectType' relations can one object have? Do we have a 'subType' (or a more general 'conceptualParent') relation? Repeat the above questions with right objects and 'hasRights'. Is the 'isPartOfProject' relation mandatory for all non-project objects (including types and rights) -- do we have an SB project for types and rights? Should the 'isPartOfProject' relation be mandatory for file objects (if we expect these to be referenced by multiple logical objects)? Do we want an explicit 'isFileType' relation in file objects or do we want to use the 'isObjectType' for file-type as well?

TODO

• Datastreams Rights: Usage rights (if I can access the material, what can I use the material for)? Creative Commons machine readable rights? Type datastream (type objects). Technical datastream (file objects).

• Syntax pages (for doms-relations, administrative, rights, type and technical datastreams)

Ingest Example Objects

This page was imported from: exampleDescription.tex

Author: bam

The ingest example is located in data/ingest2Test. The example consist of the 'SB project' and a 'logo project' in three parts. The 'SB project' example (fig. 1) contains the types and rights necessary for the 'logo project' example (note that the object 'sb:Rights\_internal' acts as bottom rights element in the example). When setting the DOMS into production, we should start by ingesting an 'SB project' containing all predefined types and rights. The 'logo1' directory contains the 'SB logos project' (fig. 2) and is dependant on the 'SB project' as illustrated. The 'logo2' directory contains an additional logo and is dependant on both the 'SB project' and the 'SB logos project'. The 'logo3' directory contains another logo and has an error in the relations and should not pass the RIFF check.

Figure 1. Original xfig source.

Figure 2. Original xfig source.

Radioavismanuskripter Small Handmade Example

This document was imported from radioavismanuskripterSmallExample.tex

Author: bam

The example is located in DOMS/Ingest/data/radioavismanuskripter. The example depends on the 'SB project' (DOMS/Ingest/data/ingest2Test/sbProject). The 'SB project' example has been updated with a 'text object type' and a 'filetype UTF8' (see fig. 1). The SB project should be further expanded with all 'pre-defined' types, filetypes and rights.

The 'radioavismanuskripter project' is shown in figure fig. 2. Not all relations are shown in the figure. All objects except the project object have an 'isPartOfProject' relation to the project object. All objects except the rights object have a 'hasRights' relation to the (collection specific) rights object. All objects except the manuscript object have an 'isObjectType' relation to a predefined type object in the SB project, and all file objects except the original text file object have an 'isFileType' relation to a predefined file-type in the SB project.

The manuscript type defines the date and time of the broadcast and the number of pages of the manuscript as compulsary metadata. The new relations defined are 'hasManuscriptText' (there should be at most one), 'hasManuscriptPage' (at least one; 40 in the example), 'hasNext' (at most one) and 'hasPrevious' (at most one). The two first relations should be followed by the index disseminator to 'assemble' the 'radioavismanuskript', whereas the last two should not.

Figure 1. Original xfig source.

Figure 2. Original xfig source.