Differences between revisions 5 and 6
Revision 5 as of 2011-01-31 12:26:08
Size: 905
Editor: jrg
Comment:
Revision 6 as of 2011-01-31 12:41:19
Size: 1248
Editor: jrg
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
== Bulk Ingest of Analogue (BART) files == = Bulk Ingest of Analogue (BART) files =
Line 3: Line 3:
From DigiTV v.0.5.0, file_ingest.sh ingests both digital and analogue recordings. A one-off ingest of all old == Done by Maintenance ==
''
From DigiTV v.0.5.0, file_ingest.sh ingests both digital and analogue recordings. A one-off ingest of all old
Line 5: Line 6:
corresponding script {{{bartfile_bulk_ingester.sh}}}. This is new functionality in DigiTV 0.5.0 corresponding script {{{bartfile_bulk_ingester.sh}}}. This is new functionality in DigiTV 0.5.0''
Line 7: Line 8:
 i. Download a list of all files to be ingested. For example all video files from 2009: {{{wget -O 2009_mpeg.txt http://pluto.statsbiblioteket.dk/~bart/get_url.cgi?*2009*mpeg}}}
 i. Ingest: {{{bartfile_bulk_ingester.sh 2009_mpeg.txt}}}
 i. Download a list of all files to be ingested, as follows. {{{
wget -O 2009_mpeg.txt http://pluto.statsbiblioteket.dk/~bart/get_url.cgi?*_20090301*_20090308*.mpeg ; wget -O 2009_wav.txt http://pluto.statsbiblioteket.dk/~bart/get_url.cgi?*_20090301*_20090308*.wav ; wget -O 2010_mpeg.txt http://pluto.statsbiblioteket.dk/~bart/get_url.cgi?*_20100301*_20100308*.mpeg ; wget -O 2010_wav.txt http://pluto.statsbiblioteket.dk/~bart/get_url.cgi?*_20100301*_20100308*.wav}}}
 i. Ingest: {{{
bartfile_bulk_ingester.sh 2009_mpeg.txt ; bartfile_bulk_ingester.sh 2009_wav.txt ; bartfile_bulk_ingester.sh 2010_mpeg.txt ; bartfile_bulk_ingester.sh 2010_wav.txt}}}
Line 10: Line 13:
 i. Repeat with audio files from the same interval, substituting "wav" for "mpeg" in the above recipe.

Bulk Ingest of Analogue (BART) files

Done by Maintenance

From DigiTV v.0.5.0, file_ingest.sh ingests both digital and analogue recordings. A one-off ingest of all old analogue recordings is also necessary. Functionality for this is implemented in BartFileBulkIngester and the corresponding script bartfile_bulk_ingester.sh. This is new functionality in DigiTV 0.5.0

  1. Download a list of all files to be ingested, as follows.

    wget -O 2009_mpeg.txt http://pluto.statsbiblioteket.dk/~bart/get_url.cgi?*_20090301*_20090308*.mpeg ; wget -O 2009_wav.txt http://pluto.statsbiblioteket.dk/~bart/get_url.cgi?*_20090301*_20090308*.wav ; wget -O 2010_mpeg.txt http://pluto.statsbiblioteket.dk/~bart/get_url.cgi?*_20100301*_20100308*.mpeg ; wget -O 2010_wav.txt http://pluto.statsbiblioteket.dk/~bart/get_url.cgi?*_20100301*_20100308*.wav
  2. Ingest:

    bartfile_bulk_ingester.sh 2009_mpeg.txt ; bartfile_bulk_ingester.sh 2009_wav.txt ; bartfile_bulk_ingester.sh 2010_mpeg.txt ; bartfile_bulk_ingester.sh 2010_wav.txt
  3. Check the corresponding logfiles bartfile_bulk_ingester.debug.log and bartfile_bulk_ingester.error.log. The error log should be empty and the debug log should list all the files ingested.

BulkIngestBartFiles (last edited 2011-01-31 13:03:10 by jrg)