##System Maintenance Documentation
## Headline
= Set-up and System Maintenance of DOMS Test Setup =
/!\ Do not use this script at the moment. It is currently broken. Please follow the instructions in the "Automated Setup" section below.

## Add detailed explanation of the set-up to test or run the result of this task/action/...
## Add detailed explanation of the expected system maintenance for this program/system/test-environment/...
## This may include UML diagrams, schematics describing architecture etc. attached like this:
## attachment:MyPackageDiagram.png
 1. Check out the DOMS project from [[SVN]].
 1. Install MySQL 5 on the machine (see below), if it is not already installed. ({{{sudo /sbin/service mysqld status}}} will tell you if it is installed)
 1. If the MySQL-daemon is not running do {{{sudo /sbin/service mysqld start}}} to start it.
 1. Execute {{{domssetup.sh}}} in {{{test/bin}}} with the wanted test installation folder (no spaces in the name, please) as argument. (F.x. create a test installation folder where the doms folder was created.)
It will prompt for the root password to the MySQL installation (not to be confused with the root password for the machine), create a database with the name {{{fedora_testfoldername}}} and install Fedora under the test folder.

If you want to install with demo-objects, add the command line switch {{{-d}}}. Example:

{{{
  ./domssetup.sh -d ~/mydoms
}}}
will install a test DOMS at the folder {{{mydoms}}} under home, with the provided demo-objects.

== Controlling Fedora ==
The installation script installs two scripts {{{bin/start.sh}}} and {{{bin/stop.sh}}} in the the root of the installation folder. These should be used for starting and stopping the Fedora server.

When the fedora server has started, installation can be verified by visiting

{{{
http://localhost:7900/fedora/search
}}}
##== Overview over the system architecture ==
## Describe system architecture
== Overview over software components ==
=== MySQL installation ===
/!\ Still current

''Note'': This was taken verbatim from [[Fedora]].

If you do not yet have MySQL 5 installed, you need to do this, set the password, and start it up.

For Fedora Core Linux 3 or above, or Redhat Enterprise Linux 4 or above, you should be able to install MySQL, and make it start up at system boot, with the following commands:

{{{
sudo yum install mysql-server
}}}
Continue with:

{{{
sudo /sbin/chkconfig mysqld on
sudo /sbin/service mysqld start
}}}
which enables start of MySQL server on startup.

Now you should set the root password:

{{{
/usr/bin/mysql -u root
mysql> set password for root@localhost=password('new-password');
mysql> set password for root@<<$HOSTNAME>>=password('new-password');
}}}
(Remember to replace {{{<<$HOSTNAME>>}}} with your host name)

== Summa on a Stick ==
/!\ Almost current. Follow the exact commands in Iteration7ReleaseTest

 1. Download the latest version from http://hera/~te/
 1. Unzip it and run {{{summa-setup.sh}}}
 1. Change the line {{{<entry key="create_new_index">false</entry>}}} to {{{<entry key="create_new_index">true</entry>}}} in {{{summa-storage-build-4327/config/controllucene.properties}}}
 1. Add the following lines to {{{summa-ingest-build-4327/config/targets.properties.xml}}}:
 {{{
<!-- Properties for the DOMS target -->
    <entry key="doms">/fedora/domsTestHarvest20080123/summa/data/doms</entry>
    <entry key="doms_run">true</entry>
    <entry key="doms_DigesterClass">dk.statsbiblioteket.summa.ingest.OAI.OAIHarvestDirectoryDigester</entry>
    <entry key="doms_full_ingest">false</entry>
    <entry key="doms_encoding">UTF-8</entry>
    <entry key="doms_prefix"></entry> <!-- All ids from DOMS are already prefixed with 'doms:'. -->
    <entry key="doms_check_output">true</entry>
    <entry key="doms_base_name">doms</entry>
}}}
  . Change the line {{{<entry key="targets">oai</entry>}}} to {{{<entry key="targets">doms</entry>}}} in the same file
 1. Copy DOMS-data to {{{data/doms}}}
 1. Copy DOMS-XSLT's to {{{summa-index-build-4327/config/targets/doms}}}
 1. In the file {{{summa-index-build-4327/config/index.properties.xml}}} remove all lines with {{{OpenArchive}}} and add lines like the following:
 {{{
    <entry key="DOMS_run">true</entry>
    <entry key="DOMS_io_storage">//localhost:8500/summa_storage</entry>
    <entry key="DOMS_xslt_url">file:///home/te/projects/doms_test/summa-build-4327/summa-index-build-4327/config/targets/doms/doms_index.xsl</entry>
    <entry key="DOMS_last_id"></entry>
    <entry key="DOMS_complete_full_index">false</entry>
    <entry key="DOMS_time"></entry>
}}}
  . If the installation is considered stable, it is advisable to point to the XSLTs in CVS instead of the local file-system. Change the line with {{{xslt-url}}} to {{{http://merkur/viewvc/trunk/summa/data/index/targets/doms/doms_index.xsl?root=doms&view=co}}}.
Note: {{{create_new_index}}} in {{{controllucene.properties}}} controls whether the index is created from scratch upon each index or if the existing index should be used.

=== Summa harvester ===
The harvester is responsible for harvesting data from Fedora (or any other OAI-PMH source) and dump them as XML in a designated data dir. The dumped data can be ingested by Summa.

The harvester is located in the folder {{{summa-harvest-build-4327}}} in the Summa on a Stick folder. The {{{config.properties}}} and the {{{targets.xml}}} files in the config folder contains setup-information. The detault-settings put harvested data in a {{{data}}}-subfolder in the {{{summa-harvest-build-4327}}} folder and performs the harvest from {{{http://alhena:7900/oaiprovider/}}}.

The harvester is started with the {{{harvest.sh}}}-script.

== Starting the Summa Prototype ==
<!> It is strongly recommended that the following guide is executed by a user called {{{summa}}}, although this should not be strictly necessary.

=== Start rmiregistry ===
Run $JAVA_HOME/bin/rmiregistry 8500 &

=== Start summa-storage ===
Start Summa Storage with {{{summa-storage-build-4327/luceneStorage.sh &}}}

=== Ingest Sample Data ===
/!\ not nessesary for runs after the first run

Run the {{{ingest.sh}}} script found in {{{summa-ingest-<version>/}}}.

This will ingest a fair amount of records. They are taken from the {{{data}}}-folder. If a smaller test is wanted, remove some of the XML-files in {{{data}}} before running the ingester.

=== Build Main Index ===
/!\ not nessesary for runs after the first run

Run the {{{index.sh}}} script found in {{{summa-index-<version>/}}}. This will take some hours, depending on hardware.

=== Build Parallel Cluster Index ===
/!\ not nessesary for runs after the first run

Run {{{cluster.sh}}} in {{{summa-cluster-<version>/}}}. This will finish in minutes.

=== Build Facet Map ===
/!\ not nessesary for runs after the first run

Run {{{facet.sh}}} in {{{summa-facets-<version>/}}}. This will take a minute or two.

=== Set up Web Server ===
/!\ not nessesary for runs after the first run

Copy all {{{.war}}} files from the root installation directory to the {{{webapps}}} directory in an Apache Tomcat installation.

=== Start Tomcat, if not already done ===
Start the Tomcat server with at least 512 MB ram, fx from the Tomcat root directory:

{{{
JAVA_OPTS=-Xmx512m bin/startup.sh
}}}
=== Test ===
Point your favorite web browser (or MS IE if that is your only option) to

{{{
localhost:7900/simplesumma
}}}
The first search will put the server under a heavy load while the facet map is loaded, but after the initial set up, the service should run smoothly.

'''Note:''' Selecting a tag with non-ASCII characters does not work in simplesumma.

=== Trouble Shooting ===
Checking if anything is ingested (ie. stored in the postgres database) - run {{{psql}}} and enter

{{{
SELECT count (*) from summa_io WHERE base='OpenArchive';
}}}
You can query the search web service directly like this

{{{
http://localhost:7900/summaws/services/SummaSearch?method=simpleSearch&arg0=0&arg1=10&arg2=hello
}}}
To request the contents of a record directly enter the url

{{{
http://localhost:7900/summaws/services/SummaSearch?method=getRecord&arg0=oai:doaj-articles:ab394eb6b5edff982c802993b6981bcd
}}}
Browse the available Summa web services at

{{{
http://localhost:7900/summaws/services
}}}
== Automated setup ==
/!\ Automated setup is not being maintained. Follow the instructions on Iteration7ReleaseTest instead.

Verify that JDK version 1.6 and ant is installed on the machine and can be invoked from the command prompt. Execute the below commands and verify that the output resembles the output shown here. The output you get may vary depending on your hardware and the current version of the software, however, as long as the version of the software is not older than the below examples then everything should be just fine.

{{{
tsh@pc214:~$ java -version
java version "1.6.0_03"
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode)
tsh@pc214:~$ javac -version
javac 1.6.0_03
tsh@pc214:~$ ant -version
Apache Ant version 1.7.0 compiled on August 29 2007
}}}
Change directory to some new folder

{{{
svn co https://merkur.statsbiblioteket.dk/svn/doms/trunk doms
cd doms/test/bin
./domssetup.sh ../../../fedora_install
./oaiprovidersetup.sh ../../../fedora_install
}}}
{{{
cd ../../..
mkdir fedora_install/ingest
mkdir fedora_install/ingest/input
mkdir fedora_install/ingest/input/bmech
cp -r doms/data/doms_base_collection fedora_install/ingest/input/base
cp -r doms/test/data/iteration7_testcollection fedora_install/ingest/input/testcollection
cp doms/gentofte_objects/data/fedora_disseminator_object/Gentofte*DigitalObjectBundle.xml fedora_install/ingest/input/bmech
cd doms/ingest
ant release.bin
cp dist/doms-ingest-0.0.1.zip ../../fedora_install/ingest
cd ../../fedora_install/ingest
unzip doms-ingest-0.0.1.zip
rm -f doms-ingest-0.0.1.zip
mv doms-ingest-0.0.1/* .
rm -r doms-ingest-0.0.1}}}
Stop here, if you need to do something before ingest Example to prevent the use of the bitstorage

{{{
mv input/testcollection/data input/testcollection/data2
}}}
{{{
./bin/ingest.sh base
./bin/ingest.sh bmech
./bin/ingest.sh testcollection
cd ../..
cd doms/disseminator_webservice
ant war
cp dist/gentofte_disseminator.war ../../fedora_install/tomcat/webapps
cd ../..
cd fedora_install
./bin/stop.sh
./bin/start.sh
}}}
This command clears all data from the bitstorage on halley. Beware, other might be using it.

{{{
ssh doms@halley "rm -rf stage/* storage/*"
}}}