Miscellaneous Course material prepared by Greenstone Digital Library Project University of Waikato, New Zealand

 Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

 Very difficult!  (too competitive?) 1.Design 2.Manual metadata 3.Format statements  Getting help  Getting professional help  (Friends Of Greenstone) Assignment

 Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

 Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

The Greenstone OAI server  Runs as a CGI program called oaiserver –Greenstone installation –OAI server  Requires a full webserver (not “local library” version)  Configuration file: etc/oai.cfg in the Greenstone filespace –repository name and version (OAI 1.1 or 2.0) –collections to be made accessible to OAI clients –metadata mapping file into DC (server only supports DC) OAI 1.1 OAI 2.0

Download panel: downloading from the web

Downloading from OAI

Downloading using Z39.50/SRW

 Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

Administration Pages  Access the administration pages by clicking on the button on the Greenstone home page  Default admin user password: “admin”

Administration Pages View usage and error logs List of collections Edit main.cfg file

 Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

User Management  Users are added/edited/removed using the links on the Administration page Show users, with links to edit or remove them (admin only) Add a new user to the system (admin only) Change password (any user with an account)

User Management  User details: –Username –Password –Groups (controls what the user can do) –Comment

 Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

Restricting Access to Documents  Restrict users to view certain documents  Define groups and users in admin pages  Restrict levels –collection (default) => the full collection –document => individual documents

Restricting Access to Documents  How to: –Set up in collection configuration file (collect_name/etc/collect.cfg) –authenticate directive –auth_groups directive –Restrict documents  Public_documents  Private_documents –Use document identifiers  Look at URL =>d argument

Restrict Collection

Authenticate Collection

Documents Browsing

Restrict Document

Public_documents Document: HASHf48ef4f9081c1a3bbc7c68

Authenticate document

Document Display

 Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

DSpace  Export: Greenstone -> DSpace  Import: DSpace -> Greenstone

News flash The Depositor

The Depositor

 Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

METS–Metadata Encoding and Transmission  Aim: encode metadata for any digital material –electronic texts, still images, digitized video, sound files  Specifies a framework that works with other XML standards  Coherent overall structure for encoding all metadata –descriptive, administrative, structural used to describe digital library objects  Metadata either –embedded within METS structure –or held in external files that METS references  Internal identifiers link all the object’s components extends the “conventional” use of the term metadata to encompass document structure and content as well

METS format has 7 top-level sections:  METS Header  Descriptive Metadata  Administrative Metadata  File Section  Structural Map  Structural Links  Behavior METS–Metadata Encoding and Transmission

METS: Structural Map  Only compulsory section  Describes overall structure of the object (its structural metadata)  Sample: [Contents of Chapter 2 omitted]

METS: Descriptive Metadata  Descriptive metadata held in sections  Structural map references using ID attribute  Descriptive metadata can be either be –held in external files referenced from within the METS –or embedded directly with it <mdRef LOCTYPE="URN" MIMETYPE="application/xml" MDTYPE="EAD" LABEL="Berol Collection Finding Aid">urn:x-nyu:fales1735 Alice's Adventures in Wonderland Lewis Carroll between 1872 and 1890 McCloughlin Brothers text

METS: Example 1

METS: Example 1

dmdSec MODS fileSec URL: page1.jpg URL: page2.jpg structMap div DMDID=1 div FILEID=1 div FILEID=2 Catalog record (MARC) Scanned images (JPEG) Structural metadata METS: Example 1

div TYPE=“score” 15me. & 16me. Nocturno composé par Frederic Chopin 2000pix wide JPEGs image 1 image 2 image 3 … 700pix wide JPEGs image 1 image 2 image 3 … div ORDER=“1” div ORDER=“2” div ORDER=“3” ORDERLABEL=“Page 1” LABEL=“Nocturne, no.15” Descriptive metadata File list Structure map METS: Example 1

MARC: MODS: ‡a Chopin, Frédéric, ‡d ‡a 15me. & 16me. nocturno / ‡c [composé par Frederic Chopin]. 300 ‡a 10 p. of music ; ‡c 33 cm. 15me. & 16me. nocturno Chopin, Frédéric p. of music ; 33 cm. [composé par Frederic Chopin] METS: Example 1 – MODS

METS: Example 2 Exporting a collection as METS –Use Greenstone export.pl in combination with “–saveas” to generate METS compliant files Tutorial exercise #21

METS: Example 2 One folder for each document: HASH0101.dir/ HASH0102.dir/ HASH0103.dir/ HASH010c.dir/... Sample document folder (HASH0101.dir): 1500s.gif docmets.xml doctxt.xml Sample docmets.xml (as before): ab.html...

METS: Issues In reality:  Lots of people generate METS  Not so many people read it in  Difficult to handle all possible METS files –e.g. can include any metadata standard with an XML manifestation—even ones that haven’t been devised yet!  Subsets have been institutionalized as METS “profiles” –e.g. Greenstone profile  Two systems may apparently use METS but fail to interoperate meaningfully You have to read the fine print

 Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

MODS – Metadata Object Description Schema Motivation:  Dublin Core “too light” for some applications  MARC, at the other extreme, is cumbersome, was not really designed with electronic resources in mind, numerous variants to worry about …  MODS tries to address the conflicting needs of interoperability and precision

MODS: Syntax Overview  19-top level elements for describing objects  64 further sub-elements under these  Covers standard bibliographic facets, e.g.  titles,  names of creators and contributors,  subject and classification numbers,  Plus provision for  physical description,  information on access restrictions, genre, and so on.  Extension mechanism: can incorporate other XML documents (e.g. related objects)  Includes facilities for authority control, and is fully granular in the level of descriptions it provides.

MODS: Top-level Elements  titleInfo  name  genre  targetAudience  classification  subject  typeOfResource  language  abstract  tableOfContents  originInfo  physicalDescription  note  identifier  relatedItem  location  accessCondition  recordInfo  extension

MODS: Sample Sound and fury : the making of the punditocracy / Alterman, Eric creator text bibliography nyu Ithaca, N.Y Cornell University Press c

MODS: Sample … monographic eng print vii, 322 p. ; 23 cm. Eric Alterman. Includes bibliographical references (p ) and index. Journalism Political aspects United States. …

MODS: Sample … PN4888.P6 A / (pbk. : acid-free, recycled paper) DLC

MODS: Extended Example Extended example:  Using MODS records, build a Greenstone digital library collection based on the Library of Congress’ British Cartoons collection (part of the American Memory project). Details:  MODS metadata records exported from LOC’s OAI server  Greenstone configured to support: –Browse by title, author, and subject –Search by title, subject and all fields –Display summary of MODS record nicely formatted –Hyperlink to original resource

MODS: Extended Example

MODS mapping into Greenstone using XSLT (specify as argument to OAIPlug): … : …

 Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

Dealing with CDS/ISIS sample_files --> isis build; collection is useless Search Indexes: use Photographer^all, Country^all; Notes^all; remove useless Source and Titles indexes Browsing Classifiers: add AZList for Photographer; remove useless Title and Source classifiers Format Features: change the Vlist format statement to display Photographer and Notes metadata remove Detach and No Highlighting (DocumentButtons) Format Features: link to the raw record (stored as ISISRawRecord metadata) by editing DocumentText

Dragging in a CDS/ISIS database

Fix search indexes

Choice of search engines

Fix browsing classifiers

Fix format features

The result

Show/hide CDS record

Multilingual support  Greenstone works well with English documents, but what about documents in other languages? –Greenstone uses Unicode internally, so it can handle these too –Searching and browsing works (with a few limitations)

Multilingual issues: build-time  Source filenames should be in ASCII (English)  GLI metadata entry: Unicode only –Install Microsoft’s IME (Input Method Editor) for your language

Multilingual issues: build-time  GLI: Make sure you’re using a Unicode font that includes your language!

Multilingual issues: build-time  Source file content: –Unicode, or –Encoding that Greenstone can convert to Unicode

Multilingual issues: run-time  Browser must have the correct fonts installed –Not usually a problem these days (Unicode fonts)  Browsing: –Many of Greenstone’s classifiers don’t support Unicode (GenericList is one that does) –Sorting may be incorrect

Multilingual issues: run-time  Searching: –Enter search terms in Unicode (using IME), or –Select native encoding from the Preferences page, and enable “Auto-select” or select the native encoding in the browser –Stemming: English only

Language Interfaces  English:

Language Interfaces  Spanish:

Language Interfaces  Russian:

Language Interfaces  Text fragments must be translated  Button images must be generated  Maintenance nightmare! –About 750 text fragments in the user interface –No one person knows all languages –Little/no overlap between developers and translators –Developers and translators geographically dispersed –Translations must keep pace with software developments –We need an automated system to help…

Language Interfaces  Greenstone Translator Interface (GTI) –Web-based system –Keeps track of which text fragments have changed, or are new –Translators are presented with pages containing the text fragments to update or translate –When submitted, these are saved to a separate installation of Greenstone –Mistakes in existing translations can be corrected –Translators can download the translated files –Translators must log in to add/edit translations

GTI  Select a file to work on:

GTI  Updating a translation:

Greenstone in Marathi मराठी ग्रीनस्टोन

Objectives  To create digital library of Marathi and in other languages using Greenstone  To translate Greenstone in Marathi  To create digital library in Marathi language Shubhada Nagarkar Department of Library and Information Science Jaykar Library, University of Pune, Pune – Phone: /

Requirements Hardware  Pentium I onwards Software  Operating system : –Microsoft windows –language pack to install Indic languages including Arial Unicode Ms font  IIS server / personal web server  Windows 95/98/Me/NT/2000/XP  Greenstone 2.71

Requirements for Translation Translation can be done by two ways : 1.Excel sheets or 2.Access to GTI at

Excel sheets Following Microsoft Excel Sheets provided by Greenstone developers for translation into Marathi : –Greenstone Interface (Core) –Greenstone Interface (Auxiliary) –GLI Dictionary –GLI Help –Perl Modules –Greenstone.org Translation in Marathi (Devnagary Script)

Implementation  Translated Excel sheets sent to Greenstone developer team for conversion of the same into.dm files  Placement of.dm files into appropriate directories  Setting up preferences of language from GLI (Greenstone Librarian Interface) Translations completed Greenstone Interface (Core) (100%) Greenstone Interface (Auxiliary) (100%) Partially completed GLI Dictionary GLI Help Perl Modules Greenstone.org

Greenstone Translation Interface (GTI) GTI URL Marathi

GTI … Select Greenstone Interface option

GTI … Type username and password provided by Greenstone

GTI … (As we have completed first two screens, this is a sample screen for GLI dictionary) Translated files Translated files (.dm files)

Translated Dictionary File

Placement of translated files Translated Greenstone macros files (marathi.dm, marathi2.dm for Greenstone Interface and Auxillary) need to be placed in appropriate directories. The macrofiles should be put in the Greenstone "macros" folder Then need to edit the Greenstone "etc/main.cfg" file and add Language shortname=mr longname=Marathi default_encoding=utf-8 with the other Language lines, and add "marathi.dm" and "marathi2.dm" to the end of the list of macrofiles

Copy macros files into c:\program files\greenstone\macro direcory

Edit etc/main.cfg (available in c:\program files\greenstone directory) Placement of marathi.dm and marathi2.dm files in main.cfg file Open main.cfg file in notepad

Place following line in main.cfg file Language shortname=mr longname=Marathi default_encoding=utf-8

Marathi Interface

Kia papapounamu te moana may peace and calmness surround you, may you reside in the warmth of a summer’s haze, may the ocean of your travels be as smooth as the polished greenstone. Greenstone DL software greenstone.org New Zealand Digital Library Projectnzdl.org kia hora te marino, kia tere te karohirohi, kia papapounamu te moana