1. Miscellaneous Course material prepared by Greenstone Digital Library Project University of Waikato, New Zealand

2.  Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

3.  Very difficult!  (too competitive?) 1.Design 2.Manual metadata 3.Format statements  Getting help  Getting professional help  (Friends Of Greenstone) Assignment

4.  Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

5.  Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

6. The Greenstone OAI server  Runs as a CGI program called oaiserver –Greenstone installation http://.../cgi-bin/gsdl –OAI server http://.../cgi-bin/oaiserver?verb=Identify http://.../cgi-bin/oaiserver?verb=ListSets http://.../cgi-bin/oaiserver?verb=ListIdentifiers&set=xxx http://.../cgi-bin/oaiserver?verb=ListIdentifiers&set=xxx&metadataPrefix=oai_dc http://.../cgi-bin/oaiserver?verb=ListRecords&set=xxx&metadataPrefix=oai_dc http://.../cgi-bin/oaiserver?verb=GetRecord&identifier=xxx&metadataPrefix=oai_dc  Requires a full webserver (not “local library” version)  Configuration file: etc/oai.cfg in the Greenstone filespace –repository name and version (OAI 1.1 or 2.0) –collections to be made accessible to OAI clients –metadata mapping file into DC (server only supports DC) OAI 1.1 OAI 2.0

7. Download panel: downloading from the web

8. Downloading from OAI

9. Downloading using Z39.50/SRW

10.  Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

11. Administration Pages  Access the administration pages by clicking on the button on the Greenstone home page  Default admin user password: “admin”

12. Administration Pages View usage and error logs List of collections Edit main.cfg file

13.  Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

14. User Management  Users are added/edited/removed using the links on the Administration page Show users, with links to edit or remove them (admin only) Add a new user to the system (admin only) Change password (any user with an account)

15. User Management  User details: –Username –Password –Groups (controls what the user can do) –Comment

16.  Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

17. Restricting Access to Documents  Restrict users to view certain documents  Define groups and users in admin pages  Restrict levels –collection (default) => the full collection –document => individual documents

18. Restricting Access to Documents  How to: –Set up in collection configuration file (collect_name/etc/collect.cfg) –authenticate directive –auth_groups directive –Restrict documents  Public_documents  Private_documents –Use document identifiers  Look at URL =>d argument

19. Restrict Collection

20. Authenticate Collection

21. Documents Browsing

22. Restrict Document

23. Public_documents Document: HASHf48ef4f9081c1a3bbc7c68

24. Authenticate document

25. Document Display

26.  Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

27. DSpace  Export: Greenstone -> DSpace  Import: DSpace -> Greenstone

28. News flash The Depositor

29. The Depositor

37.  Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

38. METS–Metadata Encoding and Transmission  Aim: encode metadata for any digital material –electronic texts, still images, digitized video, sound files  Specifies a framework that works with other XML standards  Coherent overall structure for encoding all metadata –descriptive, administrative, structural used to describe digital library objects  Metadata either –embedded within METS structure –or held in external files that METS references  Internal identifiers link all the object’s components extends the “conventional” use of the term metadata to encompass document structure and content as well

39. METS format has 7 top-level sections:  METS Header  Descriptive Metadata  Administrative Metadata  File Section  Structural Map  Structural Links  Behavior METS–Metadata Encoding and Transmission

40. METS: Structural Map  Only compulsory section  Describes overall structure of the object (its structural metadata)  Sample: [Contents of Chapter 2 omitted]

41. METS: Descriptive Metadata  Descriptive metadata held in sections  Structural map references using ID attribute  Descriptive metadata can be either be –held in external files referenced from within the METS –or embedded directly with it <mdRef LOCTYPE="URN" MIMETYPE="application/xml" MDTYPE="EAD" LABEL="Berol Collection Finding Aid">urn:x-nyu:fales1735 Alice's Adventures in Wonderland Lewis Carroll between 1872 and 1890 McCloughlin Brothers text

42. METS: Example 1

49. METS: Example 1

50. dmdSec MODS fileSec URL: page1.jpg URL: page2.jpg structMap div DMDID=1 div FILEID=1 div FILEID=2 Catalog record (MARC) Scanned images (JPEG) Structural metadata METS: Example 1

51. div TYPE=“score” 15me. & 16me. Nocturno composé par Frederic Chopin 2000pix wide JPEGs image 1 image 2 image 3 … 700pix wide JPEGs image 1 image 2 image 3 … div ORDER=“1” div ORDER=“2” div ORDER=“3” ORDERLABEL=“Page 1” LABEL=“Nocturne, no.15” Descriptive metadata File list Structure map METS: Example 1

52. MARC: MODS: 100 1 ‡a Chopin, Frédéric, ‡d 1810-1849. 245 10 ‡a 15me. & 16me. nocturno / ‡c [composé par Frederic Chopin]. 300 ‡a 10 p. of music ; ‡c 33 cm. 15me. & 16me. nocturno Chopin, Frédéric 1810-1849 10 p. of music ; 33 cm. [composé par Frederic Chopin] METS: Example 1 – MODS

53. METS: Example 2 Exporting a collection as METS –Use Greenstone export.pl in combination with “–saveas” to generate METS compliant files Tutorial exercise #21

54. METS: Example 2 One folder for each document: HASH0101.dir/ HASH0102.dir/ HASH0103.dir/ HASH010c.dir/... Sample document folder (HASH0101.dir): 1500s.gif docmets.xml doctxt.xml Sample docmets.xml (as before): 1117427985 ab.html...

55. METS: Issues In reality:  Lots of people generate METS  Not so many people read it in  Difficult to handle all possible METS files –e.g. can include any metadata standard with an XML manifestation—even ones that haven’t been devised yet!  Subsets have been institutionalized as METS “profiles” –e.g. Greenstone profile http://www.loc.gov/standards/mets/profiles/00000006.xml  Two systems may apparently use METS but fail to interoperate meaningfully You have to read the fine print

56.  Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

57. MODS – Metadata Object Description Schema Motivation:  Dublin Core “too light” for some applications  MARC, at the other extreme, is cumbersome, was not really designed with electronic resources in mind, numerous variants to worry about …  MODS tries to address the conflicting needs of interoperability and precision

58. MODS: Syntax Overview  19-top level elements for describing objects  64 further sub-elements under these  Covers standard bibliographic facets, e.g.  titles,  names of creators and contributors,  subject and classification numbers,  Plus provision for  physical description,  information on access restrictions, genre, and so on.  Extension mechanism: can incorporate other XML documents (e.g. related objects)  Includes facilities for authority control, and is fully granular in the level of descriptions it provides.

59. MODS: Top-level Elements  titleInfo  name  genre  targetAudience  classification  subject  typeOfResource  language  abstract  tableOfContents  originInfo  physicalDescription  note  identifier  relatedItem  location  accessCondition  recordInfo  extension

60. MODS: Sample Sound and fury : the making of the punditocracy / Alterman, Eric creator text bibliography nyu Ithaca, N.Y Cornell University Press c1999 1999

61. MODS: Sample … monographic eng print vii, 322 p. ; 23 cm. Eric Alterman. Includes bibliographical references (p. 291-312) and index. Journalism Political aspects United States. …

62. MODS: Sample … PN4888.P6 A48 1999 071/.3 0801486394 (pbk. : acid-free, recycled paper) 99042030 DLC 990730 20000406144503.0 11761548

63. MODS: Extended Example Extended example:  Using MODS records, build a Greenstone digital library collection based on the Library of Congress’ British Cartoons collection (part of the American Memory project). Details:  MODS metadata records exported from LOC’s OAI server  Greenstone configured to support: –Browse by title, author, and subject –Search by title, subject and all fields –Display summary of MODS record nicely formatted –Hyperlink to original resource

64. MODS: Extended Example

75. MODS mapping into Greenstone using XSLT (specify as argument to OAIPlug): … : …

76.  Assignment  OIDs  OAI  Administration pages  User management  Restricting access to docs  DSpace  METS  MODS  CDS/ISIS Agenda

77. Dealing with CDS/ISIS sample_files --> isis build; collection is useless Search Indexes: use Photographer^all, Country^all; Notes^all; remove useless Source and Titles indexes Browsing Classifiers: add AZList for Photographer; remove useless Title and Source classifiers Format Features: change the Vlist format statement to display Photographer and Notes metadata remove Detach and No Highlighting (DocumentButtons) Format Features: link to the raw record (stored as ISISRawRecord metadata) by editing DocumentText

78. Dragging in a CDS/ISIS database

79. Fix search indexes

80. Choice of search engines

81. Fix browsing classifiers

82. Fix format features

83. The result

84. Show/hide CDS record

86. Multilingual support  Greenstone works well with English documents, but what about documents in other languages? –Greenstone uses Unicode internally, so it can handle these too –Searching and browsing works (with a few limitations)

87. Multilingual issues: build-time  Source filenames should be in ASCII (English)  GLI metadata entry: Unicode only –Install Microsoft’s IME (Input Method Editor) for your language

88. Multilingual issues: build-time  GLI: Make sure you’re using a Unicode font that includes your language!

89. Multilingual issues: build-time  Source file content: –Unicode, or –Encoding that Greenstone can convert to Unicode

90. Multilingual issues: run-time  Browser must have the correct fonts installed –Not usually a problem these days (Unicode fonts)  Browsing: –Many of Greenstone’s classifiers don’t support Unicode (GenericList is one that does) –Sorting may be incorrect

91. Multilingual issues: run-time  Searching: –Enter search terms in Unicode (using IME), or –Select native encoding from the Preferences page, and enable “Auto-select” or select the native encoding in the browser –Stemming: English only

92. Language Interfaces  English:

93. Language Interfaces  Spanish:

94. Language Interfaces  Russian:

95. Language Interfaces  Text fragments must be translated  Button images must be generated  Maintenance nightmare! –About 750 text fragments in the user interface –No one person knows all languages –Little/no overlap between developers and translators –Developers and translators geographically dispersed –Translations must keep pace with software developments –We need an automated system to help…

96. Language Interfaces  Greenstone Translator Interface (GTI) –Web-based system –Keeps track of which text fragments have changed, or are new –Translators are presented with pages containing the text fragments to update or translate –When submitted, these are saved to a separate installation of Greenstone –Mistakes in existing translations can be corrected –Translators can download the translated files –Translators must log in to add/edit translations

97. GTI  Select a file to work on:

98. GTI  Updating a translation:

99. Greenstone in Marathi मराठी ग्रीनस्टोन

100. Objectives  To create digital library of Marathi and in other languages using Greenstone  To translate Greenstone in Marathi  To create digital library in Marathi language Shubhada Nagarkar Department of Library and Information Science Jaykar Library, University of Pune, Pune – 411007 Phone: 91 020 25695549 Email: shubha@unipune.ernet.in / shubha.nagarkar@gmail.comshubha@unipune.ernet.inshubha.nagarkar@gmail.com

101. Requirements Hardware  Pentium I onwards Software  Operating system : –Microsoft windows –language pack to install Indic languages including Arial Unicode Ms font  IIS server / personal web server  Windows 95/98/Me/NT/2000/XP  Greenstone 2.71

102. Requirements for Translation Translation can be done by two ways : 1.Excel sheets or 2.Access to GTI at http://puka.cs.waikato.ac.nz/cgi-bin/gti/library?a=gti&p=home

103. Excel sheets Following Microsoft Excel Sheets provided by Greenstone developers for translation into Marathi : –Greenstone Interface (Core) –Greenstone Interface (Auxiliary) –GLI Dictionary –GLI Help –Perl Modules –Greenstone.org Translation in Marathi (Devnagary Script)

104. Implementation  Translated Excel sheets sent to Greenstone developer team for conversion of the same into.dm files  Placement of.dm files into appropriate directories  Setting up preferences of language from GLI (Greenstone Librarian Interface) Translations completed Greenstone Interface (Core) (100%) Greenstone Interface (Auxiliary) (100%) Partially completed GLI Dictionary GLI Help Perl Modules Greenstone.org

105. Greenstone Translation Interface (GTI) GTI URL http://puka.cs.waikato.ac.nz/cgi-bin/gti/library?a=gti&p=home Marathi

106. GTI … Select Greenstone Interface option

107. GTI … Type username and password provided by Greenstone

108. GTI … (As we have completed first two screens, this is a sample screen for GLI dictionary) Translated files Translated files (.dm files)

109. Translated Dictionary File

110. Placement of translated files Translated Greenstone macros files (marathi.dm, marathi2.dm for Greenstone Interface and Auxillary) need to be placed in appropriate directories. The macrofiles should be put in the Greenstone "macros" folder Then need to edit the Greenstone "etc/main.cfg" file and add Language shortname=mr longname=Marathi default_encoding=utf-8 with the other Language lines, and add "marathi.dm" and "marathi2.dm" to the end of the list of macrofiles

111. Copy macros files into c:\program files\greenstone\macro direcory

112. Edit etc/main.cfg (available in c:\program files\greenstone directory) Placement of marathi.dm and marathi2.dm files in main.cfg file Open main.cfg file in notepad

113. Place following line in main.cfg file Language shortname=mr longname=Marathi default_encoding=utf-8

114. Marathi Interface

116. Kia papapounamu te moana may peace and calmness surround you, may you reside in the warmth of a summer’s haze, may the ocean of your travels be as smooth as the polished greenstone. Greenstone DL software greenstone.org New Zealand Digital Library Projectnzdl.org kia hora te marino, kia tere te karohirohi, kia papapounamu te moana