1 Oliver Spits Out a Finding Aid Using CONTENTdm with a Database Susan Hamburger, Ph.D. Penn State University Libraries Society of American Archivists, August 30, 2007

2 Background “Oliver” homegrown Oracle platform database Filemaker Pro → MS Access Rudimentary export of container list No biog/hist, scope and content narratives → SQL → Oracle Merging of five databases into one Working on exporting EAD-tagged data Finding aids created individually with XMetaL From scratch Stitch together database export of container info plus MS Word for narrative

3 Questions for Discussion How can we automate generating EADs from Oliver? How can we provide a federated search tool for finding aids? What software is out there to use?

4 Considerations Special Collections staff doesn’t know EAD New Processing Coordinator knows EAD Manuscripts Cataloger creates EAD finding aids Library’s Information Technology (I-Tech) staff doesn’t know EAD, barely literate with XML and XSLT Library Dean doesn’t want I-Tech to do development → find out-of-the-box solution

5 Ease of input of finding aid: Deal breaker, Priority 1 (when we go online), Priority 2 (within two years) Search multiple or single EAD fields including ALL of the following Unittitle, persname, corpname, formgenre, famname, subject, scopecontent, bioghist, unit within Special Collections Deal breaker Search results will display the EAD fields: unittitle, unitdate, extent, biography, abstractDeal breaker link to an outline viewDeal breaker link to full/print viewDeal breaker highlight the keyword in context in that displayDeal breaker Display finding aids using XSLT in outline and full text view with keyword highlightsDeal breaker Single and batch inputPriority 1a Search, results display, and finding aid display should be customized/customizable to suit Special Collections Priority 1a Moderate to high performance in speed, usability and site navigation equal to The CATPriority 1b Full text searchingPriority 1b Mark results list for bookbagPriority 1c Mark results list for Priority 1c Mark results list for printPriority 1c User can sort results by: frequency, author, title, date of collectionsPriority 1c Refine search—means performing an additional search on the current set of recordsPriority 2a Paraprofessional inputPriority 2a

6 Search results will: highlight all keywords in context on a separate framePriority 2b Display large result sets by alphabetical chunksPriority 2b display the finding aid’s file sizePriority 2c Search the date field separatelyPriority 2c Ease of input of finding aid: Deal breaker, Priority 1 (when we go online), Priority 2 (within two years)

7 Task Force Finding Aid Platform working group formed 3 from I-Tech, 2 from Special Collections, Manuscripts Cataloger, 1 from Digital Libraries Technology (DLT) Charge: To find and evaluate existing products that meet our criteria and make recommendation for implementation

8 Methodology Survey marketplace Informal queries Society of American Archivists annual meeting RLG conference Posting the question on the archives listserv Searched academic libraries’ websites The survey results consisted of five potential products: Archeon Archivists Toolkit CONTENTdm v. 4.2 DLXS v. 12 XTF

9 Methodology Assess and evaluate the products and determine costs Create a comprehensive set of prioritized criteria for search and display and compatibility with the Library computing environment Two-member groups evaluated products against the criteria Populate evaluation matrix

10 Prioritized Criteria List Graduated criteria: 1 = Required to 6 = Desired Back End 2 Supports Unicode 2 Ability to load full and minimal finding aids 2 Easily customizable end user output/display Specifications 5 Back end user tools for data load, maintenance 1 Product support Licensing issues?

11 Prioritized Criteria List Graduated criteria: 1 = Required to 6 = Desired Rights Management 3 Authorization at collection level and field level Search Functionality 3 Full text searchable across finding aids as a whole 3 Keyword searchable across multiple, selected fields 3 Search across all collections in system or across pre- determined subsets of collections 2 Browse collections 3 Search format and index terms 6 Search by date 3 Persistent navigation (prefer static outline view while scrolling through finding aid) 4 PURLs to individual finding aids

12 Prioritized Criteria List Graduated criteria: 1 = Required to 6 = Desired End User Output Export output/download METS/MODS/Dublin Core 1 Output includes both outline view and full view 1 Search term highlighted in results list (brief) and full finding aid view 1 Search results display 4 EAD fields: Unittitle, unitdate, extent, abstract 6 Large result set, represented in alpha list as intermediate navigation rather than number ranges (e.g., A|B|C …) vs. (1-300, , etc.) 4 Results sorted by relevance, and author, title 6 Results sorted by date 4 Save marked list from result set 4 Print, review, , etc. from marked list 3 Refine search from results list 6 Display finding aid file size 6 ADA Compliant (AD54)

13 Prioritized Criteria List Other desirables, not prioritized Discovery/Sharing OAI Harvesting Findable/crawlable by RLG spiders, etc. Findable by Google, etc. Compatible within Course Management tools Supports inter-institutional sharing of collections/items, etc. Individual contributions of material to library collections (p2p-like) Federated search support Ability to add link to CAT record from Finding Aid metadata

14

15 Evaluation of Software Archeon and Archivists’ Toolkit did not meet critical search and display criteria XTF did not meet criteria for technical support DLXS met all criteria established for a search and display, but would require significant local development to meet criteria for back-end dispersed processing of finding aids (e.g., non- technical staff at any location can process material) The current version of CONTENTdm v.4.2 would require significant local development to accommodate the large Special Collections finding aids Discussion with the developer at CONTENTdm revealed that an improved version CONTENTdm which fully supports XML ingest, indexing, output and large field sizes is in development and will be announced this summer

16 Recommendations Continue to develop the export function to generate valid EAD finding aids from Oliver Participate in the development of CONTENTdm v.n βeta Evaluate CONTENTdm v.n βeta at its production release against defined criteria If the production release meets our criteria Implement CONTENTdm as our production system by January 2008 If the production release does not meet our criteria Recommend a revised investigation of existing/new products

17 Projected Plan Work with CONTENTdm in the βeta trial of CONTENTdm v.n and launch digital finding aids in January 2008 at the latest Timeline: February – July 2007 Create Best Practices Guidelines Clean up data in Oliver Develop export tools to generate EAD finding aids Develop XSLT stylesheets July – November 2007 Work with CONTENTdm on development and βeta testing of new release Submit our list of criteria to CONTENTdm as they initiate their development January 2008 Launch next release of CONTENTdm and put finding aids into production or revert to backup plan If βeta version fails to meet expected timeline or criteria, especially for ingest, XML mapping, and large field size, platform project evaluation team will confirm these circumstances with OCLC and reevaluate available platform products against existing criteria and recommend to IT Priorities

18 Ongoing Development Usability testing Continued modification of output style sheets in response to usability testing recommendations Regular scan of marketplace to monitor new products

19 Resources Needed Staffing Implementation team with representatives from I-Tech and Special Collections to move this plan forward Digital Library Technologies support will be required if the βeta release is available to be installed locally on a development server

20 Resources Needed Training Oracle 10g: XML Fundamentals training to support the work involved in extracting EAD2002 XML finding aids out of the Oliver database XSLT refresher training may be needed for I-Tech personnel

21 Conclusion Because of infrastructure and policies, we had to select product with least amount of customization and programming DLXS is hard to ingest, but looks good and functions well CONTENTdm is easy to ingest, but doesn’t have functionality required for finding aids Open source software requires dedicated staff with expertise we don’t have

22 Recommendations Determine your needs Systematically evaluate products Have a timeline goal for decision making Know your technical limitations Include key personnel in planning

23 Contact Susan Hamburger, Ph.D. The Pennsylvania State University Paterno Library Cataloging and Metadata Services University Park, PA FAX

24 Here are Oliver and some documents Oliver database Special Collections finding aids Web pages Finding Aids Platform Product Details [MS Word → HTML document]

25

26

27

28

29

30 Finding Aids Platform Product Details Appendix A appendixa.htm