1. Putting it all together for Digital Assets Jon Morley Beck Locey

2. LDS Church History Library Our collections consist of manuscripts, books, Church records, photographs, oral histories, architectural drawings, pamphlets, newspapers, periodicals, maps, microforms, and audiovisual materials. The collection continues to grow annually and is a prime resource for the study of Church history. Our collections contains approximately: 270,000 books, pamphlets, magazines, and newspapers 240,000 collections of original, unpublished records (journals, diaries, correspondence, minutes, etc.) 3.5 million blessings for Church members 13,000 photograph collections 23,000 audiovisual items

3. Digitization Objective Patrons – Globalization – To provide access to library content to 14 million church members and the public throughout worldwide. Remote sites for new content. Internal Operations – To create an automated digital pipeline from digitizing content to patron consumption. Extending to crowdsourcing in future.

4. Agenda The first half of this presentation will focus on “what” we want to accomplish with an automated digital pipeline. The second half will focused on “how” we built the digital pipeline.

5. DIGITAL PIPELINE What we want to accomplish

6. Digital Content Aleph (Master Record) Primo (Discovery) Rosetta EAD Tool (Encoded Archival Description) Church History Library Physical Assets 20102011

7. Rosetta Dual Role Two instances of Rosetta 1. DRPS – Digital Records Preservation System (dark archive) 2.DCMS – Digital Content Management System (public display)

8. Digital Pipeline Rosetta EAD Tool (Encoded Archival Description) Aleph (Master Record) Primo Ingest Collection PID555 tag Harvest & Index 855 tag

12. EAD Tool Interface

15. Click on Thumbnail

18. Patron can a item to be digitized.

19. Organizing and displaying Collections

20. EAD Tool (Encoded Archival Description) A finding aid that adds a viewable structure to a collection. -Many large, complex collections -Aleph stores only the collection level descriptive metadata -Add / edit / delete component level metadata -Rosetta 3.0 doesn’t have enough functionality for our collections.

21. EAD Tool - Staff View

23. EDIT Drag and drop XML or CSV files

30. Challenges Configuration is fairly complex within and between systems Large Batch Ingest has been difficult Re-ingesting content into Rosetta has been problematic (preservation system) Collections Management –Built EAD tool to manage collections –Serials? PDF Content –Progressive PDF download in Adobe Reader X is not fully supported until Rosetta v3 – 2 searches to see text within a PDF

31. Successes –Ingested over 57,000 Family history books ranging from 1 MB to 800 MB for the Family History department. (books.familysearch.org) –Ingested 700,000 files. Most are linked to collections for Church History library. –Consistent user experiences with large files –Good response times 75,000 views / Month –Once the pipelines are established, they work well. –Able to create customizable solutions for multiple institutions.

32. Under Development –Restricted content Based on IP addresses authentication –Multiple viewing experiences (Responsive Design) –Reporting capabilities –Monitor usage

33. DIGITAL PIPELINE How we glued it all together

34. Digital Pipeline Aleph (Master Record) PrimoRosetta EAD Tool (Encoded Archival Description) Church History Library

35. Catalog in Aleph Physical assets are cataloged in Aleph (collection level). Staff assigns a call number and Aleph assigns a BIB number. While browsing, a patron requests that the asset be digitized. Aleph Call #: MS 2877 4 BIB #: 000114027

36. Digitized into Rosetta Rosetta The asset(s) from the Aleph collection are digitized. A custom tool (SIP tool) queries the Aleph SRU server using BIB # to get collection level metadata. Digitized assets, BIB # (CMS ID) and metadata are ingested into Rosetta. PIDs get assigned. Aleph SRU BIB #: 000114027 Call #: MS 2877 4 Title: Parley P. Pratt… 000114027 MS 2877 4 Parley P. Pratt… + Query Response

37. Rosetta to Aleph / EAD Tool Every night a custom script (cron job) queries Rosetta using the OAI harvester. The PIDs and item titles from Rosetta are inserted into the EAD Tool or prepared for Aleph (856 tags). Every night a custom script (job_list) ingests the metadata into Aleph. Rosetta OAI Aleph Query Response Dublin Core EAD Tool

38. From Aleph to Primo Every night Aleph publishes all data sets to Primo (publish- 06 in job_list). Aleph data sets include collection level metadata plus 856 tags (Rosetta PID) or 555 tags (EAD link). During harvest, Primo creates links from 555 or 856 tags which point to the EAD Tool and Rosetta respectively. Publish MARC XML Aleph Primo

39. Digital Pipeline Notes Aleph BIB # provides the linkage between the various systems. –Call # provides the reference Linux scripting glues it all together. –Cron jobs, job_list, wget and custom logs work together to get data, re-format data, move data and start new jobs. –NFS shares allow us to move data around easily. Rights are a bit of a hassle. Timing matters. –Pull from Rosetta, update EAD Tool, send to Aleph, publish to Primo, and run Primo pipes.

40. Backup Slides

41. Rosetta OAI Harvester Linux script using wget requests Rosetta data from last 24 hours –Query: by publication set with from / until –Response: Dublin Core Wget http://..org/oaiprovider/request?verb=ListRecords& metadataPrefix=oai_dc&set= &from=2012-08-20T20:00:00Z& until=2012-08-21T19:59:59Z Rosetta OAI Query Response Linux script

42. Aleph SRU Server Aleph SRU server response to Rosetta –Query: by BIB # (CMIS ID) –Response: Dublin Core (or MARC XML) Query Response RosettaAleph SRU

43. Aleph SRU Server URL http://..org:5661/ ?version=1.1&operation=searchRetrieve... …&query=dc.callno=“MS 318”&maximumRecords=1 …&query=rec.id=000082419&maximumRecords=1 …&query=dc.title=“Book of Mormon”&maximumRecords=5 …&query=dc.subject=Apostles&maximumRecords=10 …&query=dc.creator=“John Taylor”&maximumRecords=10