1. The New DRS: Plan for Metadata Migration Harvard Library & Library Technology Services February 26, 2014

2. Agenda Welcome and introduction …... Franziska Frey Migration challenges.…………... Randy Stern Creating the plan ………………..... Kate Bowers Walkthrough of plan …………….. Andrea Goethals Imaging Services: minimizing disruption……….… Bill Comstock Wrap-up & next steps ………...… Kate Bowers, Andrea Goethals Q & A ……………………………….…... All

3. WELCOME & INTRODUCTION Franziska Frey, Tracey Robinson

4. The DRS Advisory Group… …provides oversight and guidance during the rollout phase of the DRS2 project and ensures that the user community of active DRS depositors and content owners contributes to decisions about the rollout. Amy Benson Kate Bowers Bill Comstock Franziska Frey (chair) Andrea Goethals Wendy Gogel Tracey Robinson Randy Stern

5. Why a New DRS? Upgrade to best-in-breed technologies Adopt digital preservation best practices and standards Preserve metadata better Improve collection management Support preservation planning & activities Improve access to content & metadata Support more formats & genres

6. Preservation Capability Before and After the DRS2 Project Level OneLevel TwoLevel ThreeLevel Four Storage & Geographic Location File Fixity and Data Integrity Information Security Metadata File Formats = already compliant= will be compliant after the DRS2 project Based on the NDSA Levels of Digital Preservation

7. Evolution of the DRS 2000200220032004200520062007200820092010201120122001 Current DRS in production New DRS in production DRS enhancements New DRS infrastructure development 201320142015 New DRS metadata migration & user adoption

8. New DRS - Completed 20092010 20112012 convened DRS Advisory Group software in production 20132014 2015 users trained, phase 1 hardware in production migrated content to new hardware Infrastructure Development Metadata Migration & User Adoption Fedora assessment DuraCloud pilot test early releasebeta 1beta 2 beta 3 first object deposited to the new DRS

9. New DRS - Upcoming 20092010 2011201220132014 2015 Infrastructure Development Metadata Migration & User Adoption metadata migration tools created metadata migrated users moved

10. MIGRATION CHALLENGES Randy Stern

11. Why “Metadata” Migration? Why not “content” migration?

12. Pre-migration DRS Content Current DRS Database Current DRS Database

13. Post-migration DRS Content Current DRS Database New DRS Database New DRS Index New DRS Object Descriptors

14. New DRS Data Model Not a simple metadata conversion A new DRS object is a logical intellectual entity that unifies multiple DRS files – Still image objects - archival and production masters, and deliverables including thumbnails – Audio objects - archival and production masters and deliverables – PDS objects - page image and text files

15. Object Descriptors METS files generated for each object – Standards-based internal schemas (PREMIS, MODS, MIX, etc.) Metadata gathered from multiple sources – Current DRS database – Every content file – HOLLIS records

16. Technical Challenges Many formats – Images, audio, text, digitized books, web sites, documents, biomedical image stacks, opaque files Unique migration rules per format – technical metadata, roles, relationships Large (>5000 file) PDS documents 45+ million DRS files

17. Technical Challenges At 1 sec/file, 45 million files would take 520 days! We are designing the migration software tools for parallel processing We are configuring multiple servers to run the migration

19. CREATING THE PLAN Kate Bowers

20. Formulating a Migration Strategy Analysis of: – DRS content Technical (relationships, etc. for building objects) Volume and type by repository – metadata for mapping – user activity in the DRS – survey of highest volume, active users – training and testing registration lists

21. Migration Strategy Factors Combines needs of users with technical requirements User sequencing will be based on: – Current deposit & administrative activity – Level of preparation (training and participation in beta testing)

22. WALKTHROUGH OF PLAN Andrea Goethals

23. Migrating Content in 5 Stages Migrate 1 st : Tier 1 content Migrate 2 nd : Tier 2 content Migrate 3 rd : Tier 3 content Migrate 4 th : Tier 4 content Migrate 5 th : Tier 5 content

24. Migrating Content in 5 Stages Migrate 1 st : Tier 1 content Migrate 2 nd : Tier 2 content Migrate 3 rd : Tier 3 content Migrate 4 th : Tier 4 content Migrate 5 th : Tier 5 content simpler objects more complex objects

25. Migrating Content in 5 Stages Migrate 1 st : Tier 1 content Migrate 2 nd : Tier 2 content Migrate 3 rd : Tier 3 content Migrate 4 th : Tier 4 content Migrate 5 th : Tier 5 content dependencies between tiers dependencies within tiers

26. Migrating Content in 5 Stages TierContent 1Text (Methodology, ESRI World File), Document, Color Profile, Target Image 2PDS Document, Still Image 3Audio, Text (SMIL) 4Web Harvest, Opaque Container 5Biomedical Image; Google Document Container 1, 2, 3

27. Migrating Content in 5 Stages TierContent 1Text (Methodology, ESRI World File), Document, Color Profile, Target Image 2PDS Document, Still Image 3Audio, Text (SMIL) 4Web Harvest, Opaque Container 5Biomedical Image; Google Document Container 1, 2, 3

28. Migrating Content in 5 Stages TierContent 1Text (Methodology, ESRI World File), Document, Color Profile, Target Image 2PDS Document, Still Image 3Audio, Text (SMIL) 4Web Harvest, Opaque Container 5Biomedical Image; Google Document Container 1, 2, 3 Tiers 1, 3, 4, 5: Migrate across all DRS owner codes at one time Tier 2: Migrate one DRS owner code at a time

29. Tier 2: Sequence by DRS Owner Code Migrate just your unit’s PDS document and still image content Minimize the amount of time the content you manage the most is in 2 different systems

30. DRS Owner Codes to be Migrated DIV.LIBR FHCL.HOUGH FHCL.MUSI GSD.LIBR RAD.ARCH RAD.SCHL FHCL.JUD FHCL.FAL FHCL.MAPS FMUS.MCZ HLS.LIBR HUL.ARCH HUAM.MUSE HBS.BAKR VIT.BERE HUL.PRES FMUS.GRAY HPPM.PIRC DOAK.RESLIB DOAK.MUS FCOR.REISCH FMUS.ARN HLNC.LIBR ARB.AAHOD DOAK.LIBR FCOR.FORST FCOR.WOLBACH FMUS.FARL FMUS.HUH FMUS.ORC FMUS.PEAB HMS.COUNT HPRE.WARD HUAM.SARDIS HUL.GGL HUL.OIS FCOR.CARP FCOR.HCO FCOR.URI FHCL.CAB FHCL.COLL FHCL.DAVIS FHCL.ENV FHCL.FUNG FHCL.GOV FHCL.LITT FHCL.MED FHCL.SLV FHCL.TOZ FHCL.YENCH FMUS.SEM FMUS.WARE GSE.GUTMN KSG.LIBR

31. Timing Current estimates: – Building & testing migration tools: Now – Begin Tier 1 content: Spring 2014 – Begin Tier 2 content: Summer 2014 Units will be contacted about their Tier 2 migration schedule

32. After Your Tier 2 Migration You, and anyone depositing on your behalf, will begin depositing only to the new DRS All of your management tasks will be done only in the new DRS

33. IMAGING SERVICES: MINIMIZING DISRUPTION Bill Comstock

34. Minimizing Disruption Testing by Imaging Services Uninterrupted services Migration sequencing Participating as a “pioneer”

35. Testing by Imaging Services Alpha and beta testing: Depositing processes DRS content maintenance tools – Searching and assembling content for download – Editing PDS objects

36. Uninterrupted Services Providing services before and after your migration Content needs to be deposited Content needs to be searched Content needs to be assembled may need to be edited may need to be downloaded

37. Migration Sequencing We will synchronize deposits with your migration – start depositing for you in the new DRS after your Tier 2 content is migrated

38. Imaging Services as Pioneers As pioneers, we: Learn to use the new tools Refine the new depositing workflows Identify bugs Suggest improvements Create a group of local experts that can support those that follow We’ll wear the scars so that you can stay pretty!

39. WRAP-UP AND NEXT STEPS Kate Bowers and Andrea Goethals

40. Nine Pioneers Limited number of first depositors to new DRS Factors – New DRS-ready content from new systems EAS (Electronic Archiving Service), ACORN (Weissman Preservation Center conservation treatments), DASH (for ETD) – Prepared and trained staff – No content to migrate HUA example: opaque objects

41. First Deposit in the New DRS 大藏經 Da Zang Jing - Buddhist sutra, Qing dynasty (1644 -1911), China, Tibetan language

42. Email List drs-migration@hulmail.harvard.edu

43. http://hul.harvard.edu/ois/systems/drs/drs2.html

44. Coming Attractions Open meetings – Technical aspects brown bag (March) – Digital preservation & DRS intro (Summer) Training and instruction – Refresher training – New training – Onsite assistance

45. Q & A Thanks!