1. Long-term preservation aspects in the eSciDoc project Natasa Bulatovic Max-Planck Digital Library bulatovic@mpdl.mpg.de

2. eSciDoc: Background information  Joint Project between Max Planck Society and FIZ Karlsruhe  Funded by German Federal Ministry for Education and Research (BMBF) until Mid 2009  Additional substantial own efforts by both partners  Both Partners committed themselves to ongoing efforts until 2011

3. eSciDoc project landscape

4. Diversity of MPG supported by a service infrastructure 09.09.2015 ●People ●Librarians, researchers, developers, general public ●Data ●Publications, research data, supplementary material etc. ●Processes ●Various institutes apply various workflows supporting their data management

5. eSciDoc: what it addresses and provides?  Organizational aspects Business processes: introspection into the “research information and research data life-cycle” Modelling of the “research workflows” (usage, scenarios, use cases, process worfklows) “SOA” is not only technical infrastructure - it is driven by organization-wide processes and requirements  Technical aspects service infrastructure instead of silos applications solutions to visualize, publish, relate and manage data easy composition of services and data mashups and repurposing Both organizational and technical activities facilitate the growth of the organization and the dissemination, accessibility and easy reuse of the research results across disciplines

6. eSciDoc.PubMan: Management of publications

7. eSciDoc.Virr: Management of digitized textual resources

8. eSciDoc.FACES: Management of image collections

9. Challenges And facts for eSciDoc project related to the digital preservation and long-term archiving

10. Which data are managed? Support for variety of data (publications, old manuscripts, microscopic images, patents, datasets)

11. Abstraction: Data structure Abstraction in data modelling and implemented data services (items, containers).xml Specialization through content models gives contextual information on what data it is (publication, scanned book page, lexical resource, collection of “digital things”) http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Logical_Data_Model Atomistic model

12. Atomistic model - items 09.09.2015Seite 11

13. Atomistic model - containers 09.09.2015Seite 12

14. Communities and workflows Core workflow Support for variety of workflows for different user-communities Community specific workflow

15. Persistent identification Each resource and resource component (metadata, files) must be persistently identified Users may decide what to identify (resource, resource version) Users may configure when to assign the persistent identifier on a repository level Keep all persistent identifiers of the resource (if such existed before the resource has been created in the system) Different communities decide what to identify and when in a different manner and at a different state of the completeness of their data

16. Persistent identification system  But which PID system to choose out of many ?  Handle System (CNRI - Corporation for National Research Initiatives)  Digital Object Identifiers - DOI (IDF – International DOI Foundation)  Archival Resource Key –ARK (California Digital Library)  the Uniform Resource Name - URN (IANA)  National Bibliography Numbers - NBNs  the persistent URL – PURL (OCLC, Online Computer Library Center)  the Open URL  Most important is the organizational commitment for PID and PID infrastructure, PIDs are rather organizational then technical effort  EPIC – European Persistent Identifier Consortium (GWDG Germany, SARA Netherlands, CSC Finland, http://www.pidconsortium.eu/ )

17. Resource metadata  Standards and Frameworks – have to be used so data are known  Dublin Core, MODS, other  Where no standards exist, describe the profile in standard format  Dublin Core Singapore Framework  Services, metadata quality and metadata extraction  Metadata Validation service  CoNE – Control of Named Entities service  JHove – technical metadata extraction service  Supports and important aspect: quality of data, clear identification of resources, format migration  Interoperability – data redundancy  Ensures resources are disseminated via RSS, OAI-PMH Support for variety of data (publications, old manuscripts, microscopic images, patents, datasets) and their metadata

18. Metadata standards and frameworks “The Singapore Framework for Dublin Core Application Profiles is a framework for designing metadata applications for maximum interoperability and for documenting such applications for maximum reusability” http://dublincore.org/architecturewiki/SingaporeFramework/ http://dublincore.org/documents/2007/06/04/abstract-model/

19. Metadata profiles ●DCAP based (Dublin Core Application Profile) ●DC terms (identified URIs) ●eSciDoc solution specific terms (identified by URIs) ●METS/MODS ●Publicly available ●Functional description http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Application_Profiles http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Application_Profiles ●Application profiles schemas (xsd) http://metadata.mpdl.mpg.de/escidoc/metadata/schemas/0.1/ http://metadata.mpdl.mpg.de/escidoc/metadata/schemas/0.1/ ●Vocabulary encoding schemas http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Encoding_Schemes http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Encoding_Schemes ●Interoperability levels ●Shared term definitions (done) ●Semantic interoperability (done) ●Description set syntactic interoperability (started) ●Description set profile interoperability (started)

20. Services CoNE – Control of named entities Journals, Persons, Dewey Decimal Classification (3 public levels), Creative Commons Licenses (CC licenses), ISO 639-3 Languages, MIME Types, PACS classification, Custom classifications “Vocabulary Encoding schemas for resources ” Validation service Semantic metadata validation based on Shibboleth If the publication is of type “Journal Article” then the “Journal” information has to be present Rules are preserved in XML – and are not encoded in the business logic code Jhove service Technical metadata are extracted prior to binary content upload Technical metadata are kept together with the binary content Example http://faces.mpdl.mpg.de/details/escidoc:47276 http://r-coreservice.mpdl.mpg.de/ir/item/escidoc:47276

21. Data provenance  Support for event logs and version history (what happened with the content?)  PREMIS v1 created for each resource (events info) Example: http://pubman.mpdl.mpg.de/pubman/item/escidoc:66603  Upcoming: upgrade to PREMIS v2

22. Software, standards, documentation  Software all used software is an open source software eSciDoc software available under CDDL license (open source)  Documentation Accessible (still a lot of work to do) Specifications and some design are available via MPDL Colab e.g. http://colab.mpdl.mpg.de/mediawiki/PubMan_About  Standards and formats XML, XSD used for interface and service operation messages Use of metadata standards PREMIS Open formats wherever possible e.g. metadata, lexical resources TEI-XML etc. Containment: Fedora XML objects for content resources OAIS reference model – compatible OAIS Bitstream preservation

23. OAIS Archive: What we will put for LTA? XML XML/RDF XSDs Open source code Binary content with technical metadata (XML) and calculated MD5 checksums

24. Lessons learned  Have we covered all aspects?  They must be considered from the very start in infrastructural design and implementation  Things are getting more complicated in dynamic environment (data are changed often)  Selection process  What – all or only selected research data?  When – whenever data is created or after the finalization of curration activities?  Why - “institutional memory”, future research?  Data redundancy  As many copies as possible  Share and replicate as much as possible - redundancy does not hurt (it costs )  Organizational commitment  Sustainable  Long-term  Technical  Collaboration and cooperation with partners necessary (its not a single unit effort)

25. Thank you!  www.escidoc-project.de www.escidoc-project.de  http://colab.mpdl.mpg.de  bulatovic@mpdl.mpg.de bulatovic@mpdl.mpg.de

26. Resources  eSciDoc project web pages, Infrastructure download  http://www.escidoc-project.de http://www.escidoc-project.de  MPDL collaboratory network  http://colab.mpdl.mpg.de http://colab.mpdl.mpg.de  MPDL software download  http://escidoc1.escidoc.mpg.de/projects/common_services/ http://escidoc1.escidoc.mpg.de/projects/common_services/  http://escidoc1.escidoc.mpg.de/projects/pubman/ http://escidoc1.escidoc.mpg.de/projects/pubman/  http://escidoc1.escidoc.mpg.de/projects/faces/ http://escidoc1.escidoc.mpg.de/projects/faces/  http://escidoc1.escidoc.mpg.de/projects/virr/ http://escidoc1.escidoc.mpg.de/projects/virr/  http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Admin http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Admin