ICDL-Contentra Workshop 29 th November /11/2013 Contentra Technologies Confidential (RajuB)1

What Can You expect in this Presentation An overview of Issues in long-term preservation of access to Digital Content Preserving Trust, Authenticity, Intelligibility, and Usability Standards Bit Preservation Format Preservation Metadata Preservation Planning for Digital Preservation Programme Q&A 29/11/2013 Contentra Technologies Confidential (RajuB)2

Fragility of Digital Content Overly dependent on the Environment Media or Carrier Encoding or Format Descriptive Information or Metadata Rendering Environment Knowledge Base of Designated Communities 29/11/2013 Contentra Technologies Confidential (RajuB)3

Issues in Preserving Long-term access Planning Horizon is between 100 to 400 years Technical and Non-technical issues Technical Issues Bit Preservation or Media Preservation Encoding Preservation or Format Preservation Metadata Preservation or Meaning & Context preservation Rendering Environment or Viewing Environment Characterizing the Designated Communities Non-Technical Issues 29/11/2013 Contentra Technologies Confidential (RajuB)4

Non-Technical Issues in Preservation Source: Mike Kastellec “Practical limits to Scope of Digital Preservation” 29/11/2013 Contentra Technologies Confidential (RajuB)5

Technical Issues Bit Preservation or Media Preservation Risks Assessment Media Obsolescence Bit Rot Loss-less migration Encoding Preservation or Format Preservation Containers & Content Object Preservation & Set Preservation (Data sets/Fonds) 29/11/2013 Contentra Technologies Confidential (RajuB)6

Metadata Preservation Structural Metadata Administrative Metadata Provenance Metadata Rights Metadata.. Descriptive Metadata Preservation Metadata 29/11/2013 Contentra Technologies Confidential (RajuB)7

OAIS Reference Model 29/11/2013 Contentra Technologies Confidential (RajuB)8

Establishing Trust Ingest trustworthy information Is through a clear transparent communication channel to the future society Provenance, Respect du fonds, Chain of custody, Integrity The channel should preserve itself Realized through a set of: Policies & Procedures Auditable Systems & Services 29/11/2013 Contentra Technologies Confidential (RajuB)9

Attributes of TDR Compliance to International Standards (such as OAIS- RM) Administrative Responsibilities Organizational Viability Financial Sustainability Technological and Procedural Suitability Procedural Accountability 29/11/2013 Contentra Technologies Confidential (RajuB)10

Responsibilities of a TDR Curational Responsibility Operational Responsibilities Ingestion of Resource & Metadata Obtaining control over the information for lifecycle management Rights, Validation of Metadata and Preservation Metadata, Unique Identification, Binding Metadata with resource, failsafe archival storage. Access Provisioning Compliance to policies Discovery and Access service provisioning Managing Producer/TDR and TDR/Designated community relationships 29/11/2013 Contentra Technologies Confidential (RajuB)11

Recommended Policies for TDRs Collection Development Policies Digital Rights and Digital Authorization Policies Policies for storage Service Provisioning Policies System Change/Update control policies 29/11/2013 Contentra Technologies Confidential (RajuB)12

Types of Content Digitized Content Born Digital Content Web Social Network Feeds Research Data Survey Data Sets Experimental/Empirical Data Thesis and Dissertations 29/11/2013 Contentra Technologies Confidential (RajuB)13

Solution Architecture WAS NEWS Papers Digital Objects Ingest Manage- ment System Green Stone Digital Library Platform Digital Preservation WF Management Digital Library Services Workflow Management Collection Services Metadata Mgmt Services Archival Repository Preservation Services Discovery Service Interface Access Services 29/11/2013 Contentra Technologies Confidential (RajuB)14

Standards 1 29/11/2013 Contentra Technologies Confidential (RajuB)15

Standards 2 Source: 29/11/2013 Contentra Technologies Confidential (RajuB)16

29/11/2013 Contentra Technologies Confidential (RajuB)17 Source: Standards 1

29/11/2013 Contentra Technologies Confidential (RajuB)18 Source: Standards 2

Preservation Planning 29/11/2013 Contentra Technologies Confidential (RajuB)19

What might organisations need in terms of preservation services?

Six step process for digital curation Step 1: Develop Collection Development Policy and Repository services policy Target user groups Information use patterns Step 2: Define the Service experience goals Information Discovery Services Rendering environment and Readability Information Quality (integrity/comprehensive/reliable/fit-for-use) 29/11/2013 Contentra Technologies Confidential (RajuB)21

Six step process for digital curation (2) Step 3: Identify information acquisition/curation processes Information sources identification Selection criteria and ingest mechanisms Step 4: Discovery and Access mechanisms Metadata & Navigational support for users to discover information Access and Rendering environments 29/11/2013 Contentra Technologies Confidential (RajuB)22

Six step process for digital curation (3) Step 5: Instrumentation for usage data Step 6: Sustainability over a period of time Resources & Funding Maintenance of currency and value 29/11/2013 Contentra Technologies Confidential (RajuB)23

Levels of Digital Preservation (NDSA) 29/11/2013 Contentra Technologies Confidential (RajuB)24 Level 1 (Protect your data) Level 2 (Know your data) Level 3 (Mirror your data) Level 4 (Repair your data) Storage and Location Two copies & Ingested Three copies, a copy in different location, document storage system and media and what you need to use them At least one copy in a different location with different disaster threat Obsolescence monitoring process for storage systems and media At least 3 copies in locations with different disaster threats Comprehensiv e plan in place to keep files and metadata on currently accessible media

Levels of Digital Preservation 2 (NDSA) 29/11/2013 Contentra Technologies Confidential (RajuB)25 Level 1 (Protect your data) Level 2 (Know your data) Level 3 (Mirror your data) Level 4 (Repair your data) File Fixity and Data Integrity Check file fixity on ingest if it is provided otherwise create fixity info Check file fixity on all ingests Virus and Integrity Checks on high risk content Check fixity of content at fixed intervals Maintain logs of fixity info Ability to detect corrupt data Virus and integrity check on all content At least 3 copies in locations with different disaster threats Check fixity on all content in response to events Ability to repair/replace corrupt data No one person has write access to all copies

Levels of Digital Preservation 3 (NDSA) 29/11/2013 Contentra Technologies Confidential (RajuB)26 Level 1 (Protect your data) Level 2 (Know your data) Level 3 (Mirror your data) Level 4 (Repair your data) Information Security Identify who has read, move and delete authorization to individual files Restrict who has authorization to individual files Document access restrictions for content Maintain logs of all actions on the files inclusive of preservation actions Perform Audit of all logs MetadataInventory of content and its storage location Document all non-colocation of inventory Store Administrative metadata Store transformative metadata and log Store technical and descriptive metadata Store preservation metadata

Levels of Digital Preservation 4 (NDSA) 29/11/2013 Contentra Technologies Confidential (RajuB)27 Level 1 (Protect your data) Level 2 (Know your data) Level 3 (Mirror your data) Level 4 (Repair your data) File FormatsEncourage the use of a limited set of known open file formats and codecs Inventory of file formats in use Monitor file format obsolescence issues Perform format migration, emulation and similar activities as needed

Advanced Considerations of Digital Preservation Migration VS Emulations Intelligibility Preservation Modules and Dependencies Designated Community Profiles Usability Re-use by Communities other than planned designated communities Sustainability 29/11/2013 Contentra Technologies Confidential (RajuB)28

Final report (February 2010) Blue Ribbon Task Force for Sustainable Digital Preservation & Access Three principal actions are required for sustainability: Articulate a compelling value proposition Provide clear incentives to preserve in the public interest Define roles and responsibilities among stakeholders to ensure an ongoing and efficient flow of resources to preservation throughout the digital lifecycle

Questions 29/11/2013 Contentra Technologies Confidential (RajuB)30