General Model of E-ARK Services Istvan Alföldi National Archives of Hungary DLM Forum Members’ Meeting 10-11 June 2014, Athens

E-ARK General Model Conceptual framework Used methodology Agenda E-ARK General Model Conceptual framework Used methodology Results (not final)

General Model of E-ARK Services ”The scope of the E-ARK service is to provide a reference implementation, which integrates these currently non-interoperable tools into a replicable and scalable, common seamless workflow, allowing data owners and repositories to flexibly select and use the components most relevant for their specific situations. To achieve this, a set of common interfaces and information package formats will be defined by the E-ARK project and implemented using these tools.” (E-ARK DoW Part B Finalised version 2.0 - B1.3) ”The scope of the E-ARK service is to provide a reference implementation, which integrates these currently non-interoperable tools into a replicable and scalable, common seamless workflow, allowing data owners and repositories to flexibly select and use the components most relevant for their specific situations. To achieve this, a set of common interfaces and information package formats will be defined by the E-ARK project and implemented using these tools.” (E-ARK DoW Part B Finalised version 2.0 - B1.3) ”The scope of the E-ARK service is to provide a reference implementation, which integrates these currently non-interoperable tools into a replicable and scalable, common seamless workflow, allowing data owners and repositories to flexibly select and use the components most relevant for their specific situations. To achieve this, a set of common interfaces and information package formats will be defined by the E-ARK project and implemented using these tools.” (E-ARK DoW Part B Finalised version 2.0 - B1.3)

OAIS Reference Model Data and Content Management Producer Consumer e-Document Archive queries result sets Pre-Ingest Ingest Access orders SIP AIP AIP DIP Preservation Planning Archival Storage Administration

Preservation Planning Conceptual Framework Pre-Ingest Ingest Archival Storage Preservation Planning Administration Data Management Access Recommended Practices Processes and Use cases Tools and Interfaces Standards and Format Definitions

Conceptual Framework – Standards and Format Definitions Pre-Ingest Ingest Archival Storage Preservation Planning Administration Data Management Access Standards and Format Definitions OAIS, METS, ISAD(G), Moreq2010, PREMIS, SIARD, CMIS, Bagit, InterPARES 2, EAD, EAD-CPF, etc. E-ARK SIP SIP AIP E-ARK AIP DIP E-ARK DIP

Preservation Planning Conceptual Framework Pre-Ingest Ingest Archival Storage Preservation Planning Administration Data Management Access Recommended Practices Processes and Use cases Tools and Interfaces Standards and Format Definitions

Conceptual Framework – Tools and Interfaces Pre-Ingest Ingest Archival Storage Preservation Planning Administration Data Management Access Tools and Interfaces (already established) DMS RMS Content and Records Management System Alfresco CMIS interface Archival Repository ESSArch Preservation Platform (EPP) , Safety Deposit Box (SDB), RODA, The National Danish Bit Repository (NDBR) DB E-ARK SIP E-ARK AIP E-ARK DIP SOFIA Scalable Computation Staging Area Lily ET, DBExport, UAM, RODA-in Apache Hadoop

Conceptual Framework – Tools and Interfaces Pre-Ingest Ingest Archival Storage Preservation Planning Administration Data Management Access Tools and Interfaces (to be developed) AIP-DIP Conversion Comp. Search, Access and Display Interface SIP Creation Tools SIP-AIP Conversion Component DMS RMS Content and Records Management System Alfresco CMIS interface Archival Repository ESSArch Preservation Platform (EPP) , Safety Deposit Box (SDB), RODA, The National Danish Bit Repository (NDBR) DB E-ARK SIP E-ARK AIP E-ARK DIP SOFIA Scalable Computation Staging Area Lily ET, DBExport, UAM, RODA-in Data Mining Showcase Apache Hadoop

Conceptual Framework – Tools and Interfaces Pre-Ingest Ingest Archival Storage Preservation Planning Administration Data Management Access Tools and Interfaces (to be developed) AIP-DIP Conversion Comp. Search, Access and Display Interface SIP Creation Tools SIP-AIP Conversion Component DMS RMS Content and Records Management System Alfresco CMIS interface Archival Repository ESSArch Preservation Platform (EPP) , Safety Deposit Box (SDB), RODA, The National Danish Bit Repository (NDBR) DB E-ARK SIP E-ARK AIP E-ARK DIP SOFIA Scalable Computation Staging Area Lily ET, DBExport, UAM, RODA-in Data Mining Showcase Apache Hadoop

Preservation Planning Conceptual Framework Pre-Ingest Ingest Archival Storage Preservation Planning Administration Data Management Access Recomended Practices Processes and Use cases Tools and Interfaces Standards and Format Definitions

Conceptual Framework – Processes and Use Cases Process and Use Case Diagrams Producer Create SIP GM-PI-6 Start transfer to archive GM-PI-7 SIP SIP Creator (WP3) UAM RODA ESSArch

Conceptual Framework – Processes and Use Cases Process Diagrams

Conceptual Framework – Processes and Use Cases Business Process Modeling Notation (BPMN) Process Flow Participant role Activity

Conceptual Framework – Processes and Use Cases Business Process Modeling Notation (BPMN) Collapsed view

Conceptual Framework – Processes and Use Cases Standard Process Diagrams (BPMN) Detail view

Conceptual Framework – Processes and Use Cases Use Case Diagrams Producer Create SIP GM-PI-6 Start transfer to archive GM-PI-7 SIP SIP Creator (WP3) UAM RODA ESSArch Used tool Activity Participant role Input / Output

Conceptual Framework – Processes and Use Cases Producer Create SIP GM-PI-6 Start transfer to archive GM-PI-7 SIP SIP Creator (WP3) UAM RODA ESSArch Who does what? In what order? Using what kind of tool? Required input? Produced output?

Preservation Planning Conceptual Framework Pre-Ingest Ingest Archival Storage Preservation Planning Administration Data Management Access Recomended Practices Processes and Use cases Tools and Interfaces Standards and Format Definitions

Preservation Planning Conceptual Framework Pre-Ingest Ingest Archival Storage Preservation Planning Administration Data Management Access Recomended Practices To be added according to the experiences of the Pilots

Used Methodology and Schedule Surveys Questionnaire 1 (11 February, 2014) Infrastructure overview Process overview Data overview Questionnaire 2 (3 March, 2014) Detailed process information Detailed tool description General high-level requirements Input data profile survey (30 May, 2014) Draft model and Discussion Draft model sent out (26 May, 2014) Comments in e-mail E-ARK technical meeting (12-13 June 2014, Athens) Final model and approval (July 2014)

Results – Processes and Use cases Pre-Ingest

Results – Processes and Use cases Producer Select data (manual) GM-PI-3 Define SIP content GM-PI-1 Select data (with rules) GM-PI-2 Create SIP GM-PI-6 Extract data from DB GM-PI-4 Extract data from DMS/RMS GM-PI-5 Start transfer to archive GM-PI-7 SIP DB DMS/RMS DBExport ESSArch Alfresco RODA SIP Creator (WP3) UAM Pre-Ingest

Results – Processes and Use cases Ingest

Results – Processes and Use cases Ingest Technical stuff Start AIP generation workflow GM-I-2 Upload SIP GM-I-1 SIP E-ARK AIP ESSArch PP RODA SDB E-ARK SIP  AIP tool

Results – Processes and Use cases Data Management

Results – Processes and Use cases Data Management Archivist Select records to export GM-DM-1 GM-DM-2 Manipulate records GM-DM-3 Start export workflow Archivee CMIS Lily

Results – Processes and Use cases Access

Results – Processes and Use cases Access End user (Consumer) Download DIP GM-A-8 Search and select data GM-A-1 Request data GM-A-2 Archivee AIP E-ARK Request access from presentation platform GM-A-3 Presentation / Data Mining Platform

General Model – Work package overview

General Model – Pilot overview

Cross Reference Tables – Use case view

Cross Reference Tables – Tools view

Thank you!