Research Data Repository Interoperability Thomas Jejkal

2 RDA Working Group - Charter „ The Research Data Repository Interoperability Working Group will establish standards for interoperability between different research data repository platforms focusing on machine-machine communication. These standards may include (but are not limited to) a generic API specification and import/export formats [...]“ RDRIWG Case Statement (

3  First meeting at P6, phone conference and BoF session at P7  Case Statement submitted on 19th of May  18 months with adoptable outcome  2 generic use cases  Replication/Migration  Information retrieval RDA Working Group - History

4  18 nanoscience facilities all over Europe  Measurement, Analysis, Simulation  Support for multi-facility proposals  Huge variety of research data outputs (raw, analyzed, simulated)  Registered at distributed information data repository platform (IDRP)  Find, retrieve, share via data portal  Publication to publication repository on-demand Use Case – NFFA Europe

5 NFFA Europe – Metadata Model Project (NFFA) Proposal Facility 1 Instrument Experiment Measurement Raw Data Facility 2 Sample Data Analysis Analysed Data Data assets summarize all file- based output of experiments Basic metadata provided with proposal submission Additional metadata added after experiment

6 NFFA Europe – Architecture NFFA Portal Facility Local Data Repository

7 NFFA Europe – Architecture NFFA Portal Facility Local Data Repository Information and Data Repository Platform (IDRP) Distributed Repository

8 NFFA Europe – Architecture NFFA Portal Facility Local Data Repository Information and Data Repository Platform (IDRP) Distributed Repository

9 NFFA Europe – Architecture NFFA Portal Facility Local Data Repository Information and Data Repository Platform (IDRP) Distributed Repository Publication Repository

10  Many different research data repository systems at facilities  How to provide transparent access via IDRP/portal to (meta-)data?  Different models/repositories needed for internal and public data  How to migrate from the internal to the public model?  Different (research) data types  How to identify/obtain data type information from repositories?  Reasonable performance and flexibility needed due to focus on research data  What are special challenges for research data? Interoperability Aspects & Questions

11  One internal metadata model  Custom metadata provided separately, custom extraction and retrieval needed  Data remains at facility, registered by reference, access organized by policy Transparent Access to (Meta-)Data NFFA Portal Facility Local Data Repository IDRP Distributed Repository  Import/Export formats, replication

12  Transformation as crosswalk from internal model to (reduced) public model, automated PID assignment  Data dereferenced  copied to IDRP Migrate from Internal to Public Model NFFA Portal IDRP Distributed Repository Publication Repository  Replication/migration

13 Identify/Obtain Data Type Information Facility Local Data Repository IDRP Distributed Repository Data Type Registry  Manual definition of data types  Suggestion based on defined types  Adoption of results of DTR WG   Retrieval of platform/content related information

14 Challenges for (NFFA) Research Data  Stored at many different platforms (ICAT, NoMaD, iRods, AiiDA, KIT Data Manager)  Data formats and structure differ depending on repository platform and used equipment  Often no clear separation between data and metadata  Volume from 1 to 10 TB/a  Data and metadata access restricted by default, publication optional, might be covered by data policy

15  Reduce effort for realizing NFFA-like concepts  Easier “federation” of local repositories  Less vendor lock-ins (e.g. publication repository)  Standard ways for getting platform/content related information How could Interoperability help?

16  State of the art standards/technologies for access/information retrieval  OAI-PMH, OAI-ORE, SWORD, METS, Resource Sync, Re3Data....  Confederation of Open Access Repositories (COAR)  Worked since 2011 on (open access) repository interoperability  Roadmap for Future Directions for Repositories Interoperability  Related RDA IGs and WGs  Repository Platforms for Research Data IG  Data Fabric IG  The Long Tail of Research Data IG  Data Type Registry WG What is There?

17  First working meeting at P8 (if endorsed)  Start working on analyzing the state of the art and identifying gaps  Short talks and discussion  E.g. OAI-PMH, SWORD, METS, Linked Data Platform, ResourceSync  Go into D1: Research Data Repository Interoperability Primer (M6)  Basis for D2: Interface Specification Draft (M12)  Workshop proposal submitted for IEEE BigData 2016 RDA Working Group – What’s next?

18  Research data repository interoperability could help to  remove barriers,  support collaboration, and  to create commonalities.  RDA WG brings platform developers together to work on this topic  Could greatly improve data sharing and exchange  Potential of immediate adoption/benefit of outcomes for NFFA Conclusions