TCS-ICS interactions Kuvvet Atakan 1 and the WP6 and WP7 Teams 1 University of Bergen / Department of Earth Science

ICS-TCS interactions Requirements elicitation process – TCS requirements on data, data products, software and services (DDSS) – TCS use cases – TCS assets Harmonisation process – Cross-checking the interoperability of various DDSS elements from all TCSs – Identifying those DDSS elements that needs to be harmonised User feedback processes – Approval of requirements and harmonisation processes before the implementation – User feedbacks during the ICS development and testing period ICS development process – Joint work by ICS and TCS – Continuous interactions Testing process – Testing the functionalities of ICS-C, system performance and roboustness – Testing the interoperability of the DDSS from relevant TCSs – Testing the maturity of the various DDSS of TCSs Validation process – Approvals by ICS-development team and TCS teams – Approvals by IPC, BNSR and finally by BGR Further process during pre-operational phase

University of Bergen / Department of Earth Science EPOS IP project Timeline Implementation Validation Pre-operation

EPOS-IP is a complex project with 46 partner institutions and 17 WPs

WP6: ICS-TCS Integration and Interoperability

ICS-TCS communication ICS-TCS integration guidelines are designed to give necessary information about integration and interoperability to TCSs. Level-1 and Level-2 documents are already communicated to TCSs. Level-3 document will be prepared after the kick-off meeting. There will be cyclic meetings (virtual or physical) with TCSs every 3 to 6 months. These will be used for status reporting, feedback and work planning for the next cycle. Direct communication between the ICS-developers and the TCS IT-responsibles will continue more frequently between the cyclic meetings to provide an efficient work-flow. Integration workshops to be arranged with all TCSs and WP6 and WP7. EPOS annual meetings will be used for overall coordination with other TCS WPs and Transverse WPs. Special meetings will be arranged if significant unforseen challenges arise. Various communication tools will be used including virtual connections such as video-conferences, document sharing and exchange platforms, chatting services and other facilities of the new EPOS collaborative area.

National Research Infrastructures NRI Thematic Core Services TCS Integrated Core Services ICS EPOS Concept and Design WP6

ICS - Integrated Core Services ICS-C and ICS-D ICS-C ICS-D ICS-Central Hub Integrates the DDSS from TCSs and makes sure they are interoperable through a metadata catalogue Provides access, searching, visualization and processing tools Orchestrates external resources (ICS-D) EPOS member states currently compete for hosting ICS-Distributed services Provide IT resources not built by EPOS ICS-C Computational resources Visualization resources Data storage facilities Others…

Figure 6.1. Interactions between the WP6 and WP7 tasks. WP6 ICS-TCS Integrations and Interoperability WP6 ICS-TCS Integrations and Interoperability WP7 ICS Design and Development WP7 ICS Design and Development T6.1 Interactions with TCS WPs T6.1 Interactions with TCS WPs T6.2 ICS-D design and interoperability with ICS-C T6.2 ICS-D design and interoperability with ICS-C T7.1 EPOS ICS-architecture T7.1 EPOS ICS-architecture T6.6 Data Management Plan T6.6 Data Management Plan T6.3 Interactions with CES within TCS WPs T6.3 Interactions with CES within TCS WPs T6.5 Metadata catalogue development T6.5 Metadata catalogue development T7.4 CES development T7.4 CES development T7.2 ICS development T7.2 ICS development T7.3 ICS-D integration T7.3 ICS-D integration T6.4 User feed-back updates T6.4 User feed-back updates WP3 ICS-C

Figure 6.1. Interactions between the WP6 and WP7 tasks. WP6 ICS-TCS Integrations and Interoperability WP6 ICS-TCS Integrations and Interoperability WP7 ICS Design and Development WP7 ICS Design and Development T6.1 Interactions with TCS WPs T6.1 Interactions with TCS WPs T6.2 ICS-D design and interoperability with ICS-C T6.2 ICS-D design and interoperability with ICS-C T7.1 EPOS ICS-architecture T7.1 EPOS ICS-architecture T6.6 Data Management Plan T6.6 Data Management Plan T6.3 Interactions with CES within TCS WPs T6.3 Interactions with CES within TCS WPs T6.5 Metadata catalogue development T6.5 Metadata catalogue development T7.4 CES development T7.4 CES development T7.2 ICS development T7.2 ICS development T7.3 ICS-D integration T7.3 ICS-D integration T6.4 User feed-back updates T6.4 User feed-back updates WP3 ICS-C

Figure 6.1. Interactions between the WP6 and WP7 tasks. WP6 ICS-TCS Integrations and Interoperability WP6 ICS-TCS Integrations and Interoperability WP7 ICS Design and Development WP7 ICS Design and Development T6.1 Interactions with TCS WPs T6.1 Interactions with TCS WPs T6.2 ICS-D design and interoperability with ICS-C T6.2 ICS-D design and interoperability with ICS-C T7.1 EPOS ICS-architecture T7.1 EPOS ICS-architecture T6.6 Data Management Plan T6.6 Data Management Plan T6.3 Interactions with CES within TCS WPs T6.3 Interactions with CES within TCS WPs T6.5 Metadata catalogue development T6.5 Metadata catalogue development T7.4 CES development T7.4 CES development T7.2 ICS development T7.2 ICS development T7.3 ICS-D integration T7.3 ICS-D integration T6.4 User feed-back updates T6.4 User feed-back updates WP3 ICS-C

Figure 6.1. Interactions between the WP6 and WP7 tasks. WP6 ICS-TCS Integrations and Interoperability WP6 ICS-TCS Integrations and Interoperability WP7 ICS Design and Development WP7 ICS Design and Development T6.1 Interactions with TCS WPs T6.1 Interactions with TCS WPs T6.2 ICS-D design and interoperability with ICS-C T6.2 ICS-D design and interoperability with ICS-C T7.1 EPOS ICS-architecture T7.1 EPOS ICS-architecture T6.6 Data Management Plan T6.6 Data Management Plan T6.3 Interactions with CES within TCS WPs T6.3 Interactions with CES within TCS WPs T6.5 Metadata catalogue development T6.5 Metadata catalogue development T7.4 CES development T7.4 CES development T7.2 ICS development T7.2 ICS development T7.3 ICS-D integration T7.3 ICS-D integration T6.4 User feed-back updates T6.4 User feed-back updates WP3 ICS-C

Figure 6.1. Interactions between the WP6 and WP7 tasks. WP6 ICS-TCS Integrations and Interoperability WP6 ICS-TCS Integrations and Interoperability WP7 ICS Design and Development WP7 ICS Design and Development T6.1 Interactions with TCS WPs T6.1 Interactions with TCS WPs T6.2 ICS-D design and interoperability with ICS-C T6.2 ICS-D design and interoperability with ICS-C T7.1 EPOS ICS-architecture T7.1 EPOS ICS-architecture T6.6 Data Management Plan T6.6 Data Management Plan T6.3 Interactions with CES within TCS WPs T6.3 Interactions with CES within TCS WPs T6.5 Metadata catalogue development T6.5 Metadata catalogue development T7.4 CES development T7.4 CES development T7.2 ICS development T7.2 ICS development T7.3 ICS-D integration T7.3 ICS-D integration T6.4 User feed-back updates T6.4 User feed-back updates WP3 ICS-C

Figure 6.1. Interactions between the WP6 and WP7 tasks. WP6 ICS-TCS Integrations and Interoperability WP6 ICS-TCS Integrations and Interoperability WP7 ICS Design and Development WP7 ICS Design and Development T6.1 Interactions with TCS WPs T6.1 Interactions with TCS WPs T6.2 ICS-D design and interoperability with ICS-C T6.2 ICS-D design and interoperability with ICS-C T7.1 EPOS ICS-architecture T7.1 EPOS ICS-architecture T6.6 Data Management Plan T6.6 Data Management Plan T6.3 Interactions with CES within TCS WPs T6.3 Interactions with CES within TCS WPs T6.5 Metadata catalogue development T6.5 Metadata catalogue development T7.4 CES development T7.4 CES development T7.2 ICS development T7.2 ICS development T7.3 ICS-D integration T7.3 ICS-D integration T6.4 User feed-back updates T6.4 User feed-back updates WP3 ICS-C

Figure 6.1. Interactions between the WP6 and WP7 tasks. WP6 ICS-TCS Integrations and Interoperability WP6 ICS-TCS Integrations and Interoperability WP7 ICS Design and Development WP7 ICS Design and Development T6.1 Interactions with TCS WPs T6.1 Interactions with TCS WPs T6.2 ICS-D design and interoperability with ICS-C T6.2 ICS-D design and interoperability with ICS-C T7.1 EPOS ICS-architecture T7.1 EPOS ICS-architecture T6.6 Data Management Plan T6.6 Data Management Plan T6.3 Interactions with CES within TCS WPs T6.3 Interactions with CES within TCS WPs T6.5 Metadata catalogue development T6.5 Metadata catalogue development T7.4 CES development T7.4 CES development T7.2 ICS development T7.2 ICS development T7.3 ICS-D integration T7.3 ICS-D integration T6.4 User feed-back updates T6.4 User feed-back updates WP3 ICS-C

WP6 Deliverables D6.1 [M12] [M24]: Description of TCS requirements and use cases including – i) UML descriptions of use cases and – ii) Input forms and validation processes for metadata to catalogue. D6.2 [M18] [M36]: Plan for integration of selected ICS-Ds with the ICS-C, including input forms and validation processes for metadata to catalogue. D6.3 [M18] [M36]: Plan for integration of selected CES solutions including – i) UML descriptions of use cases and – ii) Input forms and validation processes for metadata to catalogue. D6.4 [M18] [M30]: Regular updates of the user feed-back from the EPOS community. D6.5 [M24] [M36]: specification for the Metadata catalogue input used in the forms and processes in the other tasks. D6.6 [M24] [M48]: Data Management Plan including specification for acquisition or development of appropriate software.

WP6 time-table for deliverables Month D6.1   D6.2   D6.3   D6.4   D6.5   D6.6  

WP6 Milestones M6.1 [M12]: Initial interactions with TCS, ICS-d and CES completed and first version of the list of data, data products and services to be included in the ICS are delivered to WP7. List of data, data products and services verified and validated by TCS, ICS-d and CES providers. M6.2 [36]: Final version of the Metadata Catalogue. Metadata catalogue verified and validated by ICS developers (WP7). M6.3 [48]: Data Management Plan are completed and delivered to WP7. Data Management Plan verified and validated by ICS developers (WP7).

TCS requirements and expectations A first crude version is collected already and will be presented in detail by TCS WPs. Based on this, it is clearly seen that there are significant differences between the TCSs and their requirements and expectations. It is therefore essential to develop tailored integration for each TCS. At the same time harmonisation of the DDSS from various TCSs needs to be done for interoperability of the services.

Requirements elicitation process Details of the TCS requirements will be collected including: – Requirements on data, data products, services and software (DDSS) to be included in EPOS ICS-C – Detailed description of use cases – Detailed description of TCS assets Approval of the selected DDSS for implementation – i.e. concensus building by: – TCS DDSS providers – ICS-developers – SCB

ICS-TCS Integration TCS DATA, SOFTWARE, RESOURCES TCS CATALOGUE or Directories TCS CATALOGUE or Directories TCS DATA, SOFTWARE, RESOURCES TCS CATALOGUE or Directories TCS CATALOGUE or Directories TCS DATA, SOFTWARE, RESOURCES ICS SOFTWARE CERIF METADATA CATALOGUE CERIF METADATA CATALOGUE ICS-C TCS to ICS CONVERTORS Joint design But ICS implementation ICS-D ICS-D SOFTWARE ICS-D ICS-D SOFTWARE ICS-D Convertor TCS CATALOGUE or Directories TCS CATALOGUE or Directories

Elements of EPOS ecosystem Complete ICT environment for research Researchers, research managers, innovators, media Processing Model User Model Data Model Resource Model User interaction with the system: AAAI, interaction workflow, multilinguality Describes functions of processing environment etc. Describes research data and ancillary info (institutions) Representing ICT resources

ICS-TCS common work Metadata Definition of community, standard-based metadata Match & Map: Match/Mapping of community metadata within the Metadata Catalogue APIs Definition of APIs for data discovery and for data access/retrieval User Software Data Resources

EPOS ICS and TCS integration FR Seismic networks National RIs FR Seismic networks National RIs EPOS GUI METADATA catalogue METADATA catalogue EPOS APIs System Managing Software IAAA system Workflow Engine PID system PID system ICS-D ICS-C TCSs National RIs FR Seismic networks National RIs FR Seismic networks National RIs Data products Data products Services Software TCS API Data set Data set Data products Data products Services Software TCS API Data set Data set Interoperability Layer: ICS-APIs (wrappers)

METADATA catalogue METADATA catalogue TCS System National Network National Repository Local HPC API / Web Service EPOS ICS Metadata: CERIF Definition of community, standard-based metadata APIs Definition of APIs for data discovery and for data access/retrieval Match & Map: Match/Mapping of community metadata within the Metadata Catalogue EPOS ICS and TCS integration EPOS-PP developed a prototype software platform Metadata catalogue standard is chosen to be CERIF (Common European Research Infrastructure Format) Implementation of the ICS (Integrated Core Services) during the EPOS-IP project relies on the availablity and maturity of the DDSS from TCSs which are based on the national RIs.

Computational Earth Science (CES) Integration of CES services require: – Availability of the dedicated software from the relevant TCSs – Access to HPC resources – Access to data storage facilities – Access to visualisation and processing tools Integrating CES requires a community effort and standardisation of the software and resources in addition to the ICS technical integration There are mature services associated with some of the TCSs CES applications related to some of the TCSs may need additional community effort Evaluation on the interoperability

ICS-D integration ICS-D includes: – Access to HPC resources – Access to data storage facilities – Access to visualisation and processing services Technical integration process for the EPOS-IP ICS-D integration requires a formal process – Application procedure for interested technology and resource providers – Selection process for the relevant ICS-D – Formal agreements regulating the access to ICS-D resources – Financial aspects of ICS-D services Final integration and committments for the EPOS-OP

Take home messages Decisions on various DDSS for each TCS requires an open, inclusive and transparent process internally at each TCS community. A realistic estimate of the maturity for implementation of various DDSS for each TCS is needed. Harmonising the requirements and expectations from the TCSs with the requirements and standards of the ICS-C is essential. Each TCS needs to have a realistic timetable for deliveries for internal processes and developments taking into account the implementation, testing and validation processes. Each TCS needs to allocate sufficient human resources for the ICS- integration. Further plans are needed for those DDSS that may not be found mature for implementation before the pre-operational phase.

Contact Points The templates collected for the first crude version of DDSS of TCSs will be distributed to all TCSs Contact points between WP6 and WP7 and the TCS leaders and IT- responsibles will be established First Integration Workshop will be arranged (where?) during February 2016 (?)

Thank you for your attention! University of Bergen / Department of Earth Science