Stuart Pullinger 24th January 2018 ICAT Roadmap Update Stuart Pullinger 24th January 2018

Outline of the Roadmap Securing Longevity Supporting Open Science address long-term and near-term technical ideas to ensure ICAT has longevity Supporting Open Science adopt standards and interfaces to support open science Sustaining the ICAT Project improve development processes and documentation accept input and effort from more people Extending ICAT functionality

Securing longevity Review of user requirements Ongoing work looking at requirements for DLS Will start reviewing user interface needs with STFC facilities soon Investigation of technologies: see below Topcat/eCat/ICAT Manager rewrite More activity recently Hope to be able to report progress next month REST API Louise has/will be looking at this (including ICAT+ and SciCat) GraphQL Louise looking at this: GatsbyJS for the website?

Supporting Open Science Data publication/DOIs Expecting schema changes for icat.server 4.10.0 Interoperability Expecting to follow lead from PANOSC (ESRF) We have a small amount of effort in EXPANDS too (if it is funded) DAAS Very active though no activity planned through this project Provenance tracking

Sustaining the ICAT Project Formalisation of development process Some additions to the developer instructions Louise has created automated build and test configurations for Travis CI Improved documentation/manual Was not able to secure extra effort for this last year Louise and Stuart need to make a plan for this New Website Louise looking at options including GatsbyJS – as a way to also learn React and assess GraphQL.

Extending ICAT Functionality Tickets Ongoing Schema review After these slides! Will return to monthly meetings after 4.10.0 is released. Richer metadata Stuart to look into this. No recent progress to report.

Discussions Based on Alex’s questions to the list

Development of a common metadata catalogues API.  Related to WP3 in the PANOSC proposal. Work on WP3 has not started yet but will start soon. ICAT is probably one if not the most widespread metadata catalogue in Europe for large facilities It might be worth to discuss internally in the ICAT's collaboration and agree what we might expect of a common API for metadata catalogues. I imagine that it is also directly connected to EXPANDS (btw any new about the funding?) 

PANOSC work package description? "Define an API to be used in the Photon and Neutron community that will allow for FAIR exposure of the data at the individual institutions through a catalogue service. The API will allow federation, and exposure of metadata relevant for the area, in a way that will enable search and facilitate access of researchers across scientific disciplines. Existing APIs (e.g OAI-PMH) and communities (e.g. openarchives.org, Dublin Core Metadata Initiative (DCMI), OpenAIRE) will be taken into account. The API will enable domain specific search extensions aware of the metadata definitions and usage at photon and neutron facilities. In order to test any implementation at facilities for compliance, a set of API tests will be developed. The test harness will be executable against a given site catalogue service and result in a report stating the status towards compliance"

Schema Discussion Status for the modification of the schema to include the equivalent to visit so there is (1 investigation -> N visits ) or (1 proposal -> N investigations).  I don't remember well the status of this or even if this agreed or done.

Steve’s slide?

Schema Discussion Question about the extension of parameters for entities.  For efficiency purposes I wonder if it could be envisaged to add to the tables calculated values like: Investigation.datasetCount, Investigation.volume or dataset.volume? I know it is not a good practice as these values can theoretically be calculated on the fly however we know it is not the case and in all the cases will have an impact in the performance. I wonder where we are or if it was rejected.

Schema Discussion Questions about the embargo period. It is a question but might be promoted and be a need at the ESRF. It might happen that an investigation is under embargo then data is not public however, for instance, a subset of the datasets needs to be released. Can this be done with ICAT today? Has someone discussed to add releaseDate at level of DataColllection, for instance?