Fifth iMarine Board meeting June 2014, FAO (Rome) Related to Agenda item 6 NEAFC to ICES data exchange Supporting collaboration and secure data exchange through the iMarine platform Related to Agenda item 6 Fifth iMarine Board meeting, June 2014, FAO (Rome)

Background NEAFC and ICES are currently exchanging data (VMS positions, catch data, authorizations, vessel details) on a regular basis Exchanged data are: Sensitive (cannot be shared outside the realm of involved parties) Anonymized (to avoid sharing vessel identification details, as per NEAFC agreements) Categorized (lengths, engine power) NEAFC to ICES data exchange support Fifth iMarine Board meeting, June 2014, FAO (Rome)

The current workflow is supervised by humans NEAFC produces the exchanged data manually (SQL result sets are transformed to CSV) once per semester Data are stored in a private area of the NEAFC web servers ICES users log in to the private area and download the data after they’ve been made available by NEAFC Old data are kept on the NEAFC server until they are removed Data access are accounted Background (continued) NEAFC to ICES data exchange support Fifth iMarine Board meeting, June 2014, FAO (Rome)

NEAFC to ICES data exchange support Current interactions and boundaries NEAFC RDBMS HTTPD ICES Internet CSVs Raw Data Raw Data Anonymization and categorization Data transformation and analysis CSVs Fifth iMarine Board meeting, June 2014, FAO (Rome)

NEAFC to ICES data exchange: the iMarine workflow proposal Fifth iMarine Board meeting, June 2014, FAO (Rome) Most tasks of the current workflow can be automated and streamlined with the support of the e-infrastructure Expected benefits Reduced need for human supervision Possible expansion to further domains Produced data could be further transformed and processed (integration with other processes in the e-infrastructure) Security and confidentiality can be enforced at any level HTTPS, Authentication, IP filtering PGP Accounting iMarine, NEAFC) NEAFC to ICES data exchange support

Proposed workflow NEAFC Data Exchange WS Data Exchange WS iMarine Internet ICES VRE ICES CSVs Portlet RDBMS Raw Data Raw Data Accounting Fifth iMarine Board meeting, June 2014, FAO (Rome)

NEAFC to ICES data exchange support Revised interactions NEAFC RDBMS HTTPD ICES Internet CSVs Raw Data Raw Data Anonymization and categorization Data transformation and analysis CSVs Fifth iMarine Board meeting, June 2014, FAO (Rome)

The data exchange web services Fifth iMarine Board meeting, June 2014, FAO (Rome) NEAFC to ICES data exchange support Simple and lightweight Standard API (reusable on different domains) Enforce data anonymization as part of their contract (Optionally) can encrypt data to enforce confidentiality Provide accounting and logging facilities Produce data upon request (no need for dedicated people in NEAFC, documents will not be stored on the filesystem) Can be easily extended to provide additional filtering capabilities (by past semesters, by operation area etc.) Source code is available and open to contributions

Fifth iMarine Board meeting, June 2014, FAO (Rome) NEAFC to ICES data exchange support Further considerations Sharing data through iMarine proved to be: Simple (just a matter of plugging in a web service) Cost effective (maintenance tasks can be greatly reduced) Secure (platform accounting facilities) Furthermore, the collaborative approach and the design principle behind the revised workflow would allow users to: Easily share data among people of the same community Add further data processing capabilities (new algorithms, distributed infrastructure) Re-apply the same principles to any data repository with little effort

Fifth iMarine Board meeting, June 2014, FAO (Rome) NEAFC to ICES data exchange support What’s next? Finalization of the deployment of the Data Exchange WS Finalization of the ICES VRE and data access portlet Test and validation Future plans and expansions Transformation of raw NEAFC data into FishFrame format (for ICES consumption). Will require: Vessel positions interpolation (to be made available as a standalone process for maximum reusage) Data aggregation by position (cSquare) and categorization (by fishing activity / métier)

Thank you! Fifth iMarine Board meeting, June 2014, FAO (Rome) NEAFC to ICES data exchange support