1 Improving Data Mobility & Management for International Climate Science, & Challenges in the provision of large data sets to multiple user communities July 2014, Boulder CO MONITORING WEATHER AND CLIMATE FROM SPACE

2 Improving Data Mobility & Management for International Climate Science, & Mission Statement We are the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT). EUMETSAT is an intergovernmental organisation and was founded in Our purpose is to supply weather and climate- related satellite data, images and products 24 hours a day, 365 days a year to the National Meteorological Services of our Member and Cooperating States in Europe, and other users worldwide.

3 Improving Data Mobility & Management for International Climate Science, & EUMETSAT Headquarters (Darmstadt, Germany)

4 Improving Data Mobility & Management for International Climate Science, & EUMETSAT Data Services - High Level Overview

5 Improving Data Mobility & Management for International Climate Science, & EUMETSAT Data Services - Data Flows EUMETSAT GROUND INFRASTRUCTURE

6 Improving Data Mobility & Management for International Climate Science, &  Single Access Point  One stop shop  Data discovery  User registration (>10.000)  Subscription for data services:  Near-real time data  Online Data Access  Long Term Archive  Additional  Image Gallery (WMS/ WCS)  User Notifications  Operational Real-Time Services Status Indicator EUMETSAT Earth Observation Portal

7 Improving Data Mobility & Management for International Climate Science, & EUMETCast Dissemination Near Real Time data delivery to users in 3 continents via Satellite Users (2/2014) User User 3,280 users worldwide, 4168 stations 5 new users per month EUMETCast Americas SES 6, C-BAND EUMETCast Africa EUTELSAT 5 WEST A, C-BAND EUMETCast Europe EUTELSAT 9 A, KU-BAND

8 Improving Data Mobility & Management for International Climate Science, & EUMETCast Dissemination: Integrating terrestrial dissemination

9 Improving Data Mobility & Management for International Climate Science, & MULTICAST ACCROSS GÉANT and NREN INFRASTRUCTURE Exploring the multicast usage on GÉANT infrastructure in Europe and their world wide connectivity;  2011 Proof of concept on terrestrial multicast on GÉANT test bed  month feasibility study on terrestrial multicast via GÉANT  2013 Multicast test to KMA Korea  2014 Multicast test to NOAA (Boulder)  2014 Start of 2 year prototype emulating operational scenarios of multicast across Europe and our overseas partners using GÉANT, NREN, I2 and Teint infrastructure.

10 Improving Data Mobility & Management for International Climate Science, & EUMETCast Terrestrial real-time monitoring -Multiple Multicast channels -SSM mode Mbit/s -FEC ¾ -Closed loop monitoring based on successful file receipt -Packet/channel based ack/nack retransmission possible but not yet enabled Acknowledgements: N-WAVE&NOAA Alex Hsia, David Hartzell

11 Improving Data Mobility & Management for International Climate Science, & EUMETCast Dissemination: Low cost, off the shelf Receiving Station DVB PCI Card EKU Antenna Laptop/ PC Software DVB USB DVB Router Typical total cost of a station < 2000 Euros

12 Improving Data Mobility & Management for International Climate Science, & EUMETSAT Long Term Archive  Archived data starting 1981  All EUMETSAT Satellite data including reprocessed data  > 1 Petabytes in the archive for MTP, MSG, Metop  Retrieval/Ingestion: 10 to 1  Long-Term Data Preservation in practice since 1995;  Disaster resilient - 3 rd Copy  Access via EUMETSAT EO Portal

13 Improving Data Mobility & Management for International Climate Science, & What we are working on…

14 Improving Data Mobility & Management for International Climate Science, & Outlook – Enhance Future abilities in terms of Scalability Flexibility, Responsiveness to emerging needs, Resource efficiency While aiming for: Better sharing of resources, Avoidance of data islands, Reducing costs Fit the organisational structure

EUM/TSS/VWG/14/762641, 7 July Many aspects to consider => Challenge Viewing Angle

16 Improving Data Mobility & Management for International Climate Science, & Outlook - Product delivery and management Generation of user tailored products Efficient for write once, read many times Suitable products for user needs and dissemination infrastructure (bandwidth) Avoid/ minimise processing where possible Efficient for slicing Usage of Format evolutions Secure the raw data (golden copy) Initially smaller products which increase over time Reprocessing to re-build formats

17 Improving Data Mobility & Management for International Climate Science, & Outlook - Bringing data to Users Adaptive and flexible services to deliver data to users/ communities (e.g. EUMETCast) Community driven formats optimised for dissemination mean Serving large user groups from different communities Managing data policies in accordance with data provider requirements via a data policy and legal framework support by the related technical mechanisms

18 Improving Data Mobility & Management for International Climate Science, & Outlook - Bringing Users to data  Standardisation activities in relation to Discovery, Access and Retrieval (DAR) of data  Strong focus on interoperability and ease of user experience  One stop shop experience for discovering services and data  Facilitating the Interoperability of portals by exploiting standards (OGC, INSPIRE)  Enhance Long Term Archive, Online Data Access  Outreach activities  Exploiting capabilities to execute user algorithms on data  Post-Processing, Hosted Processing

19 Improving Data Mobility & Management for International Climate Science, & Outlook - Information to Users Delivering Information to users rather than data Processing where the data is located Service Interfaces instead Facility interfaces Application of (private) cloud principles

20 Improving Data Mobility & Management for International Climate Science, & Thank you for your attention