GSICS 2015 GDWG Breakout Session 1 GSICS Collaboration Servers Peter Miu EUMETSAT CMA, CNES, EUMETSAT, ISRO, IMD, JMA, KMA, NASA, NIST, NOAA, ROSHYDROMET, USGS, WMO

GSICS 2015 GDWG Breakout Session Overview  The fundamental operations behind the GSICS collaboration server network are:  Collaboration network should inherently have:  Redundancy  Some Backup  Regional “faster” access 2 OperationCMAEUMNOAA Data Exchange: Source Data Exchange: Intermediate Data Preservation: Intermediate Data Archiving: Intermediate Data Upload: GSICS Products Data Preservation: GSICS Products

GSICS 2015 GDWG Breakout Session 3 Data Management Servers  Collaboration Server Network Status: 3 OperationCMAEUMNOAA Source Data Availability Intermediate Data Availability GSICS Products

GSICS 2015 GDWG Breakout Session To discuss  Configuration Updates  Data Preservation (later)  Operations Tasks  Product Development Tasks  Server Level Agreements for Operations  Point of Contacts 4

GSICS 2015 GDWG Breakout Session Data Preservation  Data Preservation (DP) means reliable access to the data.  Collaboration network shall DP – endorsed by EP  Collaboration network context; replication of GSICS products.  Concept is similar to the Hadoop Distributed File System (HDFS)  Data is uploaded 1 server  Data automatically replicated on all collaboration servers 5

GSICS 2015 GDWG Breakout Session Data Preservation Constraints Encountered  File Sizes – Define Limitation  Transfer Speeds - Limitation  Security – To Overcome  Development Resources  Financial Resources 6

GSICS 2015 GDWG Breakout Session Data Preservation Prototype  EUMETSAT has developed a prototype.  Based on Git  Master Slave replication.  Each collaboration Server will acts as a Master for its own GSICS products  Each collaboration Server will act as a Slave for other collaboration servers’ products 7

GSICS 2015 GDWG Breakout Session Implementation Details  Installation of Git on each Server  Installation of Python 2 v2.6 or greater on each Server (Not Python 3 – syntax changes)  Installation of Python scripts to detect and submit new files into the GIT repository  Installation of Python scripts to detect new files on the Master and download these to the Slave 8

GSICS 2015 GDWG Breakout Session Demo (VPN) / Implementation Overview 9

GSICS 2015 GDWG Breakout Session Data Preservation Implementation Recommendation & Actions 10

GSICS 2015 GDWG Breakout Session Additional Collaboration Server Tools  Tools to support Usability:  Plotting tool enhancements  New Products Notifications  Subscription and auto-downloads To be discussed in Agenda Item 6e, GSICS Tools 11

GSICS 2015 GDWG Breakout Session Planned Products: Source Data Sets  CMA  EUMETSAT  JMA  KMA  NOAA  Others (ISRO/IMD) 12

GSICS 2015 GDWG Breakout Session Planned Products: Intermediate Data Sets  CMA  EUMETSAT  JMA  KMA  NOAA  Others (ISRO/IMD) 13

GSICS 2015 GDWG Breakout Session Planned Products: Products  CMA  EUMETSAT  JMA  KMA  NOAA  Others (ISRO/IMD) 14

GSICS 2015 GDWG Breakout Session Agreed actions Summary  Server Configuration Updates?  Data Preservation Recommendations?  Server Updates? 15

GSICS 2015 GDWG Breakout Session 16 End of Presentation: Thank you for your attention