1. 1. GSICS Collaboration Servers Synchronisation 2
1. GSICS Collaboration Servers Synchronisation 2. NetCDF Generation Framework
XU Zhe
CMA GDWG

2. Foundation Updated to the latest THREDDS configuration in summer 2016
Following the convention of GDWG THREDDS structure
Foundation for collaboration servers synchronization
GSICS Processing and Research Centres (GPRC)

3. Motion and Challenge
EP: Collaboration Servers are expected to replicate all RAC products between themselves
Challenge: Agency’s security policy
CMA: External upload is prohibited
CMA: Internal release internet content are not allowed to put on web server directly, synchronized by “Tamper-resistant ” server
GSICS Processing and Research Centres (GPRC)
Operation Network
DMZ
Product Generate
Tamper-resistant
Web Server

4. Experimental Solution
Data flow: GET and PUT
GET：Data collecting server, download file from EUMETSAT/ECMWF/NCEP etc.
PUT：Tamper-resistant server, a back up for all static content on Web sever.
Scheduled routine job, 1/day
For backup, no timeliness requirement
Shell script and a configuration file
DMZ
Data Collecting
Remote Servers
Tamper-resistant
Web Server

5. Configuration file Remote server information
3 columns: HTML Path, Remote path, Filename string pattern

6. Shell Script (50 lines) Configuration Loop job Log
Remote server: URL by using configuration file
Local server: Directory
Put destination server: Username/PS/Dir…
Loop job
Wget
Free utility for non-interactive download of files from the Web;
supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies
Simple invoke
Manual book:
Log
Wput

7. EUM Replicate Status Green: Synchronisation Yellow: No file
Orange: More than 1 file

8. NetCDF Generation Framework Discussion
Pros:
Great idea for GSICS product generation standardized
i. Read source data;
ii. Find match data and calibrate
iii. Generate NetCDF file
Convenient for the agencies who don’t have product
Cons:
Engineer : Lack of scientific knowledge and background
Case in CMA:
CMA GOE/LEO IR product: Python
GRWG find resource to implement
Discussion:
Demonstration code
NetCDF Format Checking tool

9. Thank you for your attention
WMO GSICS Portal
GSICS Coordination Centre
GSICS Product Catalog
GSICS Wiki