WIGOS Metadata Standard & OSCAR/Surface … Supporting adequate use of observations & rational network evolution   Jörg Klausen, MeteoSwiss Timo Pröscholdt, WIGOS PO, WMO Secretariat

Outline Motivation WIGOS Metadata Standard OSCAR and OSCAR/Surface Initial data sources & Vol A transition Exchange format & automation Summary

Why do we need Metadata and OSCAR? Motivation

What are metadata? Why are they essential? Metadata = documentation describing data, context of observations Observed variable Location Environment Intended use … Essential data are exchanged globally, but metadata are not. Metadata are needed to make adequate use of observations globally, but they are not always easily obtained by users. T=17°C T=17°C T=17°C T=17°C

What are metadata? Why are they essential? AWS #1 AWS #2 Metadata = station exists, where is it? latitude longitude elevation name .. Metadata allows for more efficient network planning. All stations, including those of partners, can be taken into account when extending the network. The photos show three AWS of different organizations in the same site, each having separate instruments measuring identical variables. AWS #3

Members shall share metadata … WIGOS Manual

Use of Metadata OSCAR/Analysis OSCAR/Surface OSCAR/Requirements Evolve observing systems rationally  WIGOS “Rolling Review of Requirements” Process Metadata & Analysis of Observing Systems Societal Impact Improved services for society OSCAR/Analysis «Know the gap» Analysis «Critical Review» Guidance for Action «Know what we have» OSCAR/Surface Capabilities Surface-based observing systems Space-based observing systems «Fill the gap» RRR OSCAR/Requirements Requirements «Know what we need» OSCAR/Space Enable adequate use of observational data  Operational Meteorology, Climatology, Public Health, …

WIGOS METADATA STANDARD (WMDS)

WIGOS Metadata Standard 1. Observed variable 2. Purpose of observation 3. Station/ platform 4. Environment 5. Instruments & methods of observation 6. Sampling 7. Data processing and reporting 8. Data Quality 9. Ownership and Data Policy 10. Contact StationSet Station/Platform (fixed, mobile) Observation Contact Deployment Instrument Procedures Contact Contact

OSCAR & OSCAR/Surface

WIGOS Priority 3: WIGOS Information Resource Portal providing all relevant information on WIGOS Background material Operational status of WIGOS component systems Evolution of WIGOS implementation WIR SORT The WIR provides … The WIR has 2 main components SORT Regulatory material >>> REPOSITARY OF REFERENCE DOCUMENT AND INFORMATION, MORE TO SAY ??’ … OSCAR OSCAR is being developed to document the observational assets of NMHSs and their partners. It consists and consists of 4 components Requirements Surface capabilities Space capabilities Analysis Requirements Surface capabilities Space capabilities Analysis

OSCAR/Surface web application Industry-standard technology stack Oracle DB, ArcGIS JEE, AngularJS Safe and secure, traceable Publicly available for anonymous use Management console for registered users oscar.wmo.int/surface

OSCAR quick access & reports Map filter + click = Detailed station report

Various search targets OSCAR detailed search Various search targets Instruments (soon) Finely tuned search

Download for further analysis GoogleEarth, QuantumGIS, … OSCAR search results Download for further analysis Integrate with geospatial tools GoogleEarth, QuantumGIS, … Mashup for online maps

Security and user management Authentification by identity provider (Swiss Government) Authorization within application based on «trust-relationships» and various «user roles» Register Edit all stations in country Also: register new contact Admin NFP Edit assigned stations Also: register new contact Station contact

OSCAR management console Stations Basic characteristics Photos History Observations Location Variable Methods Instruments Quality and uncertainty Contacts Bibliographic references & documents

INITIAL DATA & Vol A transition

Data integration # stations GAWSIS WMO Pub 9 Vol A JCOMMOPS 1’053 13’026 11’387 762 GAWSIS Metadata for the Global Atmosphere Watch WMO Pub 9 Vol A Catalogue of synoptic and upper-air stations of GOS JCOMMOPS Marine element of GOS / GOOS WMO Radar DB World-wide radars Amdar Coming soon … 32’417 stations/platforms as of 9 May 2017

Vol A integration Vol A contained very limited metadata Station metadata incl. station «class» Schedules of observation Observed variables inferred from station «class» based on Manual on GOS Automatic station (from “AUT” flag in ObsRems) PTU, Wind, “Precipitation (yes/no)” Manned station PTU, Wind, plus "Present weather", "Past weather", "Cloud amount", "Type of cloud", "Height of cloud base", "Visibility“  Too many observations assumed for some stations

Vol A integration (cont’d) OSCAR generates reports similar to the historical Vol A flat-files. The new “Vol A legacy report” has all current column headings but does not contain information in all columns. Station information provided to WMO via current Vol A procedures are used to update WIGOS metadata in OSCAR (manually!) By beginning of 2018, Vol A will officially cease to exist and related procedures used to maintain station information will be discontinued. Machine-to-machine interface for automatic metadata transfer under development, expected for testing in July 2017

Focal Points for OSCAR/Surface as at 11/05/2017

EXCHANGE FORMAT & AUTOMATION

Mass changes / batch processing OSCAR machine-to-machine interface (API) for automated delivery of metadata from existing national or regional station databases retrieval of metadata for specific analyses and planning Exchange format based on WIGOS metadata standard OSCAR API WDC National DB Regional DB

WIGOS Metadata Representation Development of UML model by IPET-MDRD sub-group Model uses OGC/ISO19156 standard (Obs & Meas) Consultations prior to CBS-16 confirmed approach REST API (under development) for submission of full records or increments or parts, e.g., new observation at existing station/platform ObsFacility Instruments Observations Variable Method (homogeneous) Segments Logbooks Event Maintenance Calibration Processes Schedule Sampling Processing Reporting

summary

Integration of OSCAR components by 2019 September 11, 2018 WIGOS & OSCAR T=17°C T=17°C What? Where? How? Why? Integration of OSCAR components by 2019 T=17°C Metadata matter! API Operational since 2 May 2016 WIGOS Metadata Standard Enable adequate use of observations Evolve observing systems rationally … nationally, regionally, world-wide Start with a simple number: 17 … what is it? Ah, it’s temperature … but in what unit? Oh, there’s more temperature observations , all 17°C … need to know where, how, why! WIGOS Metadata Standard specifies essential metadata elements that give necessary context to observations, to make adequate use of observations, to support gap analysis to evolve observing systems rationally Four OSCAR components support the WIGOS RRR, a cycle of requirements specification, catalogueing capabilities, and gap analysis OSCAR/Requirements OSCAR/Space OSCAR/Surface OSCAR/Analysis (a future module) OSCAR/Surface has been developed in a joint venture of WMO and MeteoSwiss, and has been operational since May 2016. The web interface is a reference implementation of the WIGOS Metadata Standard and brings together information on observing systems on land, in the air and at sea. Humans can interact with the application and provide metadata manually. Once the machine-to-machine interface to OSCAR/Surface is ready, it will be possible to document large observing systems comprehensively. Approved by Cg-17 Amendments by CBS-16 Expected in 2017

For more information oscar@wmo.int https://oscar.wmo.int/surface

شكرا Thank you Merci You & your organization! Financial support. Swiss Federal Office of Foreign Affairs, MeteoSwiss, WMO, Met Norway Project Team at MeteoSwiss. (current) J Klausen, L Cappelletti, B Calpini, M Musa, M Brändli, L Koppa, C Walder, E Grüter, S Sandmeier, M Schäfer, A Rubli, Tom Hager, Attila Loos; (past) J Mannes, S Spreitzer, M Leutenegger, C Sigg, M Abbt, W Brunelli, J Mettler Project Team at WMO (current). F Belda, LP Riishojgaard, T Pröscholdt Project Team at European Dynamics (current). T Galousis, M Ulmann, L Christou, N Pappa, S Sklavos, … ICG-WIGOS. S Barrell, B Calpini, … TT-WMD. (current) K Monnik, J Klausen, J Swaykos, T Boston, U Looser, E Büyükbas, Zhao Licheng, T Oakley, S Foreman, D Lockett, L Nunes IPET-MDRD. D Lowe, J Tandy, … JCOMMOPS, GAW WDCs, ET-WDC, …