1. Page 1 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 V-GISC – SIMDAT Gil Ross (Met Office UK) NESC Workshop 6th to 8th September 2005

2. Page 2 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 Contents  Part 1  World Meteorological Organisation  Data - Communications now  WMO Operational Meteorology  Real-Time Daily Data Flows  WMO Information System  GISC - Global Information System Centre  data Portal data Mirror  Part 2  vGISC and SIMDAT  SIMDAT EU Project  SIMDAT Meteorology

3. Page 3 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 WMO is….  UN Agency (1950)  derived from IMO (1873)  Co-ordinate meteorology  weather  observations  forecasts  climate change  hydrology  disaster mitigation  187 Members

4. Page 4 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 WMO - Data Exchange  Now  GTS Global Telecommunications System  Observations  Forecasts  Warnings  Private Network  Node to Node network  NOT an Internet  Future  WIS - WMO Information System  Replacement for GTS  Portal for Operational Meteorology  Catalogue driven based on -  ISO 19115 Metadata WMO Core Profile

5. Page 5 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 GTS Real-Time Data in Met Office UK  Main Message Switch  300,000 bulletins received per day  5.2 million bulletins sent/day.  Main File Server  17,000 files received/day  27,000 files sent/day  Other routes – e.g. Satellite data ~ 27GB/day  Ephemeral and Persistent databases  Ephemeral completely repopulated every day  Persistent < 1 TB new data per day

6. Page 6 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 Surface Observations 0000Z 3rd September http://www.metoffice.gov.uk/research/nwp/observations/data_coverage/index.html

7. Page 7 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 Aircraft observations 000Z 3rd September

8. Page 8 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 WIS - Information collection data flow

9. Page 9 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 WIS - Information distribution data flow

10. Page 10 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 Part 2

11. SIMDAT / NESC Workshop Sept 05 11 SIMDAT V-GISC Gil Ross Met Office V-GISC – SIMDAT

12. SIMDAT / NESC Workshop Sept 05 12 SIMDAT V-GISC vGISC Instead of each National Met Service having a GISC (Global Information System Centre) a “distributed” or “virtual” GISC 3 NMSs –  Meteo France  Deutcher Wetterdienst DWD  Met Office UK – MetO and two DCPCs Data Collection and Production Centre  ECMWF  EUMETSAT

13. SIMDAT / NESC Workshop Sept 05 13 SIMDAT V-GISC SIMDAT 4 years project funded by the EU  Contract with EU was signed on 1 September 2004 Data Grids for Process and Product Development using Numerical Simulation and Knowledge Discovery SIMDAT focuses on 4 application area:  product design in automotive and aerospace,  process design in pharmacology  service provision in meteorology Budget of 11 M € of which 10.5% for meteorological activity  project managed by ECMWF

14. SIMDAT / NESC Workshop Sept 05 14 SIMDAT V-GISC SIMDAT - Consortium

15. SIMDAT / NESC Workshop Sept 05 15 SIMDAT V-GISC SIMDAT Seven Grid-technology areas have been identified to achieving SIMDAT objectives:  Integrated Grid infrastructure offering basic services to applications  Access to data distributed on Grid sites  Management of Virtual Organisation  Workflows  Ontologies  Integration of analysis services  Knowledge Services Not all will be addressed in Meteo - SIMDAT

16. SIMDAT / NESC Workshop Sept 05 16 SIMDAT V-GISC SIMDAT - Strategy. Virtual Data Repository. Introduction of grid technologies research The heart of the issue is data Phase 1 : Connectivity Phase 2 : Interoperability Phase 3 : Knowledge. Deployment of Grid infrastructure with particular attention to data transport and management.. Distributed DB access Workflows for next- generation aggregated knowledge capture, discovery and mining.

17. SIMDAT / NESC Workshop Sept 05 17 SIMDAT V-GISC SIMDAT - vGISC vGISC working groups adopted by SIMDAT (meteorology) SIMDAT is a technical solution for vGISC type operations  will NOT deliver a vGISC  has specified requirements similar to vGISC  is investigating a vGISC Virtual Organisation  is developing catalogue operations  has developed message infrastructure SIMDAT 18 month test system due - NOW!

18. SIMDAT / NESC Workshop Sept 05 18 SIMDAT V-GISC SIMDAT requirements

19. SIMDAT / NESC Workshop Sept 05 19 SIMDAT V-GISC DCI Distributed Architecture

20. SIMDAT / NESC Workshop Sept 05 20 SIMDAT V-GISC VGISC Demonstrator: UKMO local db (1)

21. SIMDAT / NESC Workshop Sept 05 21 SIMDAT V-GISC VGISC Demonstrator: UKMO local db (2) Local database is a derivative of JEDDS – the ‘Joint Environmental Dynamic Data Server’ Originally developed by Met Office to provide data on demand to UK military Features:  Live data feed; continuously refreshed  Serving global TAF, METAR & GRIB bulletins  Request-oriented SOAP web-service interface

22. SIMDAT / NESC Workshop Sept 05 22 SIMDAT V-GISC VGISC Demonstrator: UKMO local db (3)

23. SIMDAT / NESC Workshop Sept 05 23 SIMDAT V-GISC Catalogue Status in Test system

24. SIMDAT / NESC Workshop Sept 05 24 SIMDAT V-GISC Working decisions (1) Teleconference, wiki & email used for remote collaboration  focus on HOW the catalogue operates, not the minutiae of metadata standards  use a ‘random’ UID scheme and not try to ‘overload’ identifiers with semantics  use a simple ‘Google-like’ and/or directory-based search interface  3 classes of search – by Keyword, time and/or location  ignore internationalisation for now  Issue: WMO Core (& ISO19115) is TOO flexible – there are too many options for spatial & temporal descriptions … we need best practice ‘application schemas’

25. SIMDAT / NESC Workshop Sept 05 25 SIMDAT V-GISC Working decisions (2) Metadata  create linking schemas using the WMO Core Profile by using additional namespaces (i.e. )  metadata documents should be split into a ‘hierarchy’ of XML ‘fragments’ are used within the catalogue  model for merging metadata fragments will uses Java templating rather than an XSLT-based approach … key issue to demonstrate the concept is WHAT the fragments are not the mechanics of HOW they are merged. This position may be revised post-demonstrator  Agreement: nomenclature … a specific SYNOP (12:00UTC, 16/06/05 at Heathrow) is an instance the SYNOP ‘product’ is a generalised instance that requires instantiation parameters to identify a specific synop instance

26. SIMDAT / NESC Workshop Sept 05 26 SIMDAT V-GISC What Now? Metadata demo http://www.ecmwf.int/products/realtime/d/vgisc/ Re-consider parked issues extend beyond real-time data Project tasks