21 November 2018 Data Science Capabilities

NIWA: New Zealand’s Research Institute for Atmosphere, Climate, Freshwater and Oceans

NIWA Science and Applications Applied Science Environmental Monitoring Hazards and Risks Resource Management Energy Transport Agriculture Aquaculture / Fisheries Core Science Atmosphere and Climate Freshwater and Coasts Ocean, Aquaculture, and Fisheries

Research facilities in NZ Centres and Unidata in Perth NZ’s Research Vessels National / International Monitoring Networks ~700 staff NZ’s HPFC

NIWA Environmental Monitoring 350+ reference weather observations across New Zealand and growing (incl soil moisture). 300+ river flow and lake level stations (Hydro generation) Instrumentation of >20 irrigation schemes with completely automated water monitoring and control systems. IoT self deployed devices GHG and Atmospheric reference monitoring (WMO) Research vessel fleet Research drone fleet

NIWA Data Management Terrestrial sensor stations: Real-time ~2,000 / Historic ~10,000 DAS database: ~600 million records of sensor observations from research vessels, mostly in NZ & Antarctic waters. Fisheries acoustics: ~20,000 echo sounder transects GIS Platform for spatial data products based on ESRI software Imagery: ~150,000 high resolution research digital photographs Archives for freshwater and ocean biological observations (~ 3M records) Archives for ocean observations (~2M records)

NIWA OGC Data Services Developments Challenges in deploying standards based services

NIWA Data as Open Standard Data Services How NIWA is working with Open Standards Data Services NIWA Data Catalogues – Catalog Services (through GeoNetwork) NIWA Station Directory – Web Feature / Map Services (through GeoServer) NIWA Ocean Data – Web Feature / Map Services (through ODN) NIWA Timeseries Data – Sensor Observation Services (through 52 N) NIWA Bio Data – Darwin Core (through IPT) NIWA GIS Data – Web Feature / Map Services (using ESRI) NIWA works on several work streams on providing NIWA data services using Open Standards.

Sensor Observation Services NIWA Sensor Observations Sensor Observation Services NIWA publishes some of its sensor data using OGC SOS

New Zealand Ocean Data Network The New Zealand Ocean Data Network uses OGC standards to publish metadata and data at http://NZODN.NZ New Zealand Ocean Data Network NIWA has adopted the opens sources stack developed by the Australian Integrated Marine Observing System (IMOS), called “Ocean Data Network” to publish Ocean Physics / Chemistry Observations

Biological Observations NIWA Marine and Freshwater Biological Observations NIWA publishes Marine and Freshwater Observations using IPT servers in Darwin Core Standards

NIWA and open standard web services Some of the issues we are struggling with … Different standards implementations (e.g. Kisters adding required SOS query parameter) Using standard web services for ‘closed data’ (e.g. hydro energy) – how to do authentication and security? Different data cataloguing systems: ESRI vs GBIF (EML) vs OGC -> Not easy “to find everything relevant to my domain of interest” GBIF (Darwin Core) vs OGC? – A O&M profile based on Darwin Core? How to use vocabularies – best practice?? We (still) see different providers varying their implementations of OGC standards – is there scope for a OGC ‘certification’? There are and will be many ‘closed data applications’, for example NIWA monitors for competing hydro energy providers and needs to ensure data privacy and security – how can we do that using OGC standards for data services? How to bridge the gap between existing ‘competing’ mainstream data services, e.g. ESRI, GBIF, OGC, (others?)? Example: NIWA provides Ocean Physics data through NZODN, Ocean GIS data using ESRI; Ocean Biodata to GBIF – not easily reconsilable Supporting semantic searches: ‘Best practice for using vocabulary services’??