Sharing and visualizing earth science data with Web Services and Virtual Globes Jon Blower (with help from lots of others!) Reading e-Science Centre Environmental.

Slides:



Advertisements
Similar presentations
Web Mapping Service Development For DMSP Satellite Data Chris Elvidge, NOAA National Geophysical Data Center Ben Tuttle, Cooperative Institute for Research.
Advertisements

A Roadmap of Open Source components for GI Web Services and Clients A Paul R Cooper MAGIC.
© University of Reading 2007www.resc.reading.ac.uk 11 September 2007 Sharing and Visualizing Environmental Data using Virtual Globes Jon Blower, Alastair.
1 ShareGeo Discovering and Sharing Geospatial Data
Web GIS Oregon Explorer Marc G Rempel Oregon State University The Valley Library Oregon Explorer
BlogMyData A Virtual Research Environment for collaborative visualization of environmental data Andrew Milsted, Jeremy Frey Department of Chemistry, University.
NCAR GIS Program : Bridging Gaps
BlogMyData A Virtual Research Environment for collaborative visualization of environmental data Andrew Milsted | 14 September 2010.
Dynamic Quick View, interoperability and the future Jon Blower, Keith Haines, Chunlei Liu, Alastair Gemmell Environmental Systems Science Centre University.
Planned Title: Review of Evaluation of Geospatial Search Allan Doyle.
The MashMyData project Combining and comparing environmental science data on the web Alastair Gemmell 1, Jon Blower 1, Keith Haines 1, Stephen Pascoe 2,
Development of a Community Hydrologic Information System Jeffery S. Horsburgh Utah State University David G. Tarboton Utah State University.
Exploring large marine datasets using an interactive website and Google Earth Jon Blower, Dan Bretherton, Keith Haines, Chunlei Liu, Adit Santokhee Reading.
Internet enabled GIS GIS Topics and Applications John Reiser Rowan University.
NERC Data Grid Helen Snaith and the NDG consortium …
1 Alternate Title Slide: Presentation Name Goes Here Presenter’s Name Infrastructure Solutions Division Date GIS Perfct Ltd. Autodesk Value Added Reseller.
Marine GIS Applications using ArcGIS Global Classroom training course Marine GIS Applications using ArcGIS Global Classroom training course By T.Hemasundar.
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.
The use of standard OGC web services in integrating distributed model, satellite and in-situ datasets Alastair Gemmell Jon Blower Keith Haines Environmental.
Leica TITAN Amy Zeller Leica Geosystems
2012 National BDPA Technology Conference Creating Rich Data Visualizations using the Google API Yolanda M. Davis Senior Software Engineer AdvancED August.
Tools for accessing distributed in-situ data collections Donald W. Denbo, NOAA/PMEL-JISAO Jason E. Fabritz, NOAA/PMEL-JISAO Bernard J. Kilonsky, Sea Level.
GIS technologies and Web Mapping Services
© University of Reading 2008www.reading.ac.uk Reading e-Science Centre September 10, 2015 Integrating a Web Map Service into the THREDDS Data Server Jon.
Athens Feb 2008 EuroMISS and EuroDess Viewing and Validation Services Keith Haines.
Lecture 3: Geospatial Web Services From Web sites to Web services Geospatial Web service functions Web service types Interoperability and geospatial service.
AIRNow-International The future of the United States real-time air quality reporting and forecasting program and GEOSS participation John E. White U.S.
Interoperability ERRA System.
GADS: A Web Service for accessing large environmental data sets Jon Blower, Keith Haines, Adit Santokhee Reading e-Science Centre University of Reading.
NEPAnode is a Geospatial Data and Document Management System It provides a centralized and collaborative site to access the data.
, Increasing Discoverability and Accessibility of NASA Atmospheric Science Data Center (ASDC) Data Products with GIS Technology ASDC Introduction The Atmospheric.
, Implementing GIS for Expanded Data Accessibility and Discoverability ASDC Introduction The Atmospheric Science Data Center (ASDC) at NASA Langley Research.
(Images from NOAA web site). How to use satellite data ?
Unidata’s TDS Workshop TDS Overview – Part II October 2012.
Open Source Web Mapping Server Products (Spatially-enabled Internet applications)‏ Rex Thaxton & Jerry Workman Mountain CAD Corporation 339 Sixth Ave.
Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.
STOQS: The Spatial Temporal Oceanographic Query System Mike McCann Abstract Monterey Bay Aquarium Research Institute Architecture Postgres.
An Introduction To Building An Open Standard Web Map Application Joe Daigneau Pennsylvania State University.
material assembled from the web pages at
National Earth Science Infrastructure Program AuScope Limited Headquarters School of Earth Sciences University of Melbourne Victoria 3010 Tel
1 The NERC DataGrid DataGrid The NERC DataGrid DataGrid AHM 2003 – 2 Sept, 2003 e-Science Centre Metadata of the NERC DataGrid Kevin O’Neill CCLRC e-Science.
The OpenGIS Consortium Geog 516 Presentation #2 Rueben Schulz March 2004.
Achieving Interoperability using the ArcGIS Platform
NOCS, PML, STFC, BODC, BADC The NERC DataGrid = Bryan Lawrence Director of the STFC Centre for Environmental Data Archival (BADC, NEODC, IPCC-DDC.
MapTube and Streaming Google Earth – Sharing Data via Anonymous Servers Richard Milton and Andrew Hudson-Smith CASA, UCL UCL CENTRE FOR ADVANCED SPATIAL.
NcBrowse A Graphical netCDF/OPeNDAP Browser Donald Denbo 1 & John Osborne 2 1 UW/JISAO-NOAA/PMEL, 2 OceanAtlas Software
1 NOAA IOOS Program Data Integration Framework (DIF) Project Overview Adapted from a brief to the NOAA Data Management Committee August 6, 2008 by Jeff.
ORNL DAAC ORNL DAAC: Access ORNL DAAC OGC WMS Service in Virtual Globe Tools Overview: Access ORNL DAAC Open Geospatial.
OceanBrowser viewing service, overview and upgrades Alexander Barth (1), Charles Troupin (2), Aida Alvera Azcárate (1), Jean-Marie Beckers (1) (1) University.
Requests from F. Blanc Retrieve information from each individual national reports and present this. –2. Data serving –The HYCOM products are freely available.
A GeoSpatial Mapping Architecture
GIS System Design for the Coastal Storms Initiative Nazila Merati OAR/PMEL & OAR Representative to NOAA Enterprise GIS Chris Moore – OAR/PMEL Tiffany C.
GEON2 and OpenEarth Framework (OEF) Bradley Wallet School of Geology and Geophysics, University of Oklahoma
2008 ESRI International User Conference “G eography in Action” GIS User Group Meeting September 19, 2008.
Tools for Monitoring in Coastal Alaska and the Arctic Darcy Dugan Alaska Ocean Observing System.
Distributed Data Analysis & Dissemination System (D-DADS ) Special Interest Group on Data Integration June 2000.
Using Google Maps and other OpenSource GIS software for displaying geospatial data Jon Blower, Dan Bretherton, Keith Haines, Chunlei Liu, Adit Santokhee.
Topic 10- GIS layers to web
Distributed Data Servers and Web Interface in the Climate Data Portal Willa H. Zhu Joint Institute for the Study of Ocean and Atmosphere University of.
OGC Web Services with complex data Stephen Pascoe How OGC Web Services relate to GML Application Schema.
GeoSpatial Analysis UNICEF Security Advisors Workshop 20 October 2010.
NOAA IOOS SOS Implementations in 2008 Jeff de La Beaujardière, PhD NOAA IOOS Program DIF Sr Systems Architect.
GeoServer Prof. Wenwen Li School of Geographical Sciences and Urban Planning 5644 Coor Hall
Reading e-Science Centre Technical Director Jon Blower ESSC Director Rachel Harrison CS Director Keith Haines ESSC Associated Personnel External Collaborations.
Data Browsing/Mining/Metadata
Zhong Liu George Mason University and NASA GES DISC
Data Are from Mars, Tools Are from Venus
Dwane Young, U.S. EPA Office of Water
Jeffery S. Horsburgh Utah State University
Presentation transcript:

Sharing and visualizing earth science data with Web Services and Virtual Globes Jon Blower (with help from lots of others!) Reading e-Science Centre Environmental Systems Science Centre University of Reading United Kingdom

The Problem discover analyse visualize lots of software packages! WHAT GOES HERE? lots of file formats! access control

Solution 1: Web interfaces to datasets Separate websites for each data provider

What’s wrong with Solution 1? Discovery relies on web search and “just knowing it’s there” Hard to intercompare data from different sites Slow route to visualization: –Download data file(s) –Import into your tool of choice –Produce image Can’t download subsets or aggregations of files Different login for each provider In summary: not very flexible! All this is because websites are designed for humans, not machines

Solution 2: Web Services Web Services Different user interfaces for different user communities

More about Solution 2 Each data provider provides “hooks” into the data store –Discovery: results = find_data(“salinity”) –Download: data = get_data(“temperature”, “North Atlantic”, “June 2006”, “NetCDF”) –Visualize: picture = get_map(“currents”, “global”, “ ”, “PNG”) These “hooks” are Web Services Third-party systems can use Web Services as plug- ins Different user interfaces can be built on top of the same system Much more flexible than Solution 1

A closer look at Web Services Designed for machine-to-machine interaction They are “subroutines” that run on remote machines Data and messages are exchanged in platform- independent formats Serve data to another system –with one notable exception (see later) Can be made to be secure You can do anything with Web Services! BUT to be useful, communities must agree on standards for compatibility

So the question becomes… WHAT EXACTLY GOES HERE? Different user interfaces for different user communities

Some standard Web Services for earth science data OPeNDAP (Open-source Project for a Network Data Access Protocol): –formerly known as DODS –allows clients to download data subsets –aggregates files into a single, logical whole –clients can treat remote data sets just like local ones –well supported by software tools and libraries –but requires clients to have advance knowledge of the data structure Open Geospatial Consortium (OGC): –Whole suite of Web Services for different situations –Provides a logical and semantic view of the data –Tooling support less than OPeNDAP but growing quickly

OGC Web Services Web ServicePurpose Web Map Service (WMS)Serves map images (e.g. satellite images) Web Feature Service (WFS)Serves geographical features (e.g. buoy locations, radiosonde profiles) Web Coverage Service (WCS)Serves multidimensional raster data (e.g. numerical model output, 3-D seismic data) Web Processing Service (WPS)Processes data Sensor Web Enablement (SWE) – coming soon Whole suite of standards for monitoring and managing sensor systems Geography Markup Language (GML) underpins OGC Web Services

OGC implementations GeoServer MapServer (U of Minnesota) Deegree ncWMS (for NetCDF data) Lots of commercial stuff…

Standards give interoperability! NASA World Wind Cadcorp SIS Google Earth Geo-website

Web Services in Action: UK NERC Data Grid (NDG) Provides access to atmospheric and oceanographic datasets produced by NERC projects Each data provider installs a suite of Web Services –nothing is centralized! Clients can use the NDG web interface or build their own interfaces NDG allows: –discovery of data –online visualization of data –exploration of metadata –download of data

Discovering and browsing data

Web Map Service (WMS) in Action: online data visualization “Godiva2” website gives very fast previews of 4-D data on an interactive website Reads data from NetCDF files and OPeNDAP servers Serves images through an "enhanced" WMS Draggable, zoomable map Allows the fast creation of animations

Selection of depth Select from all the depth levels of the model

Selection of time (range) Select from all the timesteps in the model Selection of a time range leads to an animation

Finding the data value at a point Click on the data layer, data value and precise position is shown Lon: Lat: Value: 19.27

Timeseries plots If a time range is selected, can create a timeseries plot at a point

Godiva2 architecture NetCDF Data sources Non-standard file format NetCDF Web interfaceVirtual GlobeGIS client WMS OPeNDAP

Web Services: conclusions Web sites can be excellent, but are "dead ends" for data –You can't build on top of a website Exposing data and metadata via Web Services allows: –Building of new interfaces on top of your data –Data from different locations to be brought together OGC standards are part of the story –They handle the "geospatial" component of data –Need separate web services for other things like vocab, property databases Web Services should be standards-compliant or simple (or both!) WS and standards are no good without tools Many providers now see more traffic through Web Services than their primary web site!

How can we bridge the gap? Paper PDF or Web site (text + images) Web Service For humansFor machines Dead end for data Open end for data KML

Virtual Globes Easy to use 3-D applications for visualizing environmental data –All scales from global to sub-metre Around 30 currently in existence! –Google Earth –NASA World Wind –ArcGIS Explorer –... Can combine data from numerous sources Enable discovery of data Use open standards –Simple data formats –Standard Web Services Often free or low-cost Generally can't do data analysis –Not replacements for fully-functional GIS systems! NSIDC Snow water equivalent in NASA World Wind

Keyhole Markup Language (KML) Balances simplicity with richness of representation –Simpler than GML –Richer than GeoRSS The format of Google Earth, but understood by many other systems: –Google Maps –NASA World Wind –ArcGIS Explorer –Other GIS software Encodes simple geographic features: –Points, lines, polygons (e.g. in-situ observations) –Image overlays (e.g. satellite images, model output) Can annotate features with more information –E.g. links to website Can easily be created from existing data (Excel, databases…) Now on the standards track through OGC

Quick comparison of 3 Virtual Globes Google Earth –Aimed at "the man in the street" –Easy to use –Poor support for OGC services –Big community NASA World Wind –Aimed at scientists –Portal to NASA satellite imagery –Next version will be exciting (pluggable, customizable) ArcGIS Explorer –Aimed at GIS community (esp. existing Arc users) –Can display subsurface and submarine data –Can write plug-ins in.NET –Very young

Virtual Globe strengths and weaknesses Strengths –Easy to use –Easy to visualize data from different sources –Provide "lightweight GIS" format: KML –Low cost –Some support for discovery through VG interface Weaknesses –Poor direct support for OGC services (WMS support patchy, WCS/WFS support almost non-existent) –Many historical GIS formats (shapefiles) not usually supported –Hard to visualize subsurface/submarine data

Google Earth and Web Services Poor native support for OGC web services BUT can link with a website that generates KML dynamically –Perhaps with data sourced from OGC services Some OGC implementations have KML as a direct output format Creative use of KML can lead to sophisticated systems! (We are preparing a community website for people to share tips on VGs and geo-web)

Communication of scientific phenomena Hurricane Katrina, August 2005 Picture left shows sea surface temperature (UK Met Office) and storm position/intensity (TRACK analysis of ECMWF data) Winds cause upwelling of cooler subsurface water on right-hand side of the cyclonic storm track (much more obvious in live system!)

Highlighting of risks Eruption of Cleveland volcano modelled by PUFF (Alaska VO) 4-D simulation of ash cloud –Represented in KML –Can be animated in Google Earth Could overlay with real-time aeroplane tracks for basic risk assessment

Monitoring an observing system BODC use Google Earth as spatial metadata browsing tool for in-situ measurements Can easily check for errors –E.g. Ocean data located on land –Misplaced component of linear ship track Displays “light” metadata, with link to more sophisticated information Developed in under a week!

Direction of missions with real-time data British Antarctic Survey (BAS) used Google Earth to direct 2 scientific cruises in 2006 Multiple data streams (ship location, sea temp and salinity, air temp and pressure) streamed to Google Earth in near real time Combined with info about wider environment Enabled real-time decision- making (e.g. tracking of predators, left) King penguin track overlain with concurrent chlorophyll and satellite imagery

Diagnosis of models and observations Picture left shows comparison of NEMO model and observations for Nov 2004 Red dots show bad model-obs fits, green dots are good fits Google Earth allows very efficient browsing of these large datasets Could read obs and model data from different sources and bring together in Google Earth or another client

Search and Rescue ESSC and BMT Cordah Use Google Earth as common platform for visualizing: –Oceanographic numerical model output data –SAR predictions Drives improvements to both models.

Conclusions Web Services avoid data "dead ends" –HTML and static images (websites) are "dead ends" –Web Services and GML are "open ends" –KML, SVG are somewhere in between Serving data "the right way" allows new science to be done! –Interoperability permits intercomparison –Fast route to visualization (WMS, KML) (The best technical solution is not always the most popular with users!) Virtual Globes provide an easy way to drive collaborative work: –Quick intercomparison of data –Data discovery –... but can never be the whole story

Some recommended web searches "REST vs SOAP" : different approaches to Web Services "OpenLayers" : open-source alternative to Google Maps "KML tutorial" : (esp. NetworkLinks…) "NASA World Wind" : open-source alternative to Google Earth "ArcGIS Explorer" : ESRI virtual globe (free) "GeoRSS" : Simple georeferencing (KML-lite)

Limitations of Virtual Globes Large variability among VG applications. Generally: Only basic support for OGC Web Services Lack of support for subsurface and submarine data –Picture on right shows a workaround No data analysis functions –But can link with Web Services and websites to do this Most do not support GIS file formats (shapefiles, GeoTIFFs) –Need to do a conversion 3-D Gulf Stream shown above ground in Google Earth