NOAA Earth Information Service (NEIS) Overview GSD Technical Seminar February 28 th, 2012 ESRL Global Systems Division

NEIS Agenda Introduction Evaluation and Findings Concept Activities and Next Steps Demonstration ESRL Global Systems Division

NEIS Background GSD Director Discretionary Funds (DDF) Proposal NEIS Requirement Fast access to any data, any time, right now, on any platform. NEIS Group Objectives Build upon NEIS work in the past Assess GSD capabilities pertinent to the concept of NEIS. Evaluate existing, developmental technology, and solutions relevant to concept. Evaluate envisioned needs of primary users. Provide recommendation of how to best use GSD resources to contribute to a National capability. ESRL Global Systems Division

NEIS Team Research Team Jebb Stewart - Lead Chris MacDermaidWoody Roberts Darien DavisJeff Smith Mike RombergPaul Schultz Development Team Julien LyngeJeff Smith Chris MacDermaidJebb Stewart Randy Pierce ESRL Global Systems Division

NEIS History January 18, 2007, Roberts, et al. A OneNOAA Concept Prototype for Data Viewing, 23rd IIPS. March 10, 2009, Hansen, et al. Earth Information Services, GSD Technical Review on GFE and Other Activities. ESRL Global Systems Division

NEIS Evaluation Interviewed/Discussed concepts with variety of groups across NOAA Line Offices. More than 15 individuals. Questions Asked: How are people using your data? How are you using NOAA data yourself? What tools/systems/services/standards are you using? What related recommendations/lessons learned exist? What obstacles or roadblocks have you encountered with using/disseminating NOAA data? Listened for items to help facilitate projects or help NOAA data become more interoperable. ESRL Global Systems Division

Interviews

NEIS System Evaluation ESRL Global Systems Division xkcd.com Part of the evaluation looked at existing systems and standards including: THREDDSLASERDDAP Google EarthNASA World WindGeoPortal MERMAidREVERBiRODS Grads Data ServerMore…. Lot of Great Technology, which can be leveraged, no ‘one’ solution.

NEIS Committee and Group Involvement ESRL Global Systems Division GSD is aware of many ongoing activities We have active membership in a variety of committees and groups including: OGC Standards Committees EDMC DMIT UAF NextGen/NNEW program FIM/WRF Modeling GIS Committee GSD Central Facility has vast experience in working with data and standards

NEIS Findings Existing metadata systems contain stale or obsolete information and are not useful  Search for common term like wind returns hundreds of results.  Expertise required to understand what metadata you are looking for. Many “Stovepipes” but no ability to combine/integrate data from different disciplines (Ecological, Meteorological, Oceanographic, Fisheries, etc...) Data sets are large and getting bigger. Future data sets will be difficult if not impossible to move around network to consumer applications. Cross data source data processing service does not exist. Some data processing services exist but usually only for data available from that particular service. Feedback (broken links, data usage, etc…) to data providers and consumers is non-existent. Many more… ESRL Global Systems Division

Biological Chemical Physical 11 Biological, Chemical, and Physical data is all interrelated NEIS allows data integration in 4D time and space, providing a scientific tool for everything NOAA does professionally and the world NOAA affects. Carbon Tracker FIM Data Integration ESRL Global Systems Division

12 ESRL Global Systems Division NOAA Vision Ability to discover, integrate, understand all NOAA data (past, present, and future) from homepage. TerraViz

13 ESRL Global Systems Division NOAA Vision Most of NOAA's environmental data will be accessible via web services and will be described by good metadata A central and intelligent search engine, that has harvested the metadata of NOAA's datasets, will exist to find data NEIS server side analytics/processing will enable subsetting, aggregating, diff'ing, and other operations to be performed on data Data will be efficiently streamed to a variety of platforms and clients, including mobile devices TerraViz will be a general purpose client running on multiple platforms capable of searching for NEIS data and visualizing it in novel ways Framework is extensible, other groups can add knowledge and services.

NEIS – Concept Components

Build new Graphical User Interface system integrating and leveraging new and emerging technologies to meet NEIS goal ‘any data, any location, any platform, now’ Perform processing within cloud environment and with high speed connectivity to data sources, taking advantage of large processing power within clouds. Send graphics and server side processed/rendered/streamed data to GUI, improving bandwidth utilization. Take advantage of fast networking to make remote requests and processing appear like local application. Similar to how the new Amazon Silk Browser works. Looking to the Future ESRL Global Systems Division

Improve existing Metadata services making information useful. Improve searching by understanding what data means and providing improved filtering capability similar to how airline or hotel type search engines work. Incentivize/encourage people to use proper Metadata. Create a Metadata Dashboard to: o Gather information from users on relevance, ratings, usage patterns, search key words used (ie crowd sourcing).  This information can be used to determine what users are searching for, how they are finding data, what data they are not finding, etc... o Provide feedback on broken links, service uptime o Provide feedback on adherence to standards (Services, Metadata). o Provide feedback to users of similar or related data sets. Looking to the Future – Part 2 ESRL Global Systems Division

Impacts NOAA data ready for action. Services model facilitates agile response to events. Services can be combined or reused quickly Any data available through NEIS system can be operated on or combined with other data. Integrated standardized formats and access. New and Existing systems have access to wide variety of NOAA data. Any new data added, easy incorporated with minimal to no changes required. ESRL Global Systems Division

NEIS Use Case – Deepwater Horizon There was no way to pull disparate data sets into one display in order to track the impacts of the oil spill o Ocean currents o Ecosystems o Weather o Location of booms, ships, and other response equipment Many GIS developer labor hours spent working on converting and formatting data to make it available.

Are We Prepared? What would happen if another hurricane or other disaster occurred? With a framework such as this: Improved searching capability would allow those in need to quickly and easily find pertinent data to the situation. All data available through framework would be readily accessible to be integrated across 4D (time and space). New data that is properly documented could be added to services and become immediately available to the framework of services. Spin up time for new data reduced dramatically. Because framework exists as a collection of services, any changes or modification can be completed quickly without taking entire system down. ESRL Global Systems Division

Demonstration ESRL Global Systems Division

Prototype Demonstration Background 24 ESRL Global Systems Division What is TerraViz? A system developed from the ground up by GSD based on Unity3D. There are no licensing restrictions in distributing the TerraViz executable. Unity3D is a well supported commercial game and data visualization engine with 750,000 active users Unity/C# is designed for 3D development, making it easy to create multiple views, create 3D meshes representing environmental objects, etc. Develop once, then will run under Windows, Mac, web browsers, iPhones, iPads, Android devices, and even game systems (Wii, Xbox 360, etc.) We can modify the user interface and add any feature we need. TerraViz Handles millions of polygons and performs quickly TerraViz is one possible visualization tool built to develop, test, and showcase NEIS framework.

25 Google Earth, works great for certain applications and static products, however: Handles ~10,000 polygons before slowing to a crawl. Closed system (Google Earth source code not available) so we can’t add needed features. Google Earth plug-in works on Windows and Mac 32-bit browsers only (won’t run on iPhones, iPads, or Android devices.) Developing GE plug-in user interfaces in browser-dependent Javascript is tedious. Very Limited integrated data discovery or browsing, most uses require a separate application. Limited animation/data interaction support. Limited support for non-static data (ie Time Series) Why not Google Earth