1 CLASS – Simple NOAA Archive Access Portal SNAAP Eric Kihn and Rob Prentice NOAA/NGDC ESIP Meeting January 7 th, 2009 Simple NOAA Archive Access Portal

2 CLASS – Simple NOAA Archive Access Portal SNAAP Goals Integrate diverse data archive systems via a standards based, user focused, web services interface Create a set of tools that shows the benefit of such an API for users Discover new technology and evaluate cutting edge tools for handling integration and presentation Document integration of multiple data types and sources through an single API (time-series, grid, swath, etc..) Keep it simple. Anything tough is a client problem

3 CLASS – Simple NOAA Archive Access Portal The Data Environment Now CLASS ADS (e.g. AVHRR) Web Interface Subscriptions HumanMachine Data Providers VisualizationInventory Order NMMR/Metadata NGDC/SPIDR (e.g. DMSP, Space Physics) Web Interface Metadata SPIDR WS SNAAP Web Interface Metadata Data Access

4 CLASS – Simple NOAA Archive Access Portal What is the origin of SNAAP? The project evolved from the goal to put an API on CLASS. It was quickly realized that the boundary is then arbitrary and it makes sense to extend the API to other data systems CLASS (and other systems) already have a functional API for intra-system communication but the duplicity of systems causes real headaches The system is currently limited to archive (non-real time) data systems following its archive access origins SNAAP is part of the GEO/GEOSS effort. Its architecture and capabilities do for NOAA what is being done in GEOSS (archive)

5 CLASS – Simple NOAA Archive Access Portal SNAAP Functionality Search for appropriate data sets Discover data by time, type, location, abstract, etc. Support for multiple metadata schema (FGDC, IEEE, SPASE) Standards based (xQuery and REST) This is in already in CLASS (4.3.3) – awaiting policy Order Data Needs a standard way to describe ordering options This is done via Ordering Extensions (OE) which are linked to and extend standard metadata Needs some framework for service integration (i.e. transforms) this is provided via (Data Access and Integration) DAI Actions Provider Integration Layer We have chosen OGSA-DAI for the demo (This could change in production) We offer both a SOAP and REST interface (slightly different functionality) Its important that the backend retrieval not effect the consumer.

6 CLASS – Simple NOAA Archive Access Portal Advantages This approach mirrors that of the highly successful OGC specification for map services (what, make, retrieve) It minimizes entry burden for new providers Its simplicity makes it scalable It doesn’t preclude or interfere with on-going activity, just opens up new integration possibilities.

7 CLASS – Simple NOAA Archive Access Portal Metadata: A Complete Data Environment is More than Just the Bits

8 CLASS – Simple NOAA Archive Access Portal Snapshot of the SNAAP API architecture

9 CLASS – Simple NOAA Archive Access Portal Why OGSA-DAI? Standard tool in the Grid community Includes a “process” for extraction Built in support for asynchronous transactions Includes support for domain login Looked at alternatives like OpenDap, WCS, Custom applications –documentation of our analysis is available Problem: Its very complex and not well accepted in NOAA

10 CLASS – Simple NOAA Archive Access Portal Data Order Flow

11 CLASS – Simple NOAA Archive Access Portal CLASS Ordering Extensions (OE) schema upper level

12 CLASS – Simple NOAA Archive Access Portal CLASS OE schema: parameters and time

13 CLASS – Simple NOAA Archive Access Portal CLASS OE schema: map and ROI

14 CLASS – Simple NOAA Archive Access Portal CLASS OE schema: station map

15 CLASS – Simple NOAA Archive Access Portal Key Goal: Minimize the Burden on Providers Create a standard metadata record for the data set. (They should be doing this anyway). Create an Ordering extension document. If they are offering web distribution this is trivial. Write an adapter to the middle layer, unless they have a supported service (like JDBC, FTP, etc..) in which case adapt an existing one. To hook to SNAAP a provider would : The provider then: Continues to serve data and metadata as before plus SNAAP access.

16 CLASS – Simple NOAA Archive Access Portal Data Sets and Types Data Types Time-series – Sunspot number Grids – NCEP Reanalysis Stations – Ionospheric Soundings Swath - AVHRR Profiles – Ocean Profile Maps – Nighttime lights More?

17 CLASS – Simple NOAA Archive Access Portal Target Audience There is a clear understanding that many communities are already well served by existing data systems The SNAAP API targets developers who want to add new functionality to integrated NOAA archives When operational it will provide fully redundant support for data discovery and retrieval It is hoped that by having the data accessible in a standard open way more innovative client tools will become available.

18 CLASS – Simple NOAA Archive Access Portal Visualization screens – 1 time series and map server

19 CLASS – Simple NOAA Archive Access Portal Visualization screens – 2 NASA World Wind plugins

20 CLASS – Simple NOAA Archive Access Portal MS Excel and NASA World Wind interoperability for trajectory data Excel with track points Excel with data along tracks Drag & Drop controls NASA World Wind

21 CLASS – Simple NOAA Archive Access Portal Future Plans Plan transition to operations (CLASS Team) This will identify a CLASS build and date for SNAAP inclusion This would require some review process (including outside) and a versioning of OE Seek outside review of the OE documentation. Plan to include non-granule based asynchronus (i.e. models, transforms) data extractions Enhance the connection with GEO-IDE Upgrade to DAI DONE Expand proto-type to cover more NOAA and non-NOAA data types and services and document lessons learned Release the work to the public: Metadata tool is available as VxOware Work on single user sign-on.

22 CLASS – Simple NOAA Archive Access Portal BACK UP SLIDES

23 CLASS – Simple NOAA Archive Access Portal Asynchronous Ordering CLASS API Demonstration

24 CLASS – Simple NOAA Archive Access Portal Orbit Inventory List

25 CLASS – Simple NOAA Archive Access Portal User Basket (Order & Check- Status)

26 CLASS – Simple NOAA Archive Access Portal Order System Entities Granule List now using ECHO XML in the return New entity “Data Basket” – handle to collection of order Demo currently orders all data centrally will need to be adapted

27 CLASS – Simple NOAA Archive Access Portal Order Sequence Diagram Data basket ID is the API handle OGSA-DAI activities handle the extraction and packaging Status check is currently by basket only (XML return) Finished granules return a URL handle to the entity