ECHO Technical Interchange Meeting 2013 Timothy Goff 1 Raytheon EED Program | ECHO Technical Interchange 2013

2 What are ECHO services?

Most data is readily available online Desirable to expose services on data. Need for services to expose specific processing operations (e.g. reformat, re-project, subset). Services should be available as close to on-demand as possible. As opposed to queued orders Service framework should be able to describe services available on each dataset. 3 Raytheon EED Program | ECHO Technical Interchange 2013

Utility written by a service provider to process science data on-demand Subsetting Reformatting Reprojection May operate against a granule or collection ECHO service interface provides unified portal to disparate services Allows providers to choose to expose services most appropriate to their data 4 Raytheon EED Program | ECHO Technical Interchange 2013

5 Service Processed ‘on demand’ Real time status updates ‘progress bar’ Access to resulting data via link in reverb Submitted via service’s API Order May be queued indefinitely Status updates via Access to data via sent by provider Submitted via ECHO order API Raytheon EED Program | ECHO Technical Interchange 2013

6 A case study in ECHO services

Data stored primarily on Tape Archives. Delivered on physical media. Order fulfillment required significant processing and resource management. Orders were slow. Image URL: 7 Raytheon EED Program | ECHO Technical Interchange 2013

Most data on disk and internet accessible. Can be downloaded without need to place an order. Order fulfillment can be as quick as bandwidth allows. Need for mechanism for processing data on the fly Image URL: everything.jpg 8 Raytheon EED Program | ECHO Technical Interchange 2013

ESDIS developed API which federates service agents for common types of subsetting. “REST-Style” API – All actions are via HTTP GET/POST requests. Does not support all our needs (e.g. hierarchical data structure and reprojection). Simple, open API is easily extended to support these capabilities. SSW already integrated with some EOSDIS data services. Compatibility of APIs will foster interoperability. 9 Raytheon EED Program | ECHO Technical Interchange 2013

Custom API developed as extension to SSW. After researching many other options. Defines general API for common processing types. Per dataset API conveyed by ESI XML file or ECHO Form. Extensible to include new processing tools. 10 Raytheon EED Program | ECHO Technical Interchange 2013

Processing requests can be submitted as HTTP GETs or POSTs. Two representations of the same ESI request are below. POST GET 11 Raytheon EED Program | ECHO Technical Interchange 2013

Perform the actual work of an ESI request. Processing tools can be integrated and exposed by ESI. Develop custom ‘Tool Adapter’ to translate ESI API to tool’s native syntax. All integrated tools exposed via single external API. Hides internal complexity from end user. ClientESI Tool Adapter Processing Tool A Tool Adapter Processing Tool B Tool Adapter Processing Tool C 12 Raytheon EED Program | ECHO Technical Interchange 2013

Use ECHO services to provide service as an option on search results in Reverb Reverb integration of ESI services provides visibility to wide audience of Earth Science data users Order options UI specified via ECHO Forms Provides dynamic dataset specific options 13 Raytheon EED Program | ECHO Technical Interchange 2013

API developed by ECHO which facilitates the development of user input forms. Based on W3C XForms 1.0 standard. Separates data structure (model) from presentation (user interface). Includes ability to conditionally display UI elements. Options presented dependant on options already selected. XML format facilitates automatic/programmatic generation of forms. 14 Raytheon EED Program | ECHO Technical Interchange 2013

Generated UI: XML Produced by user selections: Input validation (based on rules embedded in form definition): 15 Raytheon EED Program | ECHO Technical Interchange 2013

Determine ECHO form output model Provide request API ECHO develops interface code Converts model output XML to service API Define service implementation Defines service end point Mapped to a service interface and datasets Define order options (ECHO Forms) Link options to service implementation and dataset 16 Raytheon EED Program | ECHO Technical Interchange 2013

17 Raytheon EED Program | ECHO Technical Interchange 2013

18 Raytheon EED Program | ECHO Technical Interchange 2013

19 Raytheon EED Program | ECHO Technical Interchange 2013

20 Raytheon EED Program | ECHO Technical Interchange 2013 Entire configuration process can be automated using ECHO REST and SOAP APIs Option forms can be dynamically uploaded based on provider configuration

21 Raytheon EED Program | ECHO Technical Interchange 2013

22 Raytheon EED Program | ECHO Technical Interchange 2013

Dataset Options Specified as ECHO Form Service Option in Reverb Cart ECHO Form rendered to solicit processing options Service Invoked 23 Raytheon EED Program | ECHO Technical Interchange 2013

Tight coupling with a generic API along with per- dataset configuration facilitates integration with clients. Allows API to be dynamic and extensible without breaking clients. ECHO Forms is a great vehicle for conveying API capabilities and an associated UI. ESI development provides template for integration of other services with Reverb. 24 Raytheon EED Program | ECHO Technical Interchange 2013

Raytheon EED Program | ECHO Technical Interchange 2013