HMAS - Task 4 OpenSearch Extensions P. Gonçalves, Terradue Srl. June 11, 2014 http://creativecommons.org/licenses/by/2.0/ HMA-S Final Presentation

Overview OGC 10-032r8 OGC OpenSearch GeoSpatial and Temporal Extensions OGC 13-026r4 OGC OpenSearch Extension for Earth Observation OGC 10-068 OGC OpenSearch Extension for Correlated Search HMA-S Final Presentation

OpenSearch Search engines have a description document used by client applications and allows syndication of search results Provides metadata about the contents along with a set of URL Templates which illustrate the parameters accepted by the service and the variety of output formats in which results can be obtained HMA-S Final Presentation

OpenSearch Description Document URL templates <Url type="text/html" template="http://example.com/?q={searchTerms?}&pw={startPage?}"/> http://www.google.com/?q=question http://www.google.com/?q={searchTerms} Simple metadata of the search engine Title, developer, contact, icons, rights, language and encoding Query element examples, correction, related, subset, superset HMA-S Final Presentation

OpenSearch Response Query element describes the query used <os:Query os:role="request" time:start="2012-08-19" time:end="2012-10-19" geo:box="-43,45,-32,50"/> Paging information <os:totalResults>37</os:totalResults> <os:startIndex>0</os:startIndex> <os:itemsPerPage>20</os:itemsPerPage> Autodiscovery in RSS/Atom/HTML/… <link rel="search" type="application/opensearchdescription+xml" href="http://example.com/search.xml" title="Content search" /> Registered mime type application/opensearchdescription+xml IANA rel=‘search’ widely used HMA-S Final Presentation

OpenSearch Response OpenSearch does not mandates a specific format, it enriches feeds in html, xml or json formats Atom is widely compatible because is foreign element friendly and is ideal to wrap and link resources Atom entries should not be considered all purposes envelopes (entries link to other resources representations) Atom is the middle ground that let mainstream application enter OGC world while linking more complex resources HMA-S Final Presentation

GeoSpatial Extension (OGC 10-032) Specify a series of parameters that can be used to spatially constrain search results Provision is made to filter results by: A bounding box A geometry using Well Known Text Within a certain radius from a given latitude-longitude point Having a certain containment relation (within, intersects, disjoint) with a geographic constraint Matching a geographic name All geographic information in EPSG 4326 (WGS84) HMA-S Final Presentation

GeoSpatial Extension (OGC 10-032) geo:box -> Geographic bounding box The box is defined by "west, south, east, north" coordinates of longitude, latitude, in a EPSG:4326 decimal degrees. geo:geometry -> Geographic area (geometry): The polygon is defined using the Well Known Text standard for geographic shapes, using EPSG:4326 geo:lat, geo:lon, geo:radius -> The latitude, longitude of a given point with a given radius geo:relation -> Spatial relation to result set, Character String; One of “intersects”, “contains”, “disjoint” (default is “intersects”) geo:name -> A string describing the location to perform the search The namespace of the OpenSearch Geo Extension is: http://a9.com/-/opensearch/extensions/geo/1.0/ HMA-S Final Presentation

Time Extension Elements (OGC 10-032) Defines the temporal range request time:start Start of the temporal interval to search (RFC-3339) time:end End of the temporal interval to search (RFC-3339) time:relation Defines the temporal relation to the result set: intersects (default), contains, during, disjoint RFC-3339 Example: 2008-01-01T23:43:23.00Z Namespace of the OpenSearch Time Extension is: http://a9.com/-/opensearch/extensions/time/1.0/ HMA-S Final Presentation

Geo and Time Extensions (OGC 10-032) A search service with <Url type="text/html” template=”http://example.com/xml/?q={searchTerms?}&start={time:start?}&stop={time:end?}&bbox={geo:box?}"/> The geospatial extension allow to formulate geospatial requests e.g. point-plus-radius, a bounding box, or a polygon bbox={geo:box?} Together with the Time extension, OpenSearch can specify time start, finish, and slices for searching data. start={time:start?}&stop={time:end?} http://example.com/xml/?bbox=6.73,36.73,18.45,47.07&start=2011-01-10&stop=2011-01-12T23:59:5 HMA-S Final Presentation

Atom Extensibility RFC4827 defines the Atom XML vocabulary Atom allow foreigns markup by design If unknown to the Atom Processors MUST NOT stop processing or signal an error unknown foreign markup is encountered as a child of atom:entry, atom:feed, or a Person construct, Atom Processors MAY bypass the markup and any textual content and MUST NOT change their behavior as a result of the markup's presence Evolvability: this makes current clients compatible with future extensions HMA-S Final Presentation

ATOM Response Mappings Usage of ATOM elements GEORSS for spatial attributes dc:date for time ranges Feed and entry mappings Resource linkage with the atom:link element atom:link/[@rel=‘search’] for search engines (2 step search) atom:link/[@rel=‘prev’ or @rel=‘next’] for paging atom:link/[@rel=‘enclosure’] for data access services atom:link/[@rel=‘up’] to link a hierarchy of resources atom:link/[@rel=‘icon’] to link a quicklook images HMA-S Final Presentation

ATOM Feed and atom:link Direct Link to the data URL using the relation “enclosure” HMA-S Final Presentation

Integration using Atom feeds HMA-S Final Presentation

Status of OGC 10-032 "OGC OpenSearch Geo and Time Extensions” adopted as an OGC standard http://www.opengeospatial.org/pressroom/pressreleases/2003 http://www.opengeospatial.org/standards/opensearchgeo http://schemas.opengis.net/opensearchgeo/1.0/ Release of ETS on OGC SVN https://svn.opengeospatial.org/ogc-projects/cite/ets/testng/ets-osxgeotime/trunk/pom.xml Repository on GitHub created https://github.com/opengeospatial/ets-osxgeotime10 HMA-S Final Presentation

Testing Activities ETS were used to evaluate GEO Virtual Archive 4 GPOD FedEO CWIC NASA ECHO Tickets and email contact to responsible entities alerting to the issues and proposing solutions HMA-S Final Presentation

Earth Observation Extension (OGC 13-026) Defines query parameters that allow the filtering of search results based on Earth Observation products properties Platform or satellite (e.g. SPOT, ENVISAT) The sensor used to acquire the data The processing center, date and software used for the processing Satellite orbit information: number, track, frame and direction … Not redefining the spatial and temporal extension Uses the geo:* elements to perform spatial querys … and time:* for temporal search mapped to the acquisition time HMA-S Final Presentation

Earth Observation Extension Defines an extensive list of OpenSearch parameters that can be supported by a search engine. It is not expected or advisable to support all the elements defined in this specification. Should be used to constrain search results and largely dependent on the search engine contents and capabilities. <Url type="application/atom+xml" template="http://example.com/myatom/?q= {searchTerms}&pw={startPage?}&platform={eo:platform?}"/> HMA-S Final Presentation

Earth Observation Extension Parameters are divided for specific use cases (informative) Collection and dataset series List of Products Acquisition Parameters Includes mappings to (informative) Product Search on EO O&M Metadata profile of Observations & Measurements (OGC 10-157r4) ebRIM Application Profile EO Products (OGC 06-131r6) Collection Search to ISO19115(-2)/ISO19139(-2) Collection Search to the CSW ebRIM I15 Extension Package (OGC 13-084r2) HMA-S Final Presentation

Search context propagation to external end points Defines how to propagate search queries in a collection or federation architecture Use of atom:link with the @rel “search” [RFC 5988] Allows the definition of recursive searches where the search context (e.g. queriables) are propagated to an internal or external services. <atom:link rel="search" type="application/opensearchdescription+xml" href="http://search.acme.com/osd.xml" title="Search this resource" /> With this link clients are recommended to propagate the query parameters to the search engine described by the OpenSearch description document located on the href attribute value using the original query parameters. HMA-S Final Presentation

Linking entry and metadata Source of entry information Use of atom:link the @rel “via” [RFC 5988] <atom:link rel=”via" type="application/rdf+xml" title=”Source of entry information" href="http://esa.int/eo/asarimage1234/rdf" length="1245"/> Entry representation on other metadata formats Use of atom:link the @rel “alternate” [RFC 5988]. Metadata format described in the type attribute <atom:link rel="alternate" type="application/gml+xml" title="EO O&M Metadata" href="http://site.com/eo/asarimage1234.xml" length="1245"/> <atom:link rel="alternate" type="application/vnd.iso.19139+xml" length="845" title="ISO 19139 Metadata" href="http://site.com/eo/asarimage1234ISO.xml"/> HMA-S Final Presentation

Linking thumbnail, quicklook or masks Reference to a resource representing a quicklook, browse image or cloud mask using the Media RSS Specification Allows categories differentiate browse types (THUMBNAIL, QUICKLOOK, ALBUM) from masks (SNOW, CLOUD, QUALITY) <feed xmlns="http://www.w3.org/2005/Atom xmlns:media="http://search.yahoo.com/mrss/" …> <entry> … <media:group> <media:content url="http://mda.com/browse/RADARSAT-2/2012/08/01/PASS_RS2_210671_3_32488_1.jpg" type="image/jpeg" medium="image"> <media:category scheme="http://www.opengis.net/spec/EOMPOM/1.0">QUICKLOOK</media:category> </media:content> … </media:group> </entry> HMA-S Final Presentation

Error Handling The Atom and the OpenSearch specifications do not directly specify how to deal with server and request errors OGC 13-026 recommends the use the HTTP error codes for client (4xx) and server (5xx) errors. 400 Bad Request: invalid syntax (i.e. bad geometry) 413 Request Entity Too Large: too many returnable hits 500 Internal Server Error: Generic server error. 501 Not Implemented: Requested an unimplemented feature. 503 Service Unavailable: Service temporarily not available 504 Gateway Timeout: When the search engine is a broker or aggregator to other services that fail to produce a answer within a giving time frame. HMA-S Final Presentation

Status of OGC 13-026 Draft widely circulated between ESA and other HMA and CEOS agencies Great number of inputs and suggestions U. Voges is adding new mappings for collections search Collection Search to ISO19115(-2)/ISO19139(-2) Collection Search to the CSW ebRIM I15 Extension Package (OGC 13-084r2) New collection queriables proposed under evaluation Internal revision of release 4 under way HMA-S Final Presentation

Extension for Correlated Search Complementary to “OpenSearch Geo Spatial and Temporal Extensions” (OGC 10-032) “OpenSearch Extension for Earth Observation” (OGC 13-026) The purpose of this extension is to define correlated searches methods. From an initial search feed, the extension allows the definition of linkages to other search feeds in terms of: Spatial footprint Temporal range of acquisition Server defined correlation functions HMA-S Final Presentation

Correlation Search Extension (OGC 13-068) Correlation search between different search engines Integrates of several search feeds and creates relations between the different entries Allows definition of linkages to other search feeds originating from potentially different search engines Creates relations between the different entries. Spatial footprint Temporal range of acquisition Server Functions (e.g. Interferometry normal baseline and burst synchronization) Links the master entry to a specific correlation on the requested feeds (and their corresponding queries) HMA-S Final Presentation

Correlation Search Extension (OGC 13-068) Need to define slave feed that will be correlated with the search engine feed (i.e. master feed) The URL identifying the correlation feed is opaque to the Search Engine it may carry specific search criteria or parameters that the search engine must not alter The slave feed URL must have the same encoding of the master feed. Each entry of the search feed will be correlated with the results originating from the Correlated Search feeds. HMA-S Final Presentation

Correlation Search Extension (OGC 13-068) cor:with encoded URLs representing the correlated search feed in same encoding as the search engine requested cor:time number of seconds separating master and correlated items cor:spatial minimum percentage of spatial coverage necessary between the master and correlated items. cor:minimum A number representing the minimum numbers of correlated entries cor:function A string identifying the local correlation function that will be applied to select the correlated items. cor:parameters A string identifying the local parameters of the correlation function http://a9.com/-/opensearch/extensions/correlation/1.0/ HMA-S Final Presentation

Spatial Correlated Search Search feed with the master entries: http://meris.net?bbox=10,10,20,20 To correlate each entry with entries from another feed from contain the slave entries: http://modis.net?bbox=12,12,18,18 For each master select slave overlapping more than 50% http://meris.net/?bbox=10,10,20,20& spatialcover=50& correlatedto=http:%2F%2Fmodis.net%3Fbbox%3D12,12,18,18 For example, with a 50% spatial cover criterion, at least 50% of the master entry must be covered by the slave entry in order for the slave entry to be selected. HMA-S Final Presentation

Example: Correlate product feeds #1 - http://web.site/search/ER01_SAR_IM__0P/atom/ ?start=1992-11-03T02:14:55Z& stop=1992-11-03T02:15:00Z #2 - http://another.site/search/ER01_SAR_IM__0P/atom/ ?start=1992-01-01T00:00:00Z& stop=1993-12-31T23:59:59.99Z& bbox=120,40.89,129.74,42.83 Task perform a correlated search to select only the entries of #2 that cover at least 95% of #1. HMA-S Final Presentation

Correlated Search http://web.site/search/ER01_SAR_IM__0P/atom/? start=1992-11-03T02:14:55Z& stop=1992-11-03T02:15:00Z& spaceCover=95& with=http%3A%2F%2Fanother.site%2Fsearch%2FER01_SAR_IM__0P %2Fatom%2F%3Fstart%3D1992-01-01T00%3A00%3A00Z%26stop%3D1993-12-31T23%3A59%3A59.99Z%26bbox%3D120%2C40.89%2C129.74%2C42.83%26%0A HMA-S Final Presentation

Correlate feeds (geometry and names) Each product geometry mapped to a names search feed HMA-S Final Presentation

Correlate feeds (geometry and names) Search Engine Friendly Direct Links to data holdings Automatic tagging of granules HMA-S Final Presentation

Interferometric Search Special case of a correlation search dealing with specific Radar imagery information (e.g. Sentinel-1, ERS, Envisat). It shares the same process and outputs but have quite different purposes and contexts. Interferometric search is only performed when products to be combined are acquired by the same mission in compatible phases (i.e. in phases having the same orbit scheme) HMA-S Final Presentation

Interferometric Search Special correlation function with corresponding parameters (baseline and burst sync). For each, it checks if it has the same track number, sensor mode and swath identifier (as defined OGC 13-026) of the master product. For every slave candidate product, the center time from the overlap area is computed and obtains the baseline and time from topsar values stored in the auxiliary files. If then those interferometric correlation criteria are also met then the slave entry is accepted From http://insar.sk HMA-S Final Presentation

Interferometric Search <OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/" xmlns:geo="http://a9.com/-/opensearch/extensions/geo/1.0/" xmlns:param="http://a9.com/-/spec/opensearch/extensions/parameters/1.0/" xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/" xmlns:cor="http://a9.com/-/opensearch/extensions/correlation/1.0/"> ... <Url type="application/atom+xml" template="http://example.com/myatom/? pw={startPage?}&bbox={geo:box?}&correlatedTo={cor:with?}&spatialCover={cor:spatial?}&timeCover={cor:time}&function={cor:function?}&param={cor:parameter}"/> <param:Parameter name="function" value="{cor:function}" minimum="0" title=”Correlation Functions”> <param:Option value="interferometric" label=”Interferometric Search”/> </param:Parameter> <param:Parameter name="param" value="{cor:parameter}" minimum="0" title="Correlation Parameters" pattern="burst:[\+\-]?[\d]*,baseline:[\+\-]?[\d]*" /> HMA-S Final Presentation

HMAS - Task 4 OpenSearch Extensions Thank YOU! RSS