1. HMAS - Task 4 OpenSearch Extensions
P. Gonçalves, Terradue Srl.
June 11, 2014
HMA-S Final Presentation

2. Overview
OGC r8 OGC OpenSearch GeoSpatial and Temporal Extensions
OGC r4 OGC OpenSearch Extension for Earth Observation
OGC OGC OpenSearch Extension for Correlated Search
HMA-S Final Presentation

3. OpenSearch
Search engines have a description document used by client applications and allows syndication of search results
Provides metadata about the contents along with a set of URL Templates which illustrate the parameters accepted by the service and the variety of output formats in which results can be obtained
HMA-S Final Presentation

4. OpenSearch Description Document
URL templates
<Url type="text/html" template="
Simple metadata of the search engine
Title, developer, contact, icons, rights, language and encoding
Query element
examples, correction, related, subset, superset
HMA-S Final Presentation

5. OpenSearch Response Query element describes the query used
<os:Query os:role="request" time:start=" " time:end=" " geo:box="-43,45,-32,50"/>
Paging information
<os:totalResults>37</os:totalResults>
<os:startIndex>0</os:startIndex>
<os:itemsPerPage>20</os:itemsPerPage>
Autodiscovery in RSS/Atom/HTML/…
<link rel="search" type="application/opensearchdescription+xml" href=" title="Content search" />
Registered mime type
application/opensearchdescription+xml
IANA rel=‘search’ widely used
HMA-S Final Presentation

6. OpenSearch Response
OpenSearch does not mandates a specific format, it enriches feeds in html, xml or json formats
Atom is widely compatible because is foreign element friendly and is ideal to wrap and link resources
Atom entries should not be considered all purposes envelopes (entries link to other resources representations)
Atom is the middle ground that let mainstream application enter OGC world while linking more complex resources
HMA-S Final Presentation

7. GeoSpatial Extension (OGC 10-032)
Specify a series of parameters that can be used to spatially constrain search results
Provision is made to filter results by:
A bounding box
A geometry using Well Known Text
Within a certain radius from a given latitude-longitude point
Having a certain containment relation (within, intersects, disjoint) with a geographic constraint
Matching a geographic name
All geographic information in EPSG 4326 (WGS84)
HMA-S Final Presentation

8. GeoSpatial Extension (OGC 10-032)
geo:box -> Geographic bounding box The box is defined by "west, south, east, north" coordinates of longitude, latitude, in a EPSG:4326 decimal degrees.
geo:geometry -> Geographic area (geometry): The polygon is defined using the Well Known Text standard for geographic shapes, using EPSG:4326
geo:lat, geo:lon, geo:radius -> The latitude, longitude of a given point with a given radius
geo:relation -> Spatial relation to result set, Character String; One of “intersects”, “contains”, “disjoint” (default is “intersects”)
geo:name -> A string describing the location to perform the search
The namespace of the OpenSearch Geo Extension is:
HMA-S Final Presentation

9. Time Extension Elements (OGC 10-032)
Defines the temporal range request
time:start Start of the temporal interval to search (RFC-3339)
time:end End of the temporal interval to search (RFC-3339)
time:relation Defines the temporal relation to the result set: intersects (default), contains, during, disjoint
RFC-3339 Example: T23:43:23.00Z
Namespace of the OpenSearch Time Extension is:
HMA-S Final Presentation

10. Geo and Time Extensions (OGC 10-032)
A search service with
<Url type="text/html” template=”
The geospatial extension allow to formulate geospatial requests e.g. point-plus-radius, a bounding box, or a polygon bbox={geo:box?}
Together with the Time extension, OpenSearch can specify time start, finish, and slices for searching data start={time:start?}&stop={time:end?}
HMA-S Final Presentation

11. Atom Extensibility RFC4827 defines the Atom XML vocabulary
Atom allow foreigns markup by design
If unknown to the Atom Processors MUST NOT stop processing or signal an error
unknown foreign markup is encountered as a child of atom:entry, atom:feed, or a Person construct, Atom Processors MAY bypass the markup and any textual content and MUST NOT change their behavior as a result of the markup's presence
Evolvability: this makes current clients compatible with future extensions
HMA-S Final Presentation

12. ATOM Response Mappings
Usage of ATOM elements
GEORSS for spatial attributes
dc:date for time ranges
Feed and entry mappings
Resource linkage with the atom:link element
for search engines (2 step search)
for paging
for data access services
to link a hierarchy of resources
to link a quicklook images
HMA-S Final Presentation

13. ATOM Feed and atom:link
Direct Link to the data URL using the relation “enclosure”
HMA-S Final Presentation

14. Integration using Atom feeds
HMA-S Final Presentation

15. Status of OGC
"OGC OpenSearch Geo and Time Extensions” adopted as an OGC standard
Release of ETS on OGC SVN
Repository on GitHub created
HMA-S Final Presentation

16. Testing Activities ETS were used to evaluate
GEO Virtual Archive 4
GPOD
FedEO
CWIC
NASA ECHO
Tickets and contact to responsible entities alerting to the issues and proposing solutions
HMA-S Final Presentation

17. Earth Observation Extension (OGC 13-026)
Defines query parameters that allow the filtering of search results based on Earth Observation products properties
Platform or satellite (e.g. SPOT, ENVISAT)
The sensor used to acquire the data
The processing center, date and software used for the processing
Satellite orbit information: number, track, frame and direction …
Not redefining the spatial and temporal extension
Uses the geo:* elements to perform spatial querys
… and time:* for temporal search mapped to the acquisition time
HMA-S Final Presentation

18. Earth Observation Extension
Defines an extensive list of OpenSearch parameters that can be supported by a search engine.
It is not expected or advisable to support all the elements defined in this specification.
Should be used to constrain search results and largely dependent on the search engine contents and capabilities.
<Url type="application/atom+xml" template=" {searchTerms}&pw={startPage?}&platform={eo:platform?}"/>
HMA-S Final Presentation

19. Earth Observation Extension
Parameters are divided for specific use cases (informative)
Collection and dataset series
List of Products
Acquisition Parameters
Includes mappings to (informative)
Product Search on EO O&M Metadata profile of Observations & Measurements (OGC r4)
ebRIM Application Profile EO Products (OGC r6)
Collection Search to ISO19115(-2)/ISO19139(-2)
Collection Search to the CSW ebRIM I15 Extension Package (OGC r2)
HMA-S Final Presentation

20. Search context propagation to external end points
Defines how to propagate search queries in a collection or federation architecture
Use of atom:link with “search” [RFC 5988]
Allows the definition of recursive searches where the search context (e.g. queriables) are propagated to an internal or external services.
<atom:link rel="search" type="application/opensearchdescription+xml" href=" title="Search this resource" />
With this link clients are recommended to propagate the query parameters to the search engine described by the OpenSearch description document located on the href attribute value using the original query parameters.
HMA-S Final Presentation

21. Linking entry and metadata
Source of entry information
Use of atom:link “via” [RFC 5988]
<atom:link rel=”via" type="application/rdf+xml" title=”Source of entry information" href=" length="1245"/>
Entry representation on other metadata formats
Use of atom:link “alternate” [RFC 5988].
Metadata format described in the type attribute
<atom:link rel="alternate" type="application/gml+xml" title="EO O&M Metadata" href=" length="1245"/>
<atom:link rel="alternate" type="application/vnd.iso xml" length="845" title="ISO Metadata" href="
HMA-S Final Presentation

22. Linking thumbnail, quicklook or masks
Reference to a resource representing a quicklook, browse image or cloud mask using the Media RSS Specification
Allows categories differentiate browse types (THUMBNAIL, QUICKLOOK, ALBUM) from masks (SNOW, CLOUD, QUALITY)
<feed xmlns="
xmlns:media=" …>
<entry> …
<media:group>
<media:content url=" type="image/jpeg" medium="image">
<media:category scheme="
</media:content> …
</media:group>
</entry>
HMA-S Final Presentation

23. Error Handling
The Atom and the OpenSearch specifications do not directly specify how to deal with server and request errors
OGC recommends the use the HTTP error codes for client (4xx) and server (5xx) errors.
400 Bad Request: invalid syntax (i.e. bad geometry)
413 Request Entity Too Large: too many returnable hits
500 Internal Server Error: Generic server error.
501 Not Implemented: Requested an unimplemented feature.
503 Service Unavailable: Service temporarily not available
504 Gateway Timeout: When the search engine is a broker or aggregator to other services that fail to produce a answer within a giving time frame.
HMA-S Final Presentation

24. Status of OGC
Draft widely circulated between ESA and other HMA and CEOS agencies
Great number of inputs and suggestions
U. Voges is adding new mappings for collections search
Collection Search to ISO19115(-2)/ISO19139(-2)
Collection Search to the CSW ebRIM I15 Extension Package (OGC r2)
New collection queriables proposed under evaluation
Internal revision of release 4 under way
HMA-S Final Presentation

25. Extension for Correlated Search
Complementary to
“OpenSearch Geo Spatial and Temporal Extensions” (OGC )
“OpenSearch Extension for Earth Observation” (OGC )
The purpose of this extension is to define correlated searches methods.
From an initial search feed, the extension allows the definition of linkages to other search feeds in terms of:
Spatial footprint
Temporal range of acquisition
Server defined correlation functions
HMA-S Final Presentation

26. Correlation Search Extension (OGC 13-068)
Correlation search between different search engines
Integrates of several search feeds and creates relations between the different entries
Allows definition of linkages to other search feeds originating from potentially different search engines
Creates relations between the different entries.
Spatial footprint
Temporal range of acquisition
Server Functions (e.g. Interferometry normal baseline and burst synchronization)
Links the master entry to a specific correlation on the requested feeds (and their corresponding queries)
HMA-S Final Presentation

27. Correlation Search Extension (OGC 13-068)
Need to define slave feed that will be correlated with the search engine feed (i.e. master feed)
The URL identifying the correlation feed is opaque to the Search Engine
it may carry specific search criteria or parameters that the search engine must not alter
The slave feed URL must have the same encoding of the master feed.
Each entry of the search feed will be correlated with the results originating from the Correlated Search feeds.
HMA-S Final Presentation

28. Correlation Search Extension (OGC 13-068)
cor:with encoded URLs representing the correlated search feed in same encoding as the search engine requested
cor:time number of seconds separating master and correlated items
cor:spatial minimum percentage of spatial coverage necessary between the master and correlated items.
cor:minimum A number representing the minimum numbers of correlated entries
cor:function A string identifying the local correlation function that will be applied to select the correlated items.
cor:parameters A string identifying the local parameters of the correlation function
HMA-S Final Presentation

29. Spatial Correlated Search
Search feed with the master entries:
To correlate each entry with entries from another feed from contain the slave entries:
For each master select slave overlapping more than 50%
spatialcover=50& correlatedto=
For example, with a 50% spatial cover criterion, at least 50% of the master entry must be covered by the slave entry in order for the slave entry to be selected.
HMA-S Final Presentation

30. Example: Correlate product feeds
# ?start= T02:14:55Z& stop= T02:15:00Z
# ?start= T00:00:00Z& stop= T23:59:59.99Z& bbox=120,40.89,129.74,42.83
Task
perform a correlated search to select only the entries of #2 that cover at least 95% of #1.
HMA-S Final Presentation

31. Correlated Search http://web.site/search/ER01_SAR_IM__0P/atom/?
start= T02:14:55Z&
stop= T02:15:00Z&
spaceCover=95&
with=http%3A%2F%2Fanother.site%2Fsearch%2FER01_SAR_IM__0P %2Fatom%2F%3Fstart%3D T00%3A00%3A00Z%26stop%3D T23%3A59%3A59.99Z%26bbox%3D120%2C40.89%2C129.74%2C42.83%26%0A
HMA-S Final Presentation

32. Correlate feeds (geometry and names)
Each product geometry mapped to a names search feed
HMA-S Final Presentation

33. Correlate feeds (geometry and names)
Search Engine Friendly
Direct Links to data holdings
Automatic tagging of granules
HMA-S Final Presentation

34. Interferometric Search
Special case of a correlation search dealing with specific Radar imagery information (e.g. Sentinel-1, ERS, Envisat).
It shares the same process and outputs but have quite different purposes and contexts.
Interferometric search is only performed when products to be combined are acquired by the same mission in compatible phases (i.e. in phases having the same orbit scheme)
HMA-S Final Presentation

35. Interferometric Search
Special correlation function with corresponding parameters (baseline and burst sync).
For each, it checks if it has the same track number, sensor mode and swath identifier (as defined OGC ) of the master product.
For every slave candidate product, the center time from the overlap area is computed and obtains the baseline and time from topsar values stored in the auxiliary files.
If then those interferometric correlation criteria are also met then the slave entry is accepted
From
HMA-S Final Presentation

36. Interferometric Search
<OpenSearchDescription xmlns=" xmlns:geo=" xmlns:param=" xmlns:time=" xmlns:cor=" ... <Url type="application/atom+xml" template=" pw={startPage?}&bbox={geo:box?}&correlatedTo={cor:with?}&spatialCover={cor:spatial?}&timeCover={cor:time}&function={cor:function?}&param={cor:parameter}"/> <param:Parameter name="function" value="{cor:function}" minimum="0" title=”Correlation Functions”> <param:Option value="interferometric" label=”Interferometric Search”/> </param:Parameter> <param:Parameter name="param" value="{cor:parameter}" minimum="0" title="Correlation Parameters" pattern="burst:[\+\-]?[\d]*,baseline:[\+\-]?[\d]*" />
HMA-S Final Presentation

37. HMAS - Task 4 OpenSearch Extensions
Thank YOU!
RSS