AR Meeting - 27 January 2011, ESA (Frascati) HMA FO Task 1 : EO Metadata Extensions Frédéric Houbie, ERDAS Steven Smolders, GIM Andrew Woolf, STFC Dominic Lowe, STFC

AR Meeting - 27 January 2011, ESA (Frascati) Overview Task description Deliverables EO Product Extensions & Cataloguing XML Schemas for EO Products Cataloguing of EO Product OGC standardisation status Testing CITE Testing HMA Skeleton SSE Workflow RIDS Open Actions Slide 2

AR Meeting - 27 January 2011, ESA (Frascati) Task 1 Planning  Slide 3

AR Meeting - 27 January 2011, ESA (Frascati) Tasks description EO Metadata Analysis of extensions for new Product Types update of the EO GML Metadata specification update of the cataloguing model OGC Process EO Metadata v2 Cataloguing of EO Metadata v2 using ebRIM model Testing CITE HMA Skeleton SSE  Slide 4

AR Meeting - 27 January 2011, ESA (Frascati) Deliverables Analysis of the extensions (HMAFOT1-TN-0002-ERDAS-17) Analysis of the Catalogue update (HMAFOT1-TN-0001-ERDAS-13) New Schemas for EO Profile of O&M Schematron r1 Earth Observation Metadata profile of Observations and Measurements r2 Cataloguing Earth Observation Products for ebRIM 3.0 based Catalogues ATS & ETS Acceptance Test Plan and Report Update of HMA Skeleton ( Update of SSE Client: BPEL workflow for GetRepositoryItems and GetRepositoryItemById and associated portal files (xsd and xslt) ( test.eoportal.org/)  Slide 5

AR Meeting - 27 January 2011, ESA (Frascati) EO Product Types Base EO Product Optical Radar Atmospheric Altimetry Limb Looking Synthesis & Systematic Products Slide 6

AR Meeting - 27 January 2011, ESA (Frascati) EO GML extension : schemas Version 1.0 based on GML Schema implementation options straightforward migration to GML model-driven approach to GML O&M with minimal refactoring O&M with more extensive refactoring  Changes to base schemas/instances due to GML Model driven approach: object/property model O&M Correction resulting from HMA-T Addition of useful properties (angles, wavelengthInformation, auxiliary dataset, orientation, parameterInformation

AR Meeting - 27 January 2011, ESA (Frascati) EO Product Extensions Slide 8

AR Meeting - 27 January 2011, ESA (Frascati) OGC Describes the data model, based on UML List of properties, with description and cardinality Removed the ISO mapping Currently not generated from UML because the structure is simplified for easier understanding. Slide 9

AR Meeting - 27 January 2011, ESA (Frascati) Samples We currently have samples on EOP, OPT, SSP Input welcome to create other « realistic » samples. Slide 10

AR Meeting - 27 January 2011, ESA (Frascati) Items to fix Cardinality of Acquisition Platform in ALT & SSP Identifier of instrument ? SPOT4-VGT1 SPOT4/VGT1 SPOT-4-VGT-1 urn:dlr:instruments:TerraSAR-X urn:spot:sensors:SPOT5:HRG Slide 11 OPTICAL 1000 VGT1 VGT2 SPOT 5 SPOT 4

AR Meeting - 27 January 2011, ESA (Frascati) ETS for EO Profile of O&M XSD Schemas & Schematron Issue : execution of both validations using TeamEngine Error of namespace In discussion with OGC TeamEngine session/training next TC in Bonn Slide 12

AR Meeting - 27 January 2011, ESA (Frascati) EO Product Cataloguing Slide 13 Current version is OGC Standard 1.0 EO Product Extension Package of ebRIM 3.0 Used with CS-W & ebRIM AP 1.0

AR Meeting - 27 January 2011, ESA (Frascati) Issues of the current model Complexity of the model structure Maintainability and extensibility of the EO EP Full mapping = EO GML duplication Simple change in cardinality impacts the model Adding new metadata element – even if they are not queryable (ex : add several product locations with different formats) – impacts the model Many request to retrieve EO GML Metadata Slide 14

AR Meeting - 27 January 2011, ESA (Frascati) Retrieving Metadata Slide 15 Number of requests to display 100 results: 1 GetRecords GetRepositoryItem = 101 requests

AR Meeting - 27 January 2011, ESA (Frascati) Improvements Simplify model Improve GetRecords operation Slide 16

AR Meeting - 27 January 2011, ESA (Frascati) ebRIM Discovery model Slide 17

AR Meeting - 27 January 2011, ESA (Frascati) Improve GetRecords Currently GetRecords  ebRIM Model (discovery) GetRepositoryItem  Metadata Proposal GetRecords  Does not change (except ElementSetName) GetRepositoryItemById to supersede GetRepositoryItem GetRepositoryItems  Retrieve a set of metadata records using a filter Accepted by ebRIM 2.0 SWG Definition of Core & Extensions to be done Slide 18

AR Meeting - 27 January 2011, ESA (Frascati) GetRepositoryItemById <wrs:GetRepositoryItemById xmlns:wrs=" version="2.0" id="DS_PHR1A_ _TLS_PX_E123N45_0101_01234_LMB"> Slide 19

AR Meeting - 27 January 2011, ESA (Frascati) GetRepositoryItemById Response Slide T11:02: T11:02: ……………………………………………………………………. TLS T11:02:

AR Meeting - 27 January 2011, ESA (Frascati) GetRepositoryItems <wrs:GetRepositoryItems xmlns:wrs=" xmlns:ogc=" maxRecords="10" startPosition="1"> /ExtrinsicObject urn:ogc:def:objectType:OGC-CSW-ebRIM-EO:2.0:EOProduct urn:ogc:def:EOP:VGT_S Slide 21

AR Meeting - 27 January 2011, ESA (Frascati) GetRepositoryItems Response Slide 22

AR Meeting - 27 January 2011, ESA (Frascati) OGC : EO O&M Presentation during Sydney Plenary FastTrackProcess can start OAB is waiting for the document for review Questions from Carl Reed : 1. Are there any implementations of the profile? Just wondering. 2. Issue of the title. Earth Observation Metadata profile of Observations & Measurements. There is some concern that this is too generic. Other domains that work earth observation may also develop profiles of O&M for their domain. So, could we add an adjective or so to further refine the title and increase the focus? Perhaps HMA Earth Observation Metadata profile of Observations & Measurements? Slide 23

AR Meeting - 27 January 2011, ESA (Frascati) OGC (EP) Presentation also done at Sydney Issue about the number of implementations to become a standard (3 are required) More important issue about the status of CS-W3 and ebRIM 2 linked to CS-W Slide 24

AR Meeting - 27 January 2011, ESA (Frascati) OGC : URN vs URI OGC policy uses now URL (HTTP URI) as identifier OASIS mandates use of URN shall use then URN for objecttypes, associationtypes Slotname could be HTTP URI, but having both style of URI in a single document is strange Update of the version in the urn to be in sync with EO SPS Slide 25

AR Meeting - 27 January 2011, ESA (Frascati) ATS  A.1.Conformance Test Class: The Core (EOP support)  A.1.1Package availability  A.1.2Collection identifiers advertisement  A.1.3EOProduct parameters instantiation  A.1.4EOProduct parameters mapping  A.1.5EOProduct classification instantiation  A.1.6EO_Product_Types classification mapping  A.1.7AcquiredBy association instantiation  A.1.8EOProductAcquisitionPlatform instantiation  A.1.9EOProductAcquisitionPlatform mapping  A.1.10EO Product metadata grouping (RegistryPackage)  A.1.11GetRepositoryItem id parameter  A.1.12Harvest operation Slide 26

AR Meeting - 27 January 2011, ESA (Frascati)  A.2.Conformance Test Class: SAR Extension : The SAR Extension conformance class extends the Core conformance class (cf. A.1).  A.2.1SAR-specific slots instantiation  A.2.2SAR-specific slots mapping  A.2.3Advertisement of SAR support for Harvest operation in Capabilities document Equivalent for all product types Slide 27

AR Meeting - 27 January 2011, ESA (Frascati) ETS Test Suite Slide 28

AR Meeting - 27 January 2011, ESA (Frascati) Live Testing Team Engine : Catalog URL endpoint : EOP item id : DS_PHR1A_ _TLS_PX_E123N45_0101_01234 OPT item id: DS_PHR1A_ _TLS_PX_E123N45_0101_01234_OPT SAR item id: DS_PHR1A_ _TLS_PX_E123N45_0101_01234_SAR ATM item id: DS_PHR1A_ _TLS_PX_E123N45_0101_01234_ATM ALT item id: DS_PHR1A_ _TLS_PX_E123N45_0101_01234_ALT LMB item id: DS_PHR1A_ _TLS_PX_E123N45_0101_01234_LMB SSP item id : urn:ogc:def:EOP:VITO:VGT_S10:V2KRNS10__ E Slide 29

AR Meeting - 27 January 2011, ESA (Frascati) ETS issues Spatial comparison Comparing Geometry is very difficult Checking the Spatial filter is well applied is difficult To check validity of spatial filtering, solution would be :  GetRecordById  EOProduct  Extract Box as RefBox  GetRecords where Id = X & Box intersect RefBox : should get one  Calculate Box outside RefBox as outsideBox  GetRecords where Id = X & Box intersect outsideBox : should get none Doing this using TeamEngine is quite challenging Slide 30

AR Meeting - 27 January 2011, ESA (Frascati) HMA Skeleton Simulation framework for Web Services Set of requests and responses messages for each WS operation that are joined using a testcase XML file with xpath expressions Catalogue_OGC _version-020_ebRIMCSW-1 Updated all requests and responses to reflect light model Implementation of new slot names Addition of requests/responses for new product types GetRepositoryItems GetRepositoryItemById Implementation of HTTP GET based GetRepositoryItem Servlet Slide 31

AR Meeting - 27 January 2011, ESA (Frascati) HMA Skeleton (ctd) Created schema for CSW-ebRIM 2.X in order to validate requests/responses Helped finetuning the 2 specifications and the ATS/ETS Slide 32

AR Meeting - 27 January 2011, ESA (Frascati) HMA Skeleton –Team engine Slide 33  Remaining Issues to be further investigated (Wrong endpoint for ATC1.8/ATC 1.9)

AR Meeting - 27 January 2011, ESA (Frascati) SSE Portal Integration Testing new GetRepositoryItems & GetRepositoryItemById approach Modifed workflows  Search: modified GetRecords workflow to use GetRepositoryItems operation using simplified model  Present: modified GetRecordById workflow to use GetRepositoryItemById operation  Present: not really required for future implementations Updated portal files: xsd and xslt  Using ERDAS CSW-ebRIM-2.X implementation Slide 34

AR Meeting - 27 January 2011, ESA (Frascati) SSE Portal Integration Demonstration Slide 35

AR Meeting - 27 January 2011, ESA (Frascati) SSE Portal Integration Demonstration Slide 36

AR Meeting - 27 January 2011, ESA (Frascati) RIDS on r1 $3 I suspect many of these are informative and not normative references (e.g ) OK $3 due to the MDA approach shouldn't the UML be presented, possibly as an annex, at least as a " web reference " towards the SVN (I also note that the SVN is a branch under the old GML spec... does it also need to be re-baselined " under the new document numbering OK : html documentation available in the SVN under folder « OGC %20(OM EO App Schema) » $A some Error reference not found (at least in my printout) OK $B examples should be at least 1 per extension... possibly also an example of mission specific extension for phr if we can get it OK, could we have some realistic samples from agencies following verison 1 of the spec, we will convert them Slide 37

AR Meeting - 27 January 2011, ESA (Frascati) RIDS on r2 $2.1 compliance has no discussion of limit to the query complexity as outlined in Annex I - shouldn't a requirement be added? We can draft a set of requirements on the queryables, but these are HMA specific. If this profile has to become an OGC standard, it should stay open on that (cf comment received from OGC) $2.3 the level 2 compliance is introduced for harvesting, this has always been an optional element of the spec, wouldn't it be simpler to redefine the reqts. such that they state " the optional harvest operation definition advertises the support for EO Products " so that the 2 levels don't need to be introduced OK, agree that this level should not appear in an extension Package as this is defined in the upper specification, CS-W ebRIM $8.2.1 you should illustrate how the spec should be extended for a mission specific profile Will ask CNES for PHR extension on this profile $8.4.1 the discussion should be improved, also to mention the advertisement of the parentIdentifiers in the capabilities (as per reqt A.1.2) OK, this is described in C.3.1 but will move it here $8.2 figure 5 has some mistakes?! ATM includes cloudCover (not according to O&M), EO acquisition platfrorm has +name-changed?? OK, wrong image inserted in document $12.2 Error reference not found in footnotes Slide 38

AR Meeting - 27 January 2011, ESA (Frascati) RIDS on r2 $C.2.3 We could also provide a Temporal Query clarification - the " usual « query pattern for EO products overlapping a temporal range is "start LE t2 AND end GE t1" (a diagram could help) Ok, will draft it $C2.3 Point to further discussion in Annex I $C.4 move the two SOAP actions for next revision of CS-W AP Into Annex D with the rest of the discussion OK $C but we shouldn't recommend this profile, since we know it to be inefficient OK $D.1.2 the definition looks VERY imprecise, should be at a level to refer at least to the parameters as per Table what is the semantic for ElementName -- brief, full etc. (to be ignored...) OK, to be imported from CS-W ebRIM 2.0 Proposal $D.1 end-to-end identifiers. In GSCDA implementations we found that there was a lack of concept for end-to-end identifiers and concluded that the registry package identifier should be used for the ordering. With the new pattern of GetRegistryPackages the ExtrinsicObject comes back as a wrapper in the responses, shouldn't this also prefer the registry package concept to reduce the number of degrees of freedom for " complex " objects. To be discussed Slide 39

AR Meeting - 27 January 2011, ESA (Frascati) RIDS on r2 $D what other effect does the CSW3.0 have on the interface?? - e.g. now provide the ability to return partial status for distributed search Not clear now, stil in early draft $E.1 SOAP 1.2 is not an HMA constraint (SOAP 1.1 is) OK $E.2 Use of ParentIdentifier was already discussed in $ explain how the parentIdentifier is used, E.2 add some constraint in HMA about the structure of the parentIdentifier $E given the above, the annex seems to be un-necessary It was a request in the previous version to move HMA specific requirements to a informative Annex to facilitate OGC process $F.3 perhaps the use of ParentIdentifier should mentioned in this section (to avoid " useless " querying of the catalogues) To be discussed, isn’t it a HMA architecture use case ? $G Examples of the GetRepositoryItems and GetRepositoryItemById are required OK $1.1.2 Circle, add a note that the " standard " UOM used in most implementations, and recommended is m. OK Slide 40