Frascati, 16 July 2009 Slide 1 CITE tests for and HMA-T Phase 2, AR-2 Meeting 16 July 2009, Frascati Nicolas Lesage, IGN

Frascati, 16 July 2009 Slide 2 Toc  Work performed  Open actions  Way forward

Frascati, 16 July 2009 Slide 3 Work performed (1/3)  WP0 – Management Meetings & telecons: KO meeting, September telecon Reporting  WP1 – Improve P&P HMAT-TN-0001-IGN, HMA-T Phase 2 Testing Policy, Issue 1, 25/08/2008HMAT-TN-0001-IGN  Defines the ATS and ETS development process for HMA-T Catalogue protocols  Specifies input, output, context, resources of the process HMAT-TN-0001-IGN, HMA-T Phase 2 Testing Policy, Issue 1, Revision 1, 03/02/2009HMAT-TN-0001-IGN  Includes an ATS template based on the other partners’ ATS layout HMAT-TN-0001-IGN, HMAT Phase 2 Testing policy - Issue 1, Revision 2, 09/03/2009HMAT-TN-0001-IGN

Frascati, 16 July 2009 Slide 4 Work performed (2/3)  WP2 – CITE Tests for OGC (EO EP) Consolidated ATS for EO EP, Issue 1, Revision 0 has been incorporated in EO EP v0.2.4Consolidated ATS for EO EP, Issue 1, Revision 0 ETS available on IGN branch of HMA svn repository Status  Test of mappings and few other tests have still to be debugged  WP3 – CITE Tests for OGC (CIM EP) CIM EP ATS and conformance CR including CIM XML files ETS available on IGN branch of HMA svn repository Status : Almost done

Frascati, 16 July 2009 Slide 5 Work performed (3/3)  Re-use and consolidation of the “best” tests developed by the different partners  Tests of the implementation of the RIM Model of the EO and CIM EP is done through a configuration file This avoids long CTL files repeating the same code and eases the development and maintenance This also provides a means to test the tests (changing the configuration files to force errors) and to debug the catalogue (changing the configuration files to accommodate some non conformances)  Package availability is tested using the EP canonical files as an input. Same advantage as the configuration file Direct from the spec to the implementation  Toward a tool box for ebRIM EP Tests Didn’t find a way to define configuration files for testing the mappings between RIM Model and repository files… There is a potential impact on the ATS (especially on EO EP)

Frascati, 16 July 2009 Slide 6 Open actions (1/3)  #35: Align conformance class to take into account proxy implementations Done in revised ATS. Action to be closed  #110: Integrate the ATS and ETS for both catalogue extension packages and interact with partners working on these topics. ATS : Done ETS : Almost done. Action to be closed (Finalisation of ETS/ATS to be discussed)  #114: Check use of coupled resource in OGC A study has been done by Frederic Houbie with no impact on CIM Action to be closed

Frascati, 16 July 2009 Slide 7 Open actions (2/3)  #118: Seek agreement from contributors on consolidated ATS Done. ATS has been incorporated in the draft EP. Action to be closed  #120: Reorganise contributed tests into test groups for EO, CIM and EbRIM AP. Check Galdos work for reuse in second iteration. Done. It has not been possible to make Galdos work working on a recent teamengine. Action to be closed  #121: Remove annex H.3 of CIM EP. Publish separate Discussion Paper. Provide Ats/ETS as 4th group. Done. Annex H.3 has been removed. Con Terra has taken the action to publish content of annex H.3 as discussion paper. Action to be closed  #122: Check that CTL scripts detect typical errors as encountered during Dali and MUIS integration i.e. name spaces, EOP, SAR, etc. ???

Frascati, 16 July 2009 Slide 8 Open actions (3/3)  #126: Update ATS and ETS based on today's discussion and distribute to partners. ATS Done Action related to ETS redundant with action #129 Action to be closed  #129: Release first version of ETS/CTL for CIM; ebRIM and EO scripts Done for CIM and EO EP Action to be closed. New action to be decided concerning ebRIM and more generally finalisation of ETS/ATS  #130: release enhanced version of ETS/CTL for CIM; ebRIM and EO scripts Done/Overcome by events (ETS under svn) Finalisation of ATS/ETS to be discussed Action to be closed

Frascati, 16 July 2009 Slide 9 Way forward  WP0 – Management As necessary  WP1 – Improve P&P Revision 3 (including ETS Template) to be submitted  INTECS or CNR-IMAA to release the documentation of the tool generating the documentation of the ETS from the CTL scripts  Proposal to reference the tool documentation from P&P  WP 2 & 3 – CITE tests To be discussed

Frascati, 16 July 2009 Slide 10 Analysis of EO EP trunk Abstraction of bindings and other enhancements  This is certainly a good work, but it should have been coordinated more closely with the different partners and particularly IGN  The trunk version is not aligned on the last EO EP Change Request to be released. INTECS?  SOAP tests shouldn’t be removed from the ATS/ETS even if there is the SOAP test binding: Support of SOAP is mandatory, SOAP test binding is optional Support of SOAP is a requirement, it should therefore be expressed as a test case in a conformance class (as defined in the OGC Policy document). SOAP capabilities may not be completely tested Error messages are unclear  Organization of tests differs from IGN One CTL file per test is easier for configuration management and development  The requirements for the documentation tool are not documented  Comments on the GUI It seems that all bindings can be tested in the same round. IGN suggests one binding at a time.  Only one endpoint  Choice of binding should be proposed as a drop-down list. Choice of conformance level should be made clear (i.e. level 1/core – EOP support, level 2 – SAR extension, etc.) Choice of CSW-ebRIM conformance level should be clearer (i.e. indicate that level 1 is read-only, level 2 is transactional)

Frascati, 16 July 2009 Slide 11 Other issues  How to ensure the convergence of EPs, their Catalogue and Test Implementations  Availability of resources and coordination  Schedule

Frascati, 16 July 2009 Slide 12 Feedback from CITE tests for HMA-T Phase 2 AR-2 Meeting 16 july 2009, Frascati Nicolas LESAGE, IGN

Frascati, 16 July 2009 Slide 13 A1.1 SOAP 1.2 support A.1.2 SOAP 1.1 support  Reuse existing tests

Frascati, 16 July 2009 Slide 14 A.1.3 GetRepositoryItem id paramter  Reuse existing tests

Frascati, 16 July 2009 Slide 15 A1.5 Instantiation of associations  Test based on a configuration file defining the characteristics of the association types and the related objects  Loop on the association types Request instances of each association type Check whether the source and target of the 1st instance exist and have a compatible type  Warning raised: No instance of an association type When no compatible type exists in the package, there may be one in an another package  this has to be ckeched by hand  Possible enhancement Sampling method: More recent instances? Random? Number of instances to be checked: More/Less than 1? All?...

Frascati, 16 July 2009 Slide 16 A1.6 Instantiation of attributes, slots and classifications Error raised  All properties : Number of property instances with respect to cardMin and cardMax  Attributes  (Conformance to Slot type définition in D2.5) Conformance of the value to the slot type:  Stereotype is classificationScheme => value is the classification node URN (what about external classsificationNode?)  Classifications Limitation to internal classification node and scheme? is set, its URN the property is set and its value is the id of is set and contains the id of the current is set to urn:oasis:names:tc:ebxml-regrep:ObjectType:RegistryObject:Classification If the property has a type, the classification should be classified  Check that the classification has a valid Classification  Associations When the type of the target object is external => check the target object instance  All Classifications and ExternalIdentifiers related to this object are “internal” to the object instance Check the cardinality of the object classifications and External Indentifiers within the object is equal to their cardinality within the repository.  Abstract object types shall not have instances (e.g. ResourceMetadata)  DataMetadata and ServiceMetadata have a wrs:repositoryItemRef subelement having a non

Frascati, 16 July 2009 Slide 17 A1.6 Instantiation of attributes, slots and classifications  Warning raised: No instance of a non-abstract object type  Test based on a configuration file defining the characteristics of the object types Implementation of constraint defined in Figure 16 of CIM (See B.6) ?  Context CitedItem inv: self.Created+self.Modified+self.Issued>0 Test of “external” object types?  All properties of an object type are tested including those inherited Teamengine does not seem to support xsl:template including ctl elements  Possible enhancement Sampling method: More recent instances? Random? Number of instance to be checked: All? More/Less than 100?...  Also applies to A.2.1, A.2.2, A.3.1, A.3.2, A3.3 and A3.4 tests

Frascati, 16 July 2009 Slide 18 A.1.7 Package availability  2 possible requests: listExtensionPackages stored queries: <csw:GetRecords … outputSchema="urn:oasis:names:tc:ebxml-regrep:xsd:rim:3.0" version="2.0.2" service="CSW" resultType="results" startPosition="1" maxRecords="10"> EP are contained by the root registry package <csw:GetRecordById … service="CSW" version="2.0.2" outputSchema="urn:oasis:names:tc:ebxml-regrep:xsd:rim:3.0" urn:ogc:def:ebRIM-RegistryPackage:OGC:Root full  No implementation supporting the root registry package  Limited support of the stored query by CNR-IMAA

Frascati, 16 July 2009 Slide 19 A.1.7 Package availability  Improvement Tests are based on CIM Package definition Tests relate to: @objectType), @associationType), …

Frascati, 16 July 2009 Slide 20 A.1.8 ISO MD_Metadata.identificationInfo implementation  Verify that the ISO repository item contains only one instance of the MD_Metadata.identificationInfo property Also verify that the ISO repository item contains at least one instance.

Frascati, 16 July 2009 Slide 21 A.1.9 Mappings implementation  No generic mechanism found to complete an EP test toolbox  Reuse of CNR-IMAA tests  Also applies to A.2.3, A.2.4 and A.3.5 tests

Frascati, 16 July 2009 Slide 22 Common Conformance Issues  Implementation of External Identifiers (Classification) as “independent” Registry objects instead of ExternalIdentifier (Classification) sub-element  Should a test be added specifically for External Identifiers and Classification?  Naming of properties and type Case? Differences between logical names (used in UML models) and implementations names (i.e. URI or identifiers in general) Basic Package slot names are URL (i.e. from Dublin Core) EO EP slot names are URN CIM EP slot names are UpperCamelCase strings Language type of Regrep 3 or LanguageTag type of Basic Package ? No classification node for InternationalString and LocalizedString in Regrep 3 and Basic Package  Impossible to access the package definitions No support of listExtensionPackage stored query Root package does not exist

Frascati, 16 July 2009 Slide 23 Conformance of ERDAS endpoint ( )  Instantiation of associations (A1.5 of CIM, …) Association and object URNs are not those defined in CIM  urn:x-ogc:specification:csw-ebrim-cim:AssociationType:* instead of urn:ogc:def:ebRIM- AssociationType:OGC:* 2/10 association types are implemented  Instantiation of attributes, slots and classifications (A1.6 of CIM, …) Association and object URNs are not those defined in CIM Implementation of External Identifiers (Classification) “independent” Registry objects instead of ExternalIdentifier (Classification) sub-element  Should a test be added specifically for External Identifiers and Classification? Uppercase/lowercase issues in property names  Package availability (A1.7 of CIM, …) Impossible to access the package definitions  No support of listExtensionPackage stored query  Root package does not exist  ISO MD_Metadata.identificationInfo implementation (A1.8 of CIM) Repository items are attached to object type *:MetadataInformation while CIM states that « the only object types that are expected to be linked to repository items are : *:ServiceMetadata and *:DataMetadata. “

Frascati, 16 July 2009 Slide 24 Conformance of CNR-IMAA endpoint (  Package availability (A1.7 of CIM, …) Limited support of listExtensionPackage stored query  Catalogue seems to be empty

Frascati, 16 July 2009 Slide 25 CIM Issues  Slot type as Classification Scheme ? #01 Format slot type defined as String in Table D20, but as FormatNameAndVersion in C.2.1 and Figure 9 #02ConformsTo slot type defined as InternationString in Table D18, but as MetadataStandardNameAndVersion in C.2.1 and Figure 9 and Figure 16  Slot type Slot type of Language is Language in CIM, but LangaugeTag in Basic Package Slot type of Conformance is Boolean (Table 28 states that it is InternationalString)  Slot name Slot name should be defined as a URI  Modified is named in Basic Packagehttp://purl.org/dc/terms/modified  Source is named in Basic Packagehttp://purl.org/dc/elements/1.1/source  date is named in Basic Packagehttp://purl.org/dc/elements/1.1/date  Language is named in Basic Packagehttp://purl.org/dc/elements/1.1/language date should be Date  Envelope Slot is not defined in CIM, neither in Basic Package  Slot type can’t be an ExtrinsicObject Coverage Slot of DataMetadata should rather be an Association  Organization should described in C.2.6, since constraints related to the Name attribute are expressed in Figure 15

Frascati, 16 July 2009 Slide 26 Basic Package Issues  Slot type Date Slot type does not exist in ebRIM  urn:oasis:names:tc:ebxml-regrep:DataType:Date should be urn:oasis:names:tc:ebxml-regrep:DataType:DateTime  LanguageTag data type Is defined and used in BP instead of ebRIM Language data type LanguageTag refers to RFC 4646 which supersedes RFC 3060 refered to by Language RFC 4646 statement  “It is important to note that all language tags formed under the guidelines in this document were either legal, well-formed tags or could have been registered under RFC “ Is this type really necessary?

Frascati, 16 July 2009 Slide 27 Teamengine Issues  Lack of documentation Installation of tomcat and configuration of teamengine sources for deployment Debugging Tests (use of log files,...)  Partial support of xsl:template Named template including ctl elements are not processed

Frascati, 16 July 2009 Slide 28 Feedback from CITE tests for HMA-T Phase 2 AR-2 Meeting 16 july 2009, Frascati Nicolas LESAGE, IGN

Frascati, 16 July 2009 Slide 29 Overview  Core conformance class : support of ‘EOP’ slots and objects 26 test cases Verify:  registry package existence,  instantiations,  mappings, SOAP 1.2 and 1.1 bindings  and other various service capabilities  SAR extension : support of SAR slots 3 test cases  OPT extension : support of OPT slots 3 test cases  ATM extension : support of ATM slots and objects 4 test cases

Frascati, 16 July 2009 Slide 30 EO EP ATS issues  Should test cases concerning verification of associations be merged into one single test case? CIM has only one test case What is the expected level of granularity for requirements & tests?  Cardinality requirements on associations need to be clarified in EO EP spec  Same question for verification of extrinsic objects

Frascati, 16 July 2009 Slide 31 EO EP Spec issues  What namespace for the EO EP ? Proposed:  Data should use URNs for data types defined in ebRIM or in the Basic Package, not simple strings E.g. urn:oasis:names:tc:ebxml-regrep:DataType:String instead of string E.g. urn:ogc:def:dataType:ISO-19107:2003:GM_Object instead of geometry  EOProduct ExtrinsicObject Contains different slots acc. to conformance level I.e. EOP slots in the core, EOP + SAR slots in SAR extension Another option would be to define a SAR EOProduct deriving from the base EOProduct  rim:externalIdentifier and rim:classification should be implemented as sub- elements of the rim/wrs:ExtrinsicObject, not as separate objects, to follow CSW ebRIM practices  URNs should be submitted to and approved by OGC Naming Authority For next version?

Frascati, 16 July 2009 Slide 32 A1.1 Support of SOAP 1.2  Test based on CNR-IMAA test  Issue : SOAP support advertisement in Capabilities document is not standardized Capabilities document cannot be used to retrieve SOAP service URL  SOAP service URL must be input by user Issue for OWS Common 1.2 SWG E.g. service CNR-IMAA - ows:Constraint is a key-value structure  very open - Values for bindings (including versions) are not standardized

Frascati, 16 July 2009 Slide 33 A1.2 Package avaibility  Purpose : verify that the canonical EO EP registry package is available and correct  Method: cf. related CIM test  Issues: listExtensionPackages stored query only supported by CNR-IMAA service

Frascati, 16 July 2009 Slide 34 A.1.3 Collection identifiers advertisement  Purpose: Verify that collection identifiers are advertised in the Capabilities  Method: GetCapabilities  Issues: Collection id advertisement is not implemented in the tested services  in fact, DALI declares xmlns:gen0=" while the test, in application of the spec, looks for the ows prefix"

Frascati, 16 July 2009 Slide 35 A.1.4 EOProduct acquisition parameters instantiation  Purpose: Verify that all mandatory slots, classifications and associations of an EOProduct extrinsic object are correctly implemented  Method: cf. related CIM test Structure of EOProduct (no inheritance, same object with different content at the different conformance levels) required to customize the tests originally designed as common to CIM and EO EP.  Issues rim:ExternalIdentifier and rim:Classification should be instantiated as subelements of the rim/wrs:ExtrinsicObject slotTypes should be URNs

Frascati, 16 July 2009 Slide 36 Tests similar to A.1.4  A.1.9 EOProductAcquisitionPlatform instantiation  A.1.12 EOProductInformation instantiation  A.1.15 EOBrowseInformation instantiation  A.1.18 EOMaskInformation instantiation  A.1.21 EOArchivingInformation instantiation  A.2.1 SAR-specific slots instantiation  A.3.1 OPT-specific slots instantiation  A.4.2 EODataLayer instantiation

Frascati, 16 July 2009 Slide 37 A.1.8 AcquiredBy association instantiation  Purpose: verify that the association is correctly implemented, i.e. that the role cardinalities are respected and the source and target objects really exist  Method: cf. CIM-related test  Issues/Warnings Warnings are raised when optional associations are not instantiated

Frascati, 16 July 2009 Slide 38 Tests similar to A.1.8  A.1.11 HasProductInformation association instantiation  A.1.14 HasBrowseInformation association instantiation  A.1.17 HasMaskInformation association instantiation  A.1.20 ArchivedIn association instantiation  A.4.1 HasDataLayer association instantiation

Frascati, 16 July 2009 Slide 39 A.1.24 GetRepositoryItem id parameter  Purpose: Verify that the GetRepositoryItem operation works with the proper extrinsic object identifiers  Method: Cf. CIM-related test

Frascati, 16 July 2009 Slide 40 Tests still under development  Mappings A.1.5 EOProduct acquisition parameters mapping A.1.7 EOProduct classification mapping A.1.10 EOProductAcquisitionPlatform mapping A.1.13 EOProductInformation mapping A.1.16 EOBrowseInformation mapping A.1.19 EOMaskInformation mapping A.1.22 EOArchivingInformation mapping A.2.2 SAR-specific slots mapping A.3.2 OPT-specific slots mapping A.4.3 EODataLayer mapping  Others A.1.6 EOProduct classification instantiation A.1.23 EO Product metadata grouping A.1.26 SOAP 1.1 support

Frascati, 16 July 2009 Slide 41 Conformance of ESA HMA-E RSC Catalog endpoint  URL:  Object id: ESA.EECF.ENVISAT_ASA_GMI_1S:EN XG0  A.1.1 – SOAP 1.2 support Not supported (not listed in the Capabilities document)  A.1.2 – Package avaibility Cannot be tested because does not support listExtensionPackage stored query  A.1.3 – Collection identifiers advertisement Not configured  A.1.4 – EOProduct acquisition parameters instantiation Fails (external identifier is outside the extrinsic object definition)  A.1.9, A.1.12, A.1.15, A.1.21 (tests on objects and slots instantiations) Pass. Warning are raised when optional slots are not instantiated.

Frascati, 16 July 2009 Slide 42 Conformance of ESA HMA-E RSC Catalog endpoint  A.1.8, A.1.11, A.1.14, A.1.17, A.1.20 (association instantiation) Pass  A EOMaskInformation instantiation Fails, some mandatory slots are missing Are these slots actually optional?  A.1.24 – GetRepositoryItem id parameter Fails, no repository item linked to object ESA.EECF.ENVISAT_ASA_GMI_1S:EN XG0

Frascati, 16 July 2009 Slide 43 Conformance of ESA HMA-E RSC Catalog endpoint  SAR extension Tests fail because sampling method chooses a non-SAR item Method should be modified to pick SAR items  OPT Extension Is there any OPT object in the catalogue?  ATM Extension Is there any ATM object in the catalogue?

Frascati, 16 July 2009 Slide 44 Conformance of ESA Infeo2 Catalog endpoint  URL:  Not accessible

Frascati, 16 July 2009 Slide 45 Conformance of Spot Image endpoint  URL:  Was not accessible as of 14 July (except GetCapabilities) Seems to be in service since 15 July Feedback to be provided to Spot Image

Frascati, 16 July 2009 Slide 46 Conformance of CNR-IMAA endpoint  URL:  Object id: urn:ogc:def:EOP:ESA.EECF.ENVISAT_ASA_GMI_1S:EN XG0:EOProduct  A.1.1 – SOAP 1.2 support Endpoint listed in the capabilities but test fails Test must be checked  A.1.2 – Package avaibility listExtensionPackages attribute missing on classification nodes in EO EP XML file Specific associations (defined in order to test the source and target objects of associations) are not part of CNR-IMAA canonical registry package. Is it the intended behaviour?  A.1.3 – Collection identifiers advertisement Pass, one collection is advertised  A.1.4 – EOProduct acquisition parameters instantiation Fails because GetRecords request does not match any record  A.1.9, A.1.12, A.1.15, A.1.21 (tests on objects and slots instantiations)* Fails because GetRecords request does not match any record

Frascati, 16 July 2009 Slide 47 Conformance of CNR-IMAA endpoint  A.1.8, A.1.11, A.1.14, A.1.17, A.1.20 (association instantiation) Fails, no record matched  A EOMaskInformation instantiation Fails, objects & slots missing  A.1.24 – GetRepositoryItem id parameter Fails, no repository item linked to object urn:ogc:def:EOP:ESA.EECF.ENVISAT_ASA_GMI_1S:EN XG0:EOProduct

Frascati, 16 July 2009 Slide 48 Conformance of VITO endpoint  Catalogue seems to be empty  SAR extension Is there any SAR object in the catalogue?  OPT Extension Is there any OPT object in the catalogue?  ATM Extension Is there any ATM object in the catalogue?

Frascati, 16 July 2009 Slide 49 Conformance of VITO endpoint  URL:  Object id: vito.vgt4africa.vpi:VGT4AFRICA_VPI_ _Africa  A.1.1 – SOAP 1.2 support Not supported (not listed in the Capabilities document)  A.1.2 – Package avaibility Cannot be tested because does not support listExtensionPackage stored query  A.1.3 – Collection identifiers advertisement Not configured  A.1.4 – EOProduct acquisition parameters instantiation Passes (external identifiers and classifications are inside the extrinsic object definition)  A.1.9, A.1.12, A.1.15, A.1.21 (tests on objects and slots instantiations) Pass. Warning are raised when optional slots are not instantiated.

Frascati, 16 July 2009 Slide 50 Conformance of VITO endpoint  A.1.8, A.1.11, A.1.14, A.1.17, A.1.20 (association instantiation) Pass  A EOMaskInformation instantiation Fails, some mandatory slots are missing Are these slots actually optional?  A.1.24 – GetRepositoryItem id parameter Fails, no repository item linked to object vito.vgt4africa.vpi:VGT4AFRICA_VPI_ _Africa

Frascati, 16 July 2009 Slide 51 Conformance of VITO endpoint  It seems that only VGT products are described in the catalogue  SAR extension Is there any SAR object in the catalogue?  OPT Extension Is there any OPT object in the catalogue?  ATM Extension Is there any ATM object in the catalogue?