1. CITE tests for 06-131 and 07-038 HMA-T Phase 2, FP Meeting
15 december 2009, Frascati
Nicolas Lesage, IGN
Slide 1

2. Toc
Work performed
Open actions
Way forward
Slide 2

3. Achievements WP0 – Management WP1 – Improve P&P
Meetings & telecons: KO meeting, September telecon
Reporting
WP1 – Improve P&P
HMAT-TN-0001-IGN, HMA-T Phase 2 Testing Policy, Issue 1, 25/08/2008
Defines the ATS and ETS development process for HMA-T Catalogue protocols
Specifies input, output, context, resources of the process
HMAT-TN-0001-IGN, HMA-T Phase 2 Testing Policy, Issue 1, Revision 1, 03/02/2009
Includes an ATS template based on the other partners’ ATS layout
HMAT-TN-0001-IGN, HMAT Phase 2 Testing policy - Issue 1, Revision 2, 09/03/2009
HMAT-TN-0001-IGN, HMAT Phase 2 Testing policy - Issue 1, Revision 3, 10/12/2009
Slide 3

4. Achievements (2/3) WP2 – CITE Tests for OGC 06-131 (EO EP)
Consolidated ATS for EO EP, Issue 1, Revision 0 has been incorporated in EO EP v0.2.4
ETS available on Trunk branch of HMA svn repository
EO EP tests can be performed on Montgomery for:
v0.2.4
v0.2.5
WP3 – CITE Tests for OGC (CIM EP)
CIM EP ATS and conformance CR including CIM XML files
CIM EP tests can be performed on Montgomery for CIM v and CIM v0.1.11
Slide 4

5. Achievements (3/3)
Re-use and consolidation of the “best” tests developed by the different partners
Tests of the implementation of the RIM Model of the EO and CIM EP is done through a configuration file
This avoids long CTL files repeating the same code and eases the development and maintenance
This also provides a means to test the tests (changing the configuration files to force errors) and to debug the catalogue (changing the configuration files to accommodate some non conformances)
Package availability is tested using the EP canonical files as an input.
Same advantage as the configuration file
Direct from the spec to the implementation
Toward a tool box for ebRIM EP Tests
Didn’t find a way to define configuration files for testing the mappings between RIM Model and repository files…
Organization of HMA svn to avoid redundancy of codes and ease maintenance
Slide 5

6. Open actions (1/2)
#130: Release enhanced version of ETS/CTL for CIM; ebRIM and EO scripts
CIM: ETS are available on svn and montgomery for 2 versions of CIM spec ( and )
EO: ETS are available on svn and montgomery for 2 versions of EO spec (0.2.4 and 0.2.5)
ebRIM: See #197
#197: Define SVN/CTL structure for ebRIM AP so other partners can contribute
Done
No contributions received
#214: take requirement about multiple endpoints in account at later stage and make available a working version first to be enhanced
See COM#01
Slide 6

7. Open actions (2/2)
#215: Check and correct test which may be broken due to CNR update
Done.
Action to be closed
#216: if specification is unclear about which ID to use, communicate information to ERDAS.
See COM#xx
#217: Discuss new parameter "obtain" off-line with Intecs and CNR
#218: Send when CIM EP are complete including canonical file
Slide 7

8. Remaining issues
Lack of endpoints implementing the recent version of EO and CIM EP spec
Few clarifications expected in CIM EP
Classifications and InternationalSTring slots
Slide 8

9. Way forward WP0 – Management WP1 – Improve P&P WP 2 & 3 – CITE tests
As necessary
WP1 – Improve P&P
Revision 3 (including ETS Template) to be submitted
INTECS or CNR-IMAA to release the documentation of the tool generating the documentation of the ETS from the CTL scripts
Proposal to reference the tool documentation from P&P
WP 2 & 3 – CITE tests
To be discussed
Slide 9

10. Analysis of EO EP trunk Abstraction of bindings and other enhancements
This is certainly a good work, but it should have been coordinated more closely with the different partners and particularly IGN
The trunk version is not aligned on the last EO EP
Change Request to be released. INTECS?
SOAP tests shouldn’t be removed from the ATS/ETS even if there is the SOAP test binding:
Support of SOAP is mandatory, SOAP test binding is optional
Support of SOAP is a requirement, it should therefore be expressed as a test case in a conformance class (as defined in the OGC Policy document).
SOAP capabilities may not be completely tested
Error messages are unclear
Organization of tests differs from IGN
One CTL file per test is easier for configuration management and development
The requirements for the documentation tool are not documented
Comments on the GUI
It seems that all bindings can be tested in the same round. IGN suggests one binding at a time.
Only one endpoint
Choice of binding should be proposed as a drop-down list.
Choice of conformance level should be made clear (i.e. level 1/core – EOP support, level 2 – SAR extension, etc.)
Choice of CSW-ebRIM conformance level should be clearer (i.e. indicate that level 1 is read-only, level 2 is transactional)
Slide 10

11. Other issues
How to ensure the convergence of EPs, their Catalogue and Test Implementations
Availability of resources and coordination
Schedule
Slide 11

12. Feedback from CITE tests for 07-038 HMA-T Phase 2 AR-2 Meeting
16 july 2009, Frascati
Nicolas LESAGE, IGN
Slide 12

13. A1.1 SOAP 1.2 support A.1.2 SOAP 1.1 support
Reuse existing tests
Slide 13

14. A.1.3 GetRepositoryItem id paramter
Reuse existing tests
Slide 14

15. A1.5 Instantiation of associations
Test based on a configuration file defining the characteristics of the association types and the related objects
Loop on the association types
Request instances of each association type
Check whether the source and target of the 1st instance exist and have a compatible type
Warning raised:
No instance of an association type
When no compatible type exists in the package, there may be one in an another package
this has to be ckeched by hand
Possible enhancement
Sampling method: More recent instances? Random?
Number of instances to be checked: More/Less than 1? All?...
Slide 15

16. A1.6 Instantiation of attributes, slots and classifications Error raised
All properties :
Number of property instances with respect to cardMin and cardMax
Attributes
Slots
@slotType (Conformance to Slot type définition in D2.5)
Conformance of the value to the slot type:
Stereotype is classificationScheme => value is the classification node URN (what about external classsificationNode?)
Classifications
Limitation to internal classification node and scheme?
is set, its URN the property Name
@classificationNode is set and its value is the id of node
@classifiedObject is set and contains the id of the current object
@objectType is set to urn:oasis:names:tc:ebxml-regrep:ObjectType:RegistryObject:Classification
If the property has a type, the classification should be classified
Check that the classification has a valid Classification
Associations
When the type of the target object is external => check the target object instance
All Classifications and ExternalIdentifiers related to this object are “internal” to the object instance
Check the cardinality of the object classifications and External Indentifiers within the object is equal to their cardinality within the repository.
Abstract object types shall not have instances (e.g. ResourceMetadata)
DataMetadata and ServiceMetadata have a wrs:repositoryItemRef subelement having a non
Slide 16

17. A1.6 Instantiation of attributes, slots and classifications
Warning raised:
No instance of a non-abstract object type
Test based on a configuration file defining the characteristics of the object types
Implementation of constraint defined in Figure 16 of CIM (See B.6) ?
Context CitedItem inv: self.Created+self.Modified+self.Issued>0
Test of “external” object types?
All properties of an object type are tested including those inherited
Teamengine does not seem to support xsl:template including ctl elements
Possible enhancement
Sampling method: More recent instances? Random?
Number of instance to be checked: All? More/Less than 100? ...
Also applies to A.2.1, A.2.2, A.3.1, A.3.2, A3.3 and A3.4 tests
Slide 17

18. A.1.7 Package availability
2 possible requests:
listExtensionPackages stored queries:
<csw:GetRecords …
outputSchema="urn:oasis:names:tc:ebxml-regrep:xsd:rim:3.0" version="2.0.2" service="CSW" resultType="results"
startPosition="1" maxRecords="10">
<rim:AdhocQuery id="urn:ogc:def:ebRIM-Query:OGC:listExtensionPackages">
</rim:AdhocQuery>
</csw:GetRecords>
EP are contained by the root registry package
<csw:GetRecordById … service="CSW" version="2.0.2"
outputSchema="urn:oasis:names:tc:ebxml-regrep:xsd:rim:3.0" 
<csw:Id>urn:ogc:def:ebRIM-RegistryPackage:OGC:Root</csw:Id>
<csw:ElementSetName typeNames="rim:RegistryPackage"> full</csw:ElementSetName>
</csw:GetRecordById>
No implementation supporting the root registry package
Limited support of the stored query by CNR-IMAA
Slide 18

19. A.1.7 Package availability
Improvement
Tests are based on CIM Package definition
Tests relate to:
@nodeType), …
Slide 19

20. A.1.8 ISO 19139 MD_Metadata.identificationInfo implementation
Verify that the ISO repository item contains only one instance of the MD_Metadata.identificationInfo property
Also verify that the ISO repository item contains at least one instance.
Slide 20

21. A.1.9 Mappings implementation
No generic mechanism found to complete an EP test toolbox
Reuse of CNR-IMAA tests
Also applies to A.2.3, A.2.4 and A.3.5 tests
Slide 21

22. Common Conformance Issues
Implementation of External Identifiers (Classification) as “independent” Registry objects instead of ExternalIdentifier (Classification) sub-element
Should a test be added specifically for External Identifiers and Classification?
Naming of properties and type
Case?
Differences between logical names (used in UML models) and implementations names (i.e. URI or identifiers in general)
Basic Package slot names are URL (i.e. from Dublin Core)
EO EP slot names are URN
CIM EP slot names are UpperCamelCase strings
Language type of Regrep 3 or LanguageTag type of Basic Package ?
No classification node for InternationalString and LocalizedString in Regrep 3 and Basic Package
Impossible to access the package definitions
No support of listExtensionPackage stored query
Root package does not exist
Slide 22

23. Conformance of ERDAS endpoint (http://projects-eu. ionicsoft
Instantiation of associations (A1.5 of CIM, …)
Association and object URNs are not those defined in CIM
urn:x-ogc:specification:csw-ebrim-cim:AssociationType:* instead of urn:ogc:def:ebRIM-AssociationType:OGC:*
2/10 association types are implemented
Instantiation of attributes, slots and classifications (A1.6 of CIM, …)
Implementation of External Identifiers (Classification) “independent” Registry objects instead of ExternalIdentifier (Classification) sub-element
Should a test be added specifically for External Identifiers and Classification?
Uppercase/lowercase issues in property names
Package availability (A1.7 of CIM, …)
Impossible to access the package definitions
No support of listExtensionPackage stored query
Root package does not exist
ISO MD_Metadata.identificationInfo implementation (A1.8 of CIM)
Repository items are attached to object type *:MetadataInformation while CIM states that « the only object types that are expected to be linked to repository items are : *:ServiceMetadata and *:DataMetadata. “
Slide 23

24. Package availability (A1.7 of CIM, …)
Conformance of CNR-IMAA endpoint (
Package availability (A1.7 of CIM, …)
Limited support of listExtensionPackage stored query
Catalogue seems to be empty
Slide 24

25. CIM Issues Slot type as Classification Scheme ? Slot type Slot name
#01 Format slot type defined as String in Table D20, but as FormatNameAndVersion in C.2.1 and Figure 9
#02 ConformsTo slot type defined as InternationString in Table D18, but as MetadataStandardNameAndVersion in C.2.1 and Figure 9 and Figure 16
Slot type
Slot type of Language is Language in CIM, but LangaugeTag in Basic Package
Slot type of Conformance is Boolean (Table 28 states that it is InternationalString)
Slot name
Slot name should be defined as a URI
Modified is named in Basic Package
Source is named in Basic Package
date is named in Basic Package
Language is named in Basic Package
date should be Date
Envelope Slot is not defined in CIM, neither in Basic Package
Slot type can’t be an ExtrinsicObject
Coverage Slot of DataMetadata should rather be an Association
Organization should described in C.2.6, since constraints related to the Name attribute are expressed in Figure 15
Slide 25

26. Basic Package Issues Slot type LanguageTag data type
Date Slot type does not exist in ebRIM
urn:oasis:names:tc:ebxml-regrep:DataType:Date should be urn:oasis:names:tc:ebxml-regrep:DataType:DateTime
LanguageTag data type
Is defined and used in BP instead of ebRIM Language data type
LanguageTag refers to RFC 4646 which supersedes RFC 3060 refered to by Language
RFC 4646 statement
“It is important to note that all language tags formed under the guidelines in this document were either legal, well-formed tags or could have been registered under RFC “
Is this type really necessary?
Slide 26

27. Teamengine Issues Lack of documentation
Installation of tomcat and configuration of teamengine sources for deployment
Debugging Tests (use of log files, ...)
Partial support of xsl:template
Named template including ctl elements are not processed
Slide 27

28. Feedback from CITE tests for 06-131 HMA-T Phase 2 AR-2 Meeting
16 july 2009, Frascati
Nicolas LESAGE, IGN
Slide 28

29. Overview Core conformance class : support of ‘EOP’ slots and objects
26 test cases
Verify:
registry package existence,
instantiations,
mappings, SOAP 1.2 and 1.1 bindings
and other various service capabilities
SAR extension : support of SAR slots
3 test cases
OPT extension : support of OPT slots
ATM extension : support of ATM slots and objects
4 test cases
Slide 29

30. Same question for verification of extrinsic objects
EO EP ATS issues
Should test cases concerning verification of associations be merged into one single test case?
CIM has only one test case
What is the expected level of granularity for requirements & tests?
Cardinality requirements on associations need to be clarified in EO EP spec
Same question for verification of extrinsic objects
Slide 30

31. EO EP Spec issues What namespace for the EO EP ? Data types
Proposed:
Data types
@slotType should use URNs for data types defined in ebRIM or in the Basic Package, not simple strings
E.g. urn:oasis:names:tc:ebxml-regrep:DataType:String instead of string
E.g. urn:ogc:def:dataType:ISO-19107:2003:GM_Object instead of geometry
EOProduct ExtrinsicObject
Contains different slots acc. to conformance level
I.e. EOP slots in the core, EOP + SAR slots in SAR extension
Another option would be to define a SAR EOProduct deriving from the base EOProduct
rim:externalIdentifier and rim:classification should be implemented as sub- elements of the rim/wrs:ExtrinsicObject, not as separate objects, to follow CSW ebRIM practices
URNs should be submitted to and approved by OGC Naming Authority
For next version?
Slide 31

32. A1.1 Support of SOAP 1.2 Test based on CNR-IMAA test
Issue : SOAP support advertisement in Capabilities document is not standardized
Capabilities document cannot be used to retrieve SOAP service URL  SOAP service URL must be input by user
Issue for OWS Common 1.2 SWG
E.g. service CNR-IMAA
ows:Constraint is a key-value structure  very open
Values for bindings (including versions) are not standardized
Slide 32

33. A1.2 Package avaibility
Purpose : verify that the canonical EO EP registry package is available and correct
Method: cf. related CIM test
Issues:
listExtensionPackages stored query only supported by CNR-IMAA service
Slide 33

34. A.1.3 Collection identifiers advertisement
Purpose: Verify that collection identifiers are advertised in the Capabilities
Method: GetCapabilities
Issues:
Collection id advertisement is not implemented in the tested services
in fact, DALI declares xmlns:gen0=" while the test, in application of the spec, looks for the ows prefix
Slide 34

35. A.1.4 EOProduct acquisition parameters instantiation
Purpose: Verify that all mandatory slots, classifications and associations of an EOProduct extrinsic object are correctly implemented
Method: cf. related CIM test
Structure of EOProduct (no inheritance, same object with different content at the different conformance levels) required to customize the tests originally designed as common to CIM and EO EP.
Issues
rim:ExternalIdentifier and rim:Classification should be instantiated as subelements of the rim/wrs:ExtrinsicObject
slotTypes should be URNs
Slide 35

36. Tests similar to A.1.4
A.1.9 EOProductAcquisitionPlatform instantiation
A.1.12 EOProductInformation instantiation
A.1.15 EOBrowseInformation instantiation
A.1.18 EOMaskInformation instantiation
A.1.21 EOArchivingInformation instantiation
A.2.1 SAR-specific slots instantiation
A.3.1 OPT-specific slots instantiation
A.4.2 EODataLayer instantiation
Slide 36

37. A.1.8 AcquiredBy association instantiation
Purpose: verify that the association is correctly implemented, i.e. that the role cardinalities are respected and the source and target objects really exist
Method: cf. CIM-related test
Issues/Warnings
Warnings are raised when optional associations are not instantiated
Slide 37

38. Tests similar to A.1.8
A.1.11 HasProductInformation association instantiation
A.1.14 HasBrowseInformation association instantiation
A.1.17 HasMaskInformation association instantiation
A.1.20 ArchivedIn association instantiation
A.4.1 HasDataLayer association instantiation
Slide 38

39. A.1.24 GetRepositoryItem id parameter
Purpose: Verify that the GetRepositoryItem operation works with the proper extrinsic object identifiers
Method: Cf. CIM-related test
Slide 39

40. Conformance of ESA HMA-E RSC Catalog endpoint
URL:
Object id: ESA.EECF.ENVISAT_ASA_GMI_1S:EN XG0
A.1.1 – SOAP 1.2 support
Not supported (not listed in the Capabilities document)
A.1.2 – Package avaibility
Cannot be tested because does not support listExtensionPackage stored query
A.1.3 – Collection identifiers advertisement
Not configured
A.1.4 – EOProduct acquisition parameters instantiation
Fails (external identifier is outside the extrinsic object definition)
A.1.9, A.1.12, A.1.15, A.1.21 (tests on objects and slots instantiations)
Pass. Warning are raised when optional slots are not instantiated.
Slide 41

41. Conformance of ESA HMA-E RSC Catalog endpoint
A.1.8, A.1.11, A.1.14, A.1.17, A.1.20 (association instantiation)
Pass
A EOMaskInformation instantiation
Fails, some mandatory slots are missing
Are these slots actually optional?
A.1.24 – GetRepositoryItem id parameter
Fails, no repository item linked to object ESA.EECF.ENVISAT_ASA_GMI_1S:EN XG0
Slide 42

42. Conformance of ESA HMA-E RSC Catalog endpoint
SAR extension
Tests fail because sampling method chooses a non-SAR item
Method should be modified to pick SAR items
OPT Extension
Is there any OPT object in the catalogue?
ATM Extension
Is there any ATM object in the catalogue?
Slide 43

43. Conformance of ESA Infeo2 Catalog endpoint
URL:
Not accessible
Slide 44

44. Conformance of Spot Image endpoint
URL:
Was not accessible as of 14 July (except GetCapabilities)
Seems to be in service since 15 July
Feedback to be provided to Spot Image
Slide 45

45. Conformance of CNR-IMAA endpoint
URL:
Object id: urn:ogc:def:EOP:ESA.EECF.ENVISAT_ASA_GMI_1S:EN XG0:EOProduct
A.1.1 – SOAP 1.2 support
Endpoint listed in the capabilities but test fails
Test must be checked
A.1.2 – Package avaibility
listExtensionPackages supported
@parent attribute missing on classification nodes in EO EP XML file
Specific associations (defined in order to test the source and target objects of associations) are not part of CNR-IMAA canonical registry package. Is it the intended behaviour?
A.1.3 – Collection identifiers advertisement
Pass, one collection is advertised
A.1.4 – EOProduct acquisition parameters instantiation
Fails because GetRecords request does not match any record
A.1.9, A.1.12, A.1.15, A.1.21 (tests on objects and slots instantiations)*
Slide 46

46. Conformance of CNR-IMAA endpoint
A.1.8, A.1.11, A.1.14, A.1.17, A.1.20 (association instantiation)
Fails, no record matched
A EOMaskInformation instantiation
Fails, objects & slots missing
A.1.24 – GetRepositoryItem id parameter
Fails, no repository item linked to object urn:ogc:def:EOP:ESA.EECF.ENVISAT_ASA_GMI_1S:EN XG0:EOProduct
Slide 47

47. Conformance of VITO endpoint
Catalogue seems to be empty
SAR extension
Is there any SAR object in the catalogue?
OPT Extension
Is there any OPT object in the catalogue?
ATM Extension
Is there any ATM object in the catalogue?
Slide 48

48. Conformance of VITO endpoint
URL:
Object id: vito.vgt4africa.vpi:VGT4AFRICA_VPI_ _Africa
A.1.1 – SOAP 1.2 support
Not supported (not listed in the Capabilities document)
A.1.2 – Package avaibility
Cannot be tested because does not support listExtensionPackage stored query
A.1.3 – Collection identifiers advertisement
Not configured
A.1.4 – EOProduct acquisition parameters instantiation
Passes (external identifiers and classifications are inside the extrinsic object definition)
A.1.9, A.1.12, A.1.15, A.1.21 (tests on objects and slots instantiations)
Pass. Warning are raised when optional slots are not instantiated.
Slide 49

49. Conformance of VITO endpoint
A.1.8, A.1.11, A.1.14, A.1.17, A.1.20 (association instantiation)
Pass
A EOMaskInformation instantiation
Fails, some mandatory slots are missing
Are these slots actually optional?
A.1.24 – GetRepositoryItem id parameter
Fails, no repository item linked to object vito.vgt4africa.vpi:VGT4AFRICA_VPI_ _Africa
Slide 50

50. Conformance of VITO endpoint
It seems that only VGT products are described in the catalogue
SAR extension
Is there any SAR object in the catalogue?
OPT Extension
Is there any OPT object in the catalogue?
ATM Extension
Is there any ATM object in the catalogue?
Slide 51