HMA Follow On Task 4 - Order Final Presentation 20-21 June 2011 Daniele Marchionni TELESPAZIO Uwe Voges conterra June 2011 – Slide 1

Migration to the updated Ordering ICD Overview Demo Task 4 – Agenda TASK4 Objectives TASK4 Achievements Future Work Migration to the updated Ordering ICD Overview Demo June 2011 – Slide 2

Promotion of Ordering ICD as OGC Standard; Task 4 – Objectives Task 4 Objectives: Promotion of Ordering ICD as OGC Standard; Implementation of a Reference Order Server; Implementation of Executable Test Suite. June 2011 – Slide 3

Task 4 – Achievements - Deliverables TASK 4 Deliverables: OPGW Software Requirements Specification : OPGW SRS V1.2 28/01/2011 OPGW Software Design Document: OPGW SDD V1.2 28/01/2011 OPGW Software Validation Testing Specification: OPGW SVS V1.2 13/06/2011 OPGW Acceptance Test Reports: OPGW ATR V1.1 13/06/2011 OPGW Software Installation Document: OPGW SID V1.2 12/06/2011 June 2011 – Slide 4

Task 4 – Achievements - Deliverables OPGW V5.1 Software: OPGW_V5.1_source_kit.tar.gz OPGW_V5.1_installation_kit.tar.gz OPGW_V5.1_COTS_kit.tar.gz OPGW Support Tools: Software: MEOPC_V1.0_installation_kit.tar.gz MOHS_V1.0_installation_kit.tar.gz SPSEO_V1.0_installation_kit.tar.gz OGL_V1.0_installation_kit.tar.gz OPGW Software Installation Document: OPGW Tools SID V1.0 June 2011 – Slide 5

Task 4 – Achievements - Deliverables HMA Ordering ICD (including ATS): Ordering Services for Earth Observation Products OGC 06-141r6 V1.0.0 22/04/2011 Ordering Services for Earth Observation Products - Executable Test Suite (ETS) – (Technical Note) Ordering Services for Earth Observation Products - Executable Test Suite V1.0 15/06/2011 Ordering Services for Earth Observation Products - Executable Test Suite (ETS) – (CTL Scripts) ETS_V1.0.zip ETS Execution Report Executable Test Suite Report V1.0 14/06/2011 June 2011 – Slide 6

TASK 4 – Achievements - Ordering ICD Created “Ordering Services for Earth Observation Products SWG” (formally activated on 04/12/2009) Chair: Daniele Marchionni Co-chair: Uwe Voges Currently 10 members, 16 observers Ordering Services submitted for Public Comments from 24/01/2011 to 23/02/2011 Comments received and document updated for final adoption vote: Ordering Services for Earth Observation Products OGC 06-141r6 V1.0.0 22/04/2011 Voting page (03/05/2011 to 02/07/2011) (http://portal.opengeospatial.org/?m=projects&a=view&project_id=82&tab=5). Currently 17 Yes and 1 No votes. June 2011 – Slide 7

Ordering ICD (OGC 06-141r6) - Overview Ordering ICD Overview: Provides operations for: Submitting order requests for the following type of orders: Orders for precisely identified EO products Orders for future EO products, derived from the interaction with an SPS EO compliant server Subscription to EO Products: to support bulk orders and subscriptions Discovering of supported collections Discovering of processing, delivery and scene selection options Order quotation SWE Common for modeling order options June 2011 – Slide 8

Ordering ICD (OGC 06-141r6) - Overview Ordering ICD Overview: Operations: GetCapabilities GetOptions GetQuotation & GetQuotationResponse Submit & SubmitResponse GetStatus Cancel & CancelResponse DescribeResultAccess June 2011 – Slide 9

Ordering ICD (OGC 06-141r6) – Requirement Classes Because Ordering ICD supports different types of orders; with different modes: sync / async; with different delivery methods (media / online); and different Service Providers are interested in few combinations only; then several conformance classes have been defined regrouping the most sensible combinations: June 2011 – Slide 10

Ordering ICD (OGC 06-141r6) – Requirement Classes June 2011 – Slide 11

Ordering ICD (OGC 06-141r6) – Conformance Classes Full compliance is very difficult to achieve, possible examples of compliance are: An Order Server supporting product ordering for scientific users should implement: Core MediaDelivery ProductOrder SceneSelection Optionally it should support also on-line delivery: OnlineDataAccess OnlineDataDelivery June 2011 – Slide 12

Ordering ICD (OGC 06-141r6) – Conformance Classes An Order Server supporting product ordering for commercial users should implement: Core MediaDelivery ProductOrder SceneSelection Quotation, and at least one of: QuotationAsync QuotationMonitoring QuotationOffLine QuotationSync Optionally it should support also on-line delivery: OnlineDataAccess OnlineDataDelivery June 2011 – Slide 13

Ordering ICD (OGC 06-141r6) – Requirement Classes Statistics: Requirement Class Number of requirements AsyncCancellation 7 Quotation 8 AsyncSubmit QuotationAsync 5 Cancellation QuotationMonitoring 4 Core 40 QuotationOffLine 3 MediaDelivery 2 QuotationSync Notification 1 SceneSelection OnlineDataAccess 12 SubscriptionOrder 6 OnlineDataDelivery TaskingOrder ProductOrder TOTAL 126 June 2011 – Slide 14

Ordering ICD (OGC 06-141r6) – Conformance Classes 1:1 mapping between Conformance Classes and Requirements Classes (as required by OGC policies): June 2011 – Slide 15

Ordering ICD (OGC 06-141r6) – Conformance Classes Each Conformance Class is composed of a set of tests, each verifying one or more requirements of the corresponding Requirements Class. Each Conformance Class covers all requirements of the corresponding Requirements Class. To be noted that the tests reported in the Conformance classes have “temporal dependencies” (e.g. to test order status at least one order needs to be created in the Order Server). Then: The tests specified in a Conformance Class must be executed in the order they are specified in the document; The tests of a Conformance Class can be started only if the tests of the parent class have been completed. June 2011 – Slide 16

Ordering ICD (OGC 06-141r6) – Conformance Classes Statistics: Conformance Class Number of tests AsyncCancellation 4 Quotation 5 AsyncSubmit QuotationAsync 2 Cancellation QuotationMonitoring 3 Core 13 QuotationOffLine 1 MediaDelivery QuotationSync Notification SceneSelection OnlineDataAccess 6 SubscriptionOrder OnlineDataDelivery TaskingOrder ProductOrder TOTAL 70 June 2011 – Slide 17

Ordering ICD (OGC 06-141r6) Evolution Summary of changes w.r.t. V0.9.4 (current operational baseline): Added comprehensive list of processing options (inputs from ESA, EUMETSAT, CNES, MDA, DLR) Added comprehensive list of scene selection options (inputs from ESA, EUMETSAT, CNES, MDA, DLR) Implemented many comments received from: conterra (TASK4 consortium), EUMETSAT, ESA (HMA-E and DAIL implementation projects), DLR Enhanced Contents section of Capabilities document (to allow clients to discover which collections, which functions, which mechanisms are supported by the server with no pre-configuration) Processing options at order level and order item level Delivery options at order level and order item level June 2011 – Slide 18

Ordering ICD (OGC 06-141r6) Evolution Enhanced Description of Scene Selection Options (one of the major weaknesses of the previous V0.9.4). Now the client can set scene selection options with minimal / no pre-configuration Order option model updated to SWE Common 2.0 Improved definition of delivery options Added extensibility of the specification: xs:any added at order level, order item level and as search criterion Document reformatted according to updated OGC policy and procedures June 2011 – Slide 19

TASK 4 – Achievements - Reference Order Server A reference order server has been set-up with the following objectives: Demonstrate the updated specification; Support the implementation of ordering clients; Support the implementation of ordering servers. The implemented order server – OPGW - provides an interface compliant with Ordering V1.0.0; the ordering function has been accomplished by translating the incoming requests in EOLI XML calls to the back-end server. It is meant to be used with M2EOS. In order to have a stand-alone installation of OPGW a number of simulators have been implemented: MOHS (Micro Order Handling System): simulator of EOLI XML Order server; SPS EO (Sensor Planning Service – Earth Observation): simulator of SPSEO server; MEOPC (Micro Earth Observation Product Catalogue): simulator of EOLI XML Catalogue Server; June 2011 – Slide 20

Reference Order Server Architecture OPGW Test Tool: CLI Java application provided with OPGW kit, in charge of automatic execution of OPGW validation tests. It calls all OSEO operations including also a valid SAML Token (pre-configured). OPGW: Web Service implementing Ordering ICD (all operations). To implement Submit, GetStatus, Cancel operations it calls corresponding operations of MOHS. To implement the SPS join scenario it calls SPSEO GetStatus operation; It does decryption and digital signature verification on incoming SAML Tokens June 2011 – Slide 21

Reference Order Server Architecture OPGW Environment: Developed in Java (JDK 1.6.0.23) Deployed within TOMCAT (6.0.32) Web Service Layer developed using AXIS2 (1.5.4) + XMLBEANS (2.4.0) data binding Light weight Java Database: HyperSQL V2 Netbeans V6.9.1 IDE OPGW is available as open source (http://sourceforge.net/projects/opgw/files/OPGW/5.1/) June 2011 – Slide 22

OPGW Support Tools Micro Order Handling System – MOHS Environment Web service implementing EOLI XML Order protocol: processAcquisitionOrderRequest processProductOrderRequest processOrderMonitorRequest Environment Developed in Java (JDK 1.6.0.23) Deployed within TOMCAT (6.0.32) Web Service Layer developed using METRO (JAX-WS + JAXB) Light weight Java Database: HyperSQL V2 Netbeans V6.9.1 IDE June 2011 – Slide 23

OPGW Support Tools SPSEO : Simulator of SPSEO protocol; implements the following operations: GetFeasibility, Submit, GetStatus Environment: Developed in Java (JDK 1.6.0.23) Deployed within TOMCAT (6.0.32) Web Service Layer developed using AXIS2 (1.5.4) + XMLBEANS (2.4.0) data binding Light weight Java Database: HyperSQL V2 Netbeans V6.9.1 IDE June 2011 – Slide 24

OPGW Support Tools OPGW GUI Client - OGL: graphical client for testing the following operations: GetOptions Submit DescribeResultAccess Used for demonstrating the updated Ordering Specification. Environment: Developed in Java (JDK 1.6.0.23) Web Application Deployed within TOMCAT (6.0.32) STRUTS V2.0 framework for Web Pages generation Netbeans V6.9.1 IDE June 2011 – Slide 25

TASK 4 – Achievements - ETS Ordering Executable Test Suite (ETS): “Implements” the test cases defined within the Conformance Classes “Implementations” are defined as Compliance Test Language (CTL) Scripts Baseline doc: Compliance Test Language (CTL), OGC® Best Practices Document, OGC 06-126r4, Version 0.8, 2010-11-23 For each Conformance Class a dedicated CTL script has been prepared e.g.: ETS_CC_OSEO_Core.ctl ETS_CC_OSEO_Cancellation.ctl ETS_CC_OSEO_MediaDelivery.ctl ETS_CC_OSEO_OnlineDataDelivery.ctl ... Tests within a Conformance Class are implemented within dedicated CTL scripts, one for each operation e.g.: ETS_CC_OSEO_Core_GetCapabilities.ctl ETS_CC_OSEO_Core_GetOptions.ctl ETS_CC_OSEO_Core_GetStatus.ctl June 2011 – Slide 26

TASK 4 – Achievements - ETS All tests are started from ETS.ctl CTL script ETS.ctl: includes main .ctl-scripts of Conformance Classes defines the variables provided to the test scripts calls the main test scripts of the Conformance Classes Some test implement a request-chain as some requests must be processed (e.g. submit) to test another one (e.g. getStatus) June 2011 – Slide 27

TASK 4 – Achievements - ETS Test messages are outside the CTL scripts, unless they are generated on the fly re-using results from previous tests. If a test needs a test message, it is named with this rule: <TEST_ID prefix>_<Operation name>.xml e.g. for Core Conformance Class (§A.3.1) we have: A_3_01_01_GetCapabilities.xml A_3_01_02_GetCapabilities.xml A_3_01_03_GetOptions.xml A_3_01_04_01_GetOptions.xml A_3_01_04_02_GetOptions.xml .... ctl scripts are in the \ETS dir the request messages are in the following path: ETS\messages\requests June 2011 – Slide 28

TASK 4 – Achievements - ETS Currently UserManagement is not considered in the ETS For running the tests different preparatory steps are necessary: Setting up the TEAMEngine Adaptation of the request messages (in \ETS\messages) for the following operations to consider the IUT-specific collections/products and their related options/sceneSelections: Submit GetOptions GetStatus GetQuotation Adapt different parameters in ets.ctl (in \ETS) All tests defined in the ATS have been implemented and tested against the Reference Order Server June 2011 – Slide 29

TASK 4 – Achievements – OPGW Compliance OPGW Compliance w.r.t. ETS: Conformance Class Core (13 tests): PASSED Conformance Class ProductOrder (6 tests): PASSED Conformance Class SceneSelection (4 tests): PASSED Conformance Class SubscriptionOrder (5 tests): FAILED Conformance Class TaskingOrder (5 tests): PASSED Conformance Class AsyncSubmit: NOT IMPLEMENTED Conformance Class Quotation (5 tests): PASSED Conformance Class QuotationSync (2 tests): PASSED Conformance Class QuotationMonitoring: (3 tests): PASSED Conformance Class QuotationAsync: NOT IMPLEMENTED Conformance Class QuotationOffLine: NOT IMPLEMENTED June 2011 – Slide 30

TASK 4 – Achievements – OPGW Compliance Conformance Class OnlineDataAccess (6 tests): PASSED Conformance Class OnlineDataDelivery (2 tests): PASSED Conformance Class MediaDelivery (2 tests): PASSED Conformance Class Cancellation (4 tests): PASSED Conformance Class AsyncCancellation: NOT IMPLEMENTED Conformance Class Notification (1 test): FAILED June 2011 – Slide 31

TASK 4 – Achievements – Future Work ETS: To implement asynchronous tests (TEAM Engine issue) To test User Management June 2011 – Slide 32

Migration to updated Ordering ICD Ordering ICD V1.0.0 is not backward compatible with V0.9.4 (changed namespaces, updated tags, etc.) However the operations have not changed and their basic function is very similar OPGW V5.1, implementing Ordering ICD V1.0.0, is freely available and can be re-used for implementing the backbone of any Order Server Then no major issues to migrate to the updated protocol June 2011 – Slide 33

TASK 4 Demo Overview Put in “shop cart” the items to order (derived from SPSEO interaction) June 2011 – Slide 34

TASK 4 Demo Overview Select Order Options June 2011 – Slide 35

TASK 4 Demo Overview Set options values: June 2011 – Slide 36

TASK 4 Demo Overview Prepare order June 2011 – Slide 37

TASK 4 Demo Overview Submit order June 2011 – Slide 38

TASK 4 Demo Overview Get URL: June 2011 – Slide 39

TASK 4 Demo Overview Access the Online Product: June 2011 – Slide 40