1. Test and Integration Robyn Allsman LSST Corp DC3 Applications Design Workshop IPAC August 21 - 22, 2008

2. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 2 LSST Software Quality Control Will provide the user community with the confidence that the telescope system and its output is –scientifically accurate, –technically reliable, and –supports the mandated features. Data Management SQC uses a two-pronged approach –Validating the design –Validating the design's implementation

3. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 3 Validating Design thru Traceability Requirements Use Cases Preliminary Design Detailed Design Codin g Unit Testing Integration Testing System Testing Acceptance Testing Based on: “Use Case Driven Object Modelling with UML”, Rosenberg&Stephens, Figure 12-1 Robustness diagram & Refined Domain Model Sequence Diagram & Logical/Class Model Domain Model

4. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 4 Validating Design thru Prototyping DM specified four Data Challenges whose deliverables –Validate the ability to achieve the projected computational and resource load; and –Validate new functionality.

5. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 5 Validating Implementation thru Testing DM uses a bottom-up testing method where –Validated objects are tested with, then added to, a validated baseline which in turn is used as the new validated baseline for further iterative testing.

6. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 6 Flavors of Validation Testing Acceptance Testing Unit Testing Observatory Integration Testing System (behavioral) Testing DMOCS NightlyDay Subsystem Integration DAQ LSSTO

7. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 7 1 st Level: Unit Testing Validates an object works according to detailed design spec Unit Test scenarios taken from either the logical (or class) model or the sequence diagram. Uses white-box testing of internal conditions known only to the developer: boundaries, exceptions, functional accuracy, etc. Uses black-box testing of the parameter space and API specification. Tests object in isolation by using test harness to simulate external interfaces.

8. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 8 Next Level: Integration Testing Validates interfaces between unit-tested modules using black box testing. Tests interfaces between design objects. Test scenarios are derived from the domain model or from the robustness diagram. LSST uses two subtypes: Subsystem & Observatory Integrations

9. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 9 DM Subsystem Integration Image Processing Detection Association Alert NightMOPS DayMOPS Day Pipelines Subsystem Night Pipelines Subsystem Validates interfaces btwn major components in same admin area Calibration Subsystem Data Release Subystem

10. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 10 Observatory Integration Validates Interfaces between Administrative areas OCS NightPostStage B:NightPreStage B:NightPipelines A:DayPipelines A:Databases Databases Telescope/Camera B:Databases Data Management.....

11. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 11 Penultimate Level: System Tests –Validate the end-to-end compliance with project behavioral requirements such as Reliability, Capacity, Speed, Security, Failsafe Recovery, Acceptable Failure Modes, Grace under Stress, etc. –Occur after all Observatory subsystems are integrated. –Test Scenarios are derived from System Requirements and Use Cases.

12. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 12 Final Check: Acceptance Testing Validate that the system works according to the User Requirements. Test scenarios are derived from the use cases.

13. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 13 Generic Test Scenario Specify... test type determines UML product to use: Integration: test connections between objs in domain model Unit Test: test the classes in the logical model Implement & Test... implement test suite as specified above Analyse Coverage... did test exercise enough code? Analyse Performance... robustness, reliability, speed, capacity Check Outputs... did results correspond to input? (Save Results... SQA tracking of validated components)

14. DC3 Applications Design Workshop IPAC August 21 - 22, 2008 14 Preparing for the PDR The EA models and diagrams of the LSST DM Reference Design will be used to generate the Integration, System and Acceptance test specifications. The Integration and Test timeline will follow....