1. Flight-like Flight Software Testing Methods Celina A Garcia Jet Propulsion Laboratory California Institute of Technology ©2009 California Institute of Technology. Government sponsorship acknowledged.

2. Introduction  Flight Software (FSW) testing methods used by Mars Science Laboratory (MSL) –Not a new concept, many contributors  What is “flight-like” testing? –Same means & methods used in Mission Operations –Use flight scenarios similar to those defined by Systems to test FSW Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 11/5/20092

3. Previous FSW Test Methods  Past experience with FSW test methods –FSW Developer: white box testing –FSW Build & Integration: minimal gray box testing –Systems Engineering: black box testing  Normal S/W development, the schedule can absorb any changes due to design flaws  Tight S/W schedules can’t afford to catch design flaws during Systems Eng. & ATLO (Assembly Test & Launch Ops) testing. Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 11/5/20093

4. Issue  FSW schedule is impacted –Very tight schedule –Unscheduled point releases due to problems found during systems tests  Causes: –FSW team testing did not necessarily include sending commands and verifying telemetry –Dynamic design and/or poor interpretation of requirements & functionality Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 11/5/20094

5. Solution  Formation of MSL FSW internal test (FIT) team –Objective: Use gray box testing approach to discover FSW problems before Systems Eng. testing does –Employ same interfaces to S/C as Systems Eng. –Use FSW knowledge to test & stress FSW in ways Systems Eng. may not FSW Event Reporting throttling Stressing uplink & downlink  Results in fewer point builds & ensures a more stable FSW product Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 11/5/20095

6. FIT Approach  Gray box testing –“Test as you fly and fly as you test” –Mix of black & white box testing –Going beyond traditional unit testing –Testing against FSW requirements & functionality described in Functional Description Documents –Ensure that FSW works how Systems Eng. expects it to work –Make use of Ground S/W tools scripting interface 11/5/20096 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods

7. FIT Approach (cont.) –Test using same venues as Systems Eng. Workstation Testset (WSTS) –simulated avionics with closed loop simulation for Attitude Control Sys. & Entry, Descent, & Landing Testbed –combination of simulated and real avionics with closed loop simulation as well 11/5/20097 Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods

8. Flight-like FSW Testing Process  FIT testers create functional tests by: –Reviewing Functional Description Documents (FDDs) and FSW design docs –Consulting FSW developer and Systems Engineer –Test script design & test results are reviewed & approved –Testing done using Workstation Simulation Testset and/or Testbed Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 11/5/20098

9. Initial Results  In progress  Short term benefits: –FSW integration is an easier task near code freeze and release –FSW has access to the FIT test suite –FIT can act as a liaison between Systems, Ground S/W and FSW developer –Provide early feedback to Ground & Simulation S/W –Functional tests become means for regression testing Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 11/5/20099

10. Contact Info  E-mail: Celina.A.Garcia@jpl.nasa.govCelina.A.Garcia@jpl.nasa.gov  MSL FSW Build & Integration Lead & FIT team member Celina A Garcia, Nov 2009, Flight-like FSW Testing Methods 11/5/200910