1. Table of Contents Program Model Checking: Case Studies and Practitioner’s Guide John Penix, ARC Howard Hu, JSC

2. Program Model Checking Case Studies and Practitioner’s Guide Problem ► Research has shown that program model checking and be effective at detecting critical software errors that are difficult to detect via testing. ► However, applying model checking remains a “black art”. Best practices are only just emerging and remain a relatively ad-hoc combination of methods.

3. Program Model Checking Case Studies and Practitioner’s Guide Approach ► Assemble the emerging best practices in program model checking ► Demonstrate and validate their use in several case studies ► Document the results in a Practitioner’s Guide for Program Model Checking

4. Program Model Checking Case Studies and Practitioner’s Guide Importance and Benefits The guidebook will provide: ► Methods for formalizing requirements and identifying critical properties – may help a test team develop verification goals for V&V or IV&V ► Test driver development and test coverage assessment methods to support testing or model checking ► Methods for improving the verifiability of designs ► Guidance in configuring model checking options and organizing and validating model checking results

5. Program Model Checking Case Studies and Practitioner’s Guide Importance and Benefits Improves testing: ▀ Ability to control thread scheduling and environment responses ▀ Stress test critical software states ▀ Directed search for specific errors: deadlock, race conditions, assertion violations, … Testing covers one path Model checking searches all paths

6. Program Model Checking Case Studies and Practitioner’s Guide Relevance to NASA Shuttle Abort Flight Manager (SAFM) Provides onboard abort performance assessment during powered flight and landing site evaluation and monitoring during glided flight in Cockpit Avionics Upgrade. 30KLOC in C++. Initial Case Study Application

7. Program Model Checking Case Studies and Practitioner’s Guide Accomplishments ► Kick-off meeting at JSC with overview of SAFM ► Delivery of requirements document, design document, source code and test infrastructure from JSC to ARC ► Evaluation of SAFM source code and requirements for applicability to model checking & identify critical issues ► Hosted SAFM test lead at ARC for a week and to elicit requirements and design properties that are currently unchecked. ► Identified Sequencer as a critical subsystem ► Obtained the SAFM test system, requirements simulator, and test data from the SAFM development team. ► Set up SAFM build & test environment at ARC ► Gathered data on existing test coverage

8. Program Model Checking Case Studies and Practitioner’s Guide Next Steps ► Assessment of critical SAFM properties and current test coverage ► Evaluate use of property patterns to formalize critical SAFM requirements Future Year Deliverables and MilestonesDue Application/test environment modifications planned03-2005 Application/test environment modifications complete12-2005 Draft guidebook03-2006 Assessment of modified application test coverage06-2006 Guidebook12-2006