Table of Contents Program Model Checking: Case Studies and Practitioner’s Guide John Penix, ARC Howard Hu, JSC

Program Model Checking Case Studies and Practitioner’s Guide Problem ► Research has shown that program model checking and be effective at detecting critical software errors that are difficult to detect via testing. ► However, applying model checking remains a “black art”. Best practices are only just emerging and remain a relatively ad-hoc combination of methods.

Program Model Checking Case Studies and Practitioner’s Guide Approach ► Assemble the emerging best practices in program model checking ► Demonstrate and validate their use in several case studies ► Document the results in a Practitioner’s Guide for Program Model Checking

Program Model Checking Case Studies and Practitioner’s Guide Importance and Benefits The guidebook will provide: ► Methods for formalizing requirements and identifying critical properties – may help a test team develop verification goals for V&V or IV&V ► Test driver development and test coverage assessment methods to support testing or model checking ► Methods for improving the verifiability of designs ► Guidance in configuring model checking options and organizing and validating model checking results

Program Model Checking Case Studies and Practitioner’s Guide Importance and Benefits Improves testing: ▀ Ability to control thread scheduling and environment responses ▀ Stress test critical software states ▀ Directed search for specific errors: deadlock, race conditions, assertion violations, … Testing covers one path Model checking searches all paths

Program Model Checking Case Studies and Practitioner’s Guide Relevance to NASA Shuttle Abort Flight Manager (SAFM) Provides onboard abort performance assessment during powered flight and landing site evaluation and monitoring during glided flight in Cockpit Avionics Upgrade. 30KLOC in C++. Initial Case Study Application

Program Model Checking Case Studies and Practitioner’s Guide Accomplishments ► Kick-off meeting at JSC with overview of SAFM ► Delivery of requirements document, design document, source code and test infrastructure from JSC to ARC ► Evaluation of SAFM source code and requirements for applicability to model checking & identify critical issues ► Hosted SAFM test lead at ARC for a week and to elicit requirements and design properties that are currently unchecked. ► Identified Sequencer as a critical subsystem ► Obtained the SAFM test system, requirements simulator, and test data from the SAFM development team. ► Set up SAFM build & test environment at ARC ► Gathered data on existing test coverage

Program Model Checking Case Studies and Practitioner’s Guide Next Steps ► Assessment of critical SAFM properties and current test coverage ► Evaluate use of property patterns to formalize critical SAFM requirements Future Year Deliverables and MilestonesDue Application/test environment modifications planned Application/test environment modifications complete Draft guidebook Assessment of modified application test coverage Guidebook