Formal Model-Based Development in Aerospace Systems: Challenges to Adoption Mats P. E. Heimdahl University of Minnesota Software Engineering Center Critical Systems Research Group Department of Computer Science and Engineering University of Minnesota

and a Plea for Help

Domain of Concern

How we Develop Software Concept Formation Requirements Specification Design Implementation Integration System Unit Test Integration Test System Test Object Code Test Analysis

Model-Based Development Specification Model Visualization Prototyping Testing Code Analysis Properties

Model-Based Development Tools Commercial Products –Esterel Studio and SCADE Studio from Esterel Technologies –Rhapsody from I-Logix –Simulink and Stateflow from Mathworks Inc. –Rose Real-Time from Rational –Etc. Etc.

System Specification/Model How we Will Develop Software Concept Formation Requirements Implementation Integration Properties Analysi s Integration Test Syste m Test Specification Test

What Does Industry Want? Better / Safer Cheaper Faster

Model-Based Development Examples

Problem 1 Believing Testing Can be Eliminated Testing will always be a crucial (and costly) component

How we Develop Software Concept Formation Requirements Specification Design Implementation Integration System Unit Test Integration Test System Test Analysis Object Code Test

System Specification/Model Testing Does not go Away Concept Formation Requirements Implementation Integration Properties Extensive Testing (MC/DC)

System Specification/Model It Simply Moves Concept Formation Requirements Implementation Integration Properties Extensive Testing (MC/DC)

System Specification/Model Do it the Right Way Concept Formation Requirements Implementation Integration Properties Analysi s Integration Test Syste m Test Specification Test Unit Test

Example: ADGS-2100 Adaptive Display & Guidance System Requirement Drive the Maximum Number of Display Units Given the Available Graphics Processors Counterexample Found in 5 Seconds! Checking 573 Properties Found 98 Errors 883 Subsystems 9,772 Simulink Blocks 2.9 x Reachable States

Remedy Be honest about the capabilities of model- based development and formal methods –Done right, provides outstanding requirements, models, analysis, etc., etc. –May greatly reduce the effort spent in testing

Problem 2 Believing the Model is Everything The model is never enough

Modeling is so much fun Properties Specification/Model Modeling Frenzy Concept Formation Requirements Implementation Integration How do we know the model is “right”? Headfirst into modeling System

Specification/Model Do it the Right Way Concept Formation Requirements Implementation Integration Properties Analysi s Integration Test Syste m Test Specification Test Unit Test

Remedies Recognize the Role of Software Requirements –The model is not everything Development Methods for Model-Based Development Badly Needed –Model-Based Software Development Process Develop Tools and Techniques for Model, Properties, and Requirements Management Develop Inspection Checklists and Style Guidelines for Models

Problem 3 Trusting Verification To really mess things up, you need formal verification

Model Checking Process Does the system have property X? Model Engineer SMV Automatic Translation SMV Properties Properties Automated Check Yes! SMV Spec. Automatic Translation

Model Checking Process Does the system have property X? Model Engineer SMV Automatic Translation SMV Properties Properties SMV Spec. Automatic Translation Counter Example Automated Check No!

Property or Model: Who is Right? AG(Onside_FD_On -> Mode_Annunciations_On) The Mode Annunciations shall be turned on when the Flight Director is turned on AG( (Is_This_Side_Active & Onside_FD_On) -> Mode_Annunciations_On) If this side is active, the Mode Annunciations shall be turned on when the Flight Director is turned on If this side is active and the Mode Annunciations are off, the Mode Annunciations shall be turned on when the Flight Director is turned on AG( ! Mode_Annunciations_On -> AX ((Is_This_Side_Active & Onside_FD_On) -> Mode_Annunciations_On)))

Translated All the “Shalls” into SMV Properties

Analysis Process Steps All properties verified (!), or… Counterexamples found for some properties Simulate counterexample in MBD environment and make corrections to: –model –properties –requirements –assumptions (invariants)

Remedies Develop techniques to determine adequacy of model and property set –How do we know they are any “good” Techniques for management of invariants –How do we validate the assumptions we make Methodology and guidance badly needed –Tools with training wheels –“Verification for Dummies” All we need is one high-profile verified system to fail spectacularly to set us back a decade or more

Model Checking Process Why? Guru Does the system have property X? Model Engineer SMV Automatic Translation SMV Properties Properties SMV Spec. Automatic Translation Out to Lunch ?

Problem 4 Believing One Tool Will Be Enough To be effective, we need a suite of notations and analysis tools (and the ability to continually integrate new ones)

Original Tool Chain RSML -e NuSMV Model Checker PVS Theorem Prover Rockwell Collins/U of Minnesota SRI International RSML -e to NuSMV Translator RSML -e to PVS Translator

Conversion to SCADE Design Verifier SCADE Lustre NuSMV PVS Safe State Machines Simulink Gateway StateFlow SPY Esterel Technologies MathWorks University of Minnesota/Rockwell Collins (NASA LaRC Funded) University of Minnesota (NASA IV&V Funded)

Reactive Systems Esterel Technologies MathWorks SRI International University of Minnesota/Rockwell Collins (NASA LaRC) University of Minnesota (NASA IV&V) Current(?) Tool Status Design Verifier SCADE Lustre NuSMV PVS Safe State Machines SAL ICS Symbolic Model Checker Bounded Model Checker Infinite Model Checker Simulink Gateway StateFlow Reactis SPY

Three Conjectures No one modeling language will be universally accepted, nor universally applicable No one verification/validation tool will satisfy the analysis needs of a user Languages and tools must be tested on real world problems by practicing engineers –Preferably in commercial tools

Translation – with no IL Effort = m * n High quality translations Lustre ++ poly tables SCADE RSML -e PVS poly’ SMVC m modeling languages n target languages poly

Translation – with IL Effort = m + n Low quality translations Lustre IL Lustre ++ poly tables SCADE RSML -e PVS poly’ SMVC m modeling languages n target languages poly

A Proposed Framework (Van Wyk) Based on techniques from extensible programming languages, specifically attribute grammars extended with forwarding. Hypothesis: –An extensible language may serve as a host language for domain specific extensions (to construct new modeling languages), –while forwarding enables the feasible construction of high quality translations from source specification languages to target analysis languages. Provided to spur discussion only! There may be better solutions.

Translation – with lang. exts. Effort = m + n + Σ t I High quality translations Lustre Host Lustre ++ poly tables SCADE RSML -e PVS poly’ SMVC m modeling languages n target languages forwarding poly pvs_trans (t2) pvs_trans (t1) c_trans (t3) forwarding c_trans smv_trans pvs_trans

Remedies Next generation tools must allow easy extension and modification of notations to meet domain specific needs They must allow easy construction of high- quality translations from modeling notations to analysis tools They also must enable controlled reuse of tool infrastructure to make tool extensions cost effective

Problem Summary Believing Testing Can be Eliminated Believing the Model is Everything Trusting Verification Believing One Tool Will Be Enough

Thank You Rockwell Collins –Steven Miller –Michael Whalen –Alan Tribble –Michael Peterson NASA Langley –Ricky Butler –Kelly Hayhurst –Celeste Bellcastro NASA Ames –Michael Lowry NASA IV&V Facility –Kurt Woodham (L3-Titan) My Students at Minnesota –Anjali Joshi –Ajitha Rajan –Yunja Choi, –Sanjai Rayadurgam –Devaraj George –Dan O'Brien Opinions in talk are mine. Do not blame the innocent.

Discussion

For More Information Michael W. Whalen et. al., Formal Validation of Avionics Software in a Model- Based Development Process, Formal Methods in Industrial Critical Systems (FMICS’2007), July Steven P. Miller, Alan C. Tribble, Michael W. Whalen, Mats P. E. Heimdahl, Providing the Shalls, International Journal on Software Tools for Technology Transfer (STTT), Feb Michael W. Whalen, John D. Innis, Steven P. Miller, and Lucas G. Wagner, ADGS-2100 Adaptive Display & Guidance System, NASA Contractor Report NASA-2006-CR213952, Feb Available at A lot of good reading at Eric Van Wyk and Mats Heimdahl. Flexibility in modeling languages and tools: A Call to Arms. To appear in Software Tools for Technology Transfer.