Ed Brinksma Dept. of CS, University of Twente, NL joint work with Angelika Mader Monterey Workshop 2003 Chicago Verification Modelling of Embedded systems

September, 2003Monterey Workshop, Chicago Area flooded in 1953 catastrophe Last construction of the Delta Works ES Verification Example: storm surge barrier control dimensions

September, 2003Monterey Workshop, Chicago The control system l no human intervention human operation too unreliable l responsible for closing & opening online meteorological & hydrological data l very low failure rates event failure rate ~10 -5 /barrier event l design & verification with FM considered successful

September, 2003Monterey Workshop, Chicago Design validation ldata types & operations formally specified in Z lcrucial control parts modelled in Promela & model checked with Spin limplementations were hand-coded using Z specs limplementations were tested lno actual code was proved correct

September, 2003Monterey Workshop, Chicago Questions lHow to do practical verification of ES ? lIs it methodologically sound? lHow should this affect research?

September, 2003Monterey Workshop, Chicago The Setting lverification of ES designs is desirable critical aspects are common: safety-critical, high replication, costly, etc. lverification needs formalization operational model, logical theory, requirements lformalization is problematic nthe (standard) combinatorial explosion nincorporation of (physical) environment

September, 2003Monterey Workshop, Chicago Typical Situation lverification model is constructed in an ad hoc and opportunistic manner lthe success of verification is crucially dependent on scarce expertise lthe relation of the verification model to the actual design is opaque the verification crisis: model hacking precedes model checking

September, 2003Monterey Workshop, Chicago What do we need? Verification models should have/be l limited complexity must be open to computer-aided verification l faithful must capture relevant properties l traceable clear relation to actual design or system

September, 2003Monterey Workshop, Chicago Complexity Issues lmodels must be sufficiently small limited capacity verification tools limited capacity verification management lhybrid nature ES complicates models mixed techniques, symbolic analysis ltool capacity growth exceeds Moore’s law better algorithms & data structures

September, 2003Monterey Workshop, Chicago Abstractions Verification models are abstractions: n inherent abstractions mathematical modelling of physical aspects n controlled abstractions simplifications reducing complexity

September, 2003Monterey Workshop, Chicago Faithfulness lVerification of erroneous models is useless (or even worse). lModels must obviously capture the relevant system properties. However: lwhat are relevant (formal) properties? these are often part of the design problem ldo our abstractions preserve them? this can be difficult to show (begging the question) verification models & properties must be validated !

September, 2003Monterey Workshop, Chicago Model validation In addition to traceability verification models can be validated by experimental means: 1.simulation of the model requires constructive modelling 2.analysis of verification results in practice model validation and verification are mixed

September, 2003Monterey Workshop, Chicago Separating the errors la verification run may fail due to 1. an error in the implementation 2. an error in the verification model 3. an error in the formal property l errors must be analysed to modify appropriate entity l requires rigourous protocol for analysis & documentation verification should always include a systematic error discussion (cf. physics)

September, 2003Monterey Workshop, Chicago Software Model Extraction lprogram code as model lreduction by abstract interpretation ndata/predicate abstraction nvariable slicing lmodel check abstractions leliminate false negatives can be done concurrently on many abstractions does not work for non-programmable model parts

September, 2003Monterey Workshop, Chicago Verification Engineering lVerification modelling as a design problem nclosely related but different from system design nmain design criterion: limited complexity & tool support lSystematic approach to model construction ncapture physical aspects nreduce complexity nformal and experimental justification lTool support for verification management nmodel/property version management nmeta-level specification of verification campaigns

September, 2003Monterey Workshop, Chicago Systematic Verification Model interpret error M, S abstract M i,  i model check correct errorreverify models error all done? correct modelling Design phase Modelling phase system descriptionrequirements simulation design error back to design phase no yes verified M w.r.t. S Verification phase

September, 2003Monterey Workshop, Chicago Xspin/Project - usage “Sandbox” environment: Accessing PRCS Saving validation results Forcing version integrity Xspin/Project adds an extra Project-menu. Verification data can be saved into the repository.

September, 2003Monterey Workshop, Chicago Challenges lverification methodology (MoMS project) nsystematic model traceability combining formal and non-formal aspects nmodelling & abstraction patterns libraries, domain dependent solutions nsystematic model validation lverification management tools ndocumentation & version control nverification integrity control nverification campaign management

September, 2003Monterey Workshop, Chicago And finally … lthe company involved got very enthusiastic about FM la 1-year technology transfer project was carried out lafter 5 years they are still only using the (model-driven) testing tools