Formal Methods and Testing: Possible Attributes for Success A. J. Cowling Department of Computer Science University of Sheffield

CSEE&T 2001University of Sheffield Department of Computer Science Rationale The Number of Different Methods Many formal and semi-formal methods exist New ones are still being created Their Usefulness Determining which are useful requires empirical work The number of methods would imply a lot of work Therefore priorities need to be set Technical Features Some methods appear to be more successful than others Possibly because of their technical features These would be more appropriate candidates for study

CSEE&T 2001University of Sheffield Department of Computer Science Role of Models Requirements Model Specification Model Design Model Implementation Model Development Process Required V & V Formal steps Testing

CSEE&T 2001University of Sheffield Department of Computer Science Testing and Models Basic Testing Methods Base the generation of test cases on one of these models –Black-box testing uses the specification model –White-box testing uses the implementation model Hybrid Testing Methods Combine the approaches – eg: –Black-box methods to generate the test sets –White-box methods to measure their coverage May provide more effective testing than individual basic methods –At least, according to some papers State-based Testing Uses state-machine models for specification and implementation Extended models (eg the X-machine) allow powerful results: –absence of faults up to some bounds, –under some assumptions, complete absence of faults

CSEE&T 2001University of Sheffield Department of Computer Science Key Formal Methods Model Checking Requires state-based specification models Shows whether required properties hold for the models Can handle very large systems (10^20 states) Machine Model Verification Uses state-based specification and implementation models (eg B) Can verify that implementation is consistent with specification Refinement Typically uses relational models (eg Z, VDM) Refinement steps produce correct-by-construction implementations Discontinuities in the models need to be accommodated –Retrenchment has been proposed for this

CSEE&T 2001University of Sheffield Department of Computer Science Attributes for Success Role of Models Successful approaches appear to all be model-based State machine models are particularly successful Extended state-machine models even more so Differences between Models The different stages require different models for one system Any form of V & V must accommodate these differences –ie must represent design transformations –currently an interest within model-driven architecture It appears that successful methods: –explicitly handle multiple models, and –explicitly represent the differences between them.