Hiding the Formalism in Formal Methods Lori A. Clarke Laboratory for Advanced Software Engineering Research (LASER) University of Massachusetts, Amherst Work done in collaboration with George S. Avrunin, Jamieson M. Cobleigh, Rachel Cobleigh, Heather M. Conboy, Matthew B. Dwyer, Gleb Naumovich, & Leon J. Osterweil

Model Checking Includes a wide range of approaches for determining if finite models of systems are consistent with specified properties Includes a wide range of approaches for determining if finite models of systems are consistent with specified properties E.g., SPIN, SMV,FLAVERS E.g., SPIN, SMV,FLAVERS Verifies properties about system behavior Verifies properties about system behavior Successfully applied to hardware, software, and various modeling languages (BPEL, Petri Nets, UML, Little-JIL) Successfully applied to hardware, software, and various modeling languages (BPEL, Petri Nets, UML, Little-JIL) Seeks a middle ground between testing and theorem proving Seeks a middle ground between testing and theorem proving Testing requires “executable” semantics and only provides selective results Testing requires “executable” semantics and only provides selective results Theorem proving can deal with a wider range of properties, but usually requires more mathematical expertise Theorem proving can deal with a wider range of properties, but usually requires more mathematical expertise

Model checking is still not widely applied State Explosion Problem State Explosion Problem The cost of analysis can be exponential in the size of the system being analyzed The cost of analysis can be exponential in the size of the system being analyzed Restricted to small systems Restricted to small systems Even with extensive optimizations, creating a concise, but valid, model requires considerable insight Even with extensive optimizations, creating a concise, but valid, model requires considerable insight Specifying properties is difficult Specifying properties is difficult Notations are cumbersome Notations are cumbersome Many details must be taken into consideration Many details must be taken into consideration

Two Approaches for increasing acceptance Automatically produce the model that is the basis for analysis Automatically produce the model that is the basis for analysis Produce a very concise, but conservative model Produce a very concise, but conservative model Incrementally add precision Incrementally add precision Provide natural language Interfaces for creating properties Provide natural language Interfaces for creating properties

Outline FLAVERS approach for automatically creating and improving the model FLAVERS approach for automatically creating and improving the model Checking Properties Checking Properties Improving Precision Improving Precision Experimental Results Experimental Results PROPEL approach for elucidating properties PROPEL approach for elucidating properties Question Tree Question Tree Disciplined Natural Language Disciplined Natural Language Finite-state automaton Finite-state automaton

FLAVERS FLow Analysis for VERification of Systems FLow Analysis for VERification of Systems Verifies properties about concurrent and sequential systems Verifies properties about concurrent and sequential systems Properties are represented as finite state automata Properties are represented as finite state automata Checked using an efficient state propagation algorithm Checked using an efficient state propagation algorithm Uses an abstract, event-based graph model of the system Uses an abstract, event-based graph model of the system Imprecise, but conservative Imprecise, but conservative Precision can be improved incrementally Precision can be improved incrementally

Models for Concurrent Systems One model for a concurrent system is a reachability graph One model for a concurrent system is a reachability graph Represents all of the states a concurrent system may reach Represents all of the states a concurrent system may reach Location within each task Location within each task Values of variables Values of variables

Reachability Graph task body t1 is begin u; u; t2.send_synch; t2.send_synch; v; v; w; w; end t1; task t2 body is begin x; x; t1.rec_ synch; t1.rec_ synch; y; y; z; z; end t2; b,bb,bb,bb,b u,b u,x b,x s,ss,ss,ss,s s,y v,s w,s v,y w,y e,ee,ee,ee,e s,z v,z w,z

Trace Flow Graph (TFG) A TFG represents control flow through a concurrent system A TFG represents control flow through a concurrent system Built from Control Flow Graphs for the tasks in the system Built from Control Flow Graphs for the tasks in the system Nodes and edges are added to represent concurrency Nodes and edges are added to represent concurrency

`` TFG Construction x y u v w synch rec_ synch send_synch task body t1 is begin u; u; t2.send_synch; t2.send_synch; v; v; w; w; end t1; task t2 body is begin x; x; t1.rec_synch; t1.rec_synch; y; y; z; z; end t2; z

x y u v w synch z b,bb,bb,bb,b u,b u,x b,x s,ss,ss,ss,s s,y v,s w,s v,y w,y e,ee,ee,ee,e s,z v,z w,z u b,bb,b u,b synch s,ss,s v v,s w w,s z w,z e,ee,e u,x x w,y y Feasible Paths

x y u v w synch z b,bb,bb,bb,b u,b u,x b,x s,ss,ss,ss,s s,y v,s w,s v,y w,y e,ee,ee,ee,e s,z v,z w,z Infeasible Paths synch u b,bb,b u,b

Elevator Property The elevator does not move while its doors are open. L (P) is the set of all strings accepted by P close open move close move open open close move

Control Flow Graph (CFG) A CFG G is A CFG G is Associate events with nodes Associate events with nodes  G is the alphabet of G  G is the alphabet of G L (G) is the language of G L (G) is the language of G The set of all strings in (  G )  that occur on paths from the initial node to the final node The set of all strings in (  G )  that occur on paths from the initial node to the final node CFG is alphabet refined CFG is alphabet refined Remove nodes that do not affect the property being verified Remove nodes that do not affect the property being verified

Simple Sequential Example … 1:if (stopped) then 2:open; end if; … 3:if (stopped) then 4:close; end if; … 5:move; … 1: if 2: open 3: if 4: close 5: move

Proving Properties Given a CFG G and a property P Given a CFG G and a property P Alphabet refine G with respect to  P Alphabet refine G with respect to  P Need to show L (G)  L (P) Need to show L (G)  L (P) Use data-flow analysis to propagate states of P to the nodes of G Use data-flow analysis to propagate states of P to the nodes of G Worst-case cost is O((N G ) 2  S P ) Worst-case cost is O((N G ) 2  S P )

State Propagation 2: open 4: close 5: move Worklist: 2, 3 {1} {2} {1} {1,2} {1,3}, 4, close open move close move open open 3: if 1: if

State Propagation close open move close move open open Worklist: 2, 3 {1} {2} {1} {1,2} {1,3}{1,3}{1,3}{1,3}, 4, 5 2: open 4: close 5: move 3: if 1: if

State Propagation 1: if 2: open 3: if 4: close 5: move {1} {2} {1} {1,2} {1,3} close open move close move open open

State Propagation 1: if 2: open 3: if 4: close 5: move … 1:if (stopped) then 2:open; end if; … 3:if (stopped) then 4:close; end if; … 5:move; …

Boolean Variable Constraint Boolean Variable Constraint == is a predicate = is assignment S==t S=t S==tS=t S==t S==f S=f S==f S==t S=t S==f S=f S==f S=f S=f S=t u ft v

Boolean Variable Constraint Boolean Variable Constraint == is a predicate = is assignment S==t S=t S==tS=t S==t S==f S=f S==f S==t S=t S==f S=f S==f S=f S=f S=t u ft v

Improving Precision Use constraints to improve precision Use constraints to improve precision Represented as FSAs Represented as FSAs Given a CFG G, a property P, and constraints C 1,…,C n Given a CFG G, a property P, and constraints C 1,…,C n Alphabet refine G with respect to (  P   C1  …   Cn ) Alphabet refine G with respect to (  P   C1  …   Cn ) Want ( L (G)  L (C 1 )  …  L (C n ))  L (P) Want ( L (G)  L (C 1 )  …  L (C n ))  L (P) Worst-case cost is O(N G 2  S P  S C1  …  S Cn ) Worst-case cost is O(N G 2  S P  S C1  …  S Cn )

Elevator Revisited 1: if 2: S==t 5: if 9: move 4: S==f 3: open 6: S==t 8: S==f 7: close … 1,2,4:if (stopped) then 3: open; end if; … 5,6,8:if (stopped) then 7: close; end if; … 9:move; …

, 6, 8, 5, 3 State Propagation 2: S==t 1: if 5: if 9: move 4: S==f 3: open 6: S==t 8: S==f 7: close close open move close move open open t v S==t fuS==f S==f S==t S==t S==f S==fS==t <2,t>,<1,v> Worklist: 2, 4 <1,u> <1,t> <2,t> <1,f> <2,t>,<1,f>

1: if 5: if, 6, 8, 5, 3 State Propagation 2: S==t 9: move 4: S==f 3: open 6: S==t 8: S==f 7: close,, Worklist: 2, 4 <1,u> <1,t> <2,t> <1,f> <2,t>,<1,f> close open move close move open open t v S==t fuS==f S==f S==t S==t S==f S==fS==t

1: if 5: if, 6, 8, 5, 3 State Propagation 2: S==t 9: move 4: S==f 3: open 6: S==t 8: S==f 7: close <2,t>,<1,v> Worklist: 2, 4 <1,u> <1,t> <2,t> <1,f> <2,t>,<1,f> <1,t><2,v>,<1,f><1,t>,<1,f> close open move close move open open t v S==t fuS==f S==f S==t S==t S==f S==fS==t, 7, 9

1: if 5: if, 6, 8, 5, 3 State Propagation 2: S==t 9: move 4: S==f 3: open 6: S==t 8: S==f 7: close <2,t>,<1,v> Worklist: 2, 4 <1,u> <1,t> <2,t> <1,f> <2,t>,<1,f> <1,t> <2,v>,<1,f> <1,t>,<1,f> close open move close move open open t v S==t fuS==f S==f S==t S==t S==f S==fS==t, 7, 9

Automatically Add Constraints as Needed Variable Automata - model variables that impact important predicates Variable Automata - model variables that impact important predicates Task Automata - model control flow of selective tasks Task Automata - model control flow of selective tasks

x y u v w synch z b,bb,bb,bb,b u,b u,x b,x s,ss,ss,ss,s s,y v,s w,s v,y w,y e,ee,ee,ee,e s,z v,z w,z Infeasible Paths synch u b,bb,b u,b

Experimental Results Evaluate how FLAVERS performance scales as program size increases Evaluate how FLAVERS performance scales as program size increases Time Time Memory Memory Number of constraints Number of constraints

Example: Chiron User interface system developed at UC Irvine User interface system developed at UC Irvine Uses event-based notification Uses event-based notification Scaled by increasing the number of listened for events Scaled by increasing the number of listened for events Lines of code Lines of code 2 events events events3, events3,557 Proved several properties about Chiron Proved several properties about Chiron (Avrunin, Corbett, Dwyer, Pasareanu, Siegel) p07 - If listener1 registers for event1 before listener2, then listener1 will be notified of event1 before listener2 p07 - If listener1 registers for event1 before listener2, then listener1 will be notified of event1 before listener2 p09 - The program never terminates while a listener is listening for an event p09 - The program never terminates while a listener is listening for an event

Observations about Using Constraints In our experiments In our experiments For the vast majority of programs and properties, the constraints needed to verify a property for the smallest configuration of the system are sufficient to verify the property for larger configurations For the vast majority of programs and properties, the constraints needed to verify a property for the smallest configuration of the system are sufficient to verify the property for larger configurations Never needed more than 4 constraints Never needed more than 4 constraints Have not tried to find the minimal number of constraints Have not tried to find the minimal number of constraints Vast majority of constraints could be determined automatically using simple heuristics Vast majority of constraints could be determined automatically using simple heuristics A useful modeling approach A useful modeling approach Can model aspects of the environment Can model aspects of the environment Can model malicious behavior Can model malicious behavior

Outline FLAVERS approach for automatically creating and improving the model FLAVERS approach for automatically creating and improving the model Checking Properties Checking Properties Improving Precision Improving Precision Experimental Results Experimental Results PROPEL approach for elucidating properties PROPEL approach for elucidating properties Question Tree Question Tree Disciplined Natural Language Disciplined Natural Language Finite-state automaton Finite-state automaton

Property Specifications A property focuses on describing one particular aspect of system behavior A property focuses on describing one particular aspect of system behavior Even with such focus, it can still be difficult to write a property correctly Even with such focus, it can still be difficult to write a property correctly A property should be precise and accessible A property should be precise and accessible precise enough to support unambiguous communication and automated analyses precise enough to support unambiguous communication and automated analyses accessible enough to be readily understood accessible enough to be readily understood

What is the problem? Specifications need to be… Accessible Accessible Rigorous Rigorous Precise/accurate Precise/accurate Consistent Consistent Analyzable Analyzable natural language is accessible “ the nurse must verify the patient's identifying information before starting to transfuse a unit of blood product into the patient”  ..but it is neither rigorous nor precise  Is the nurse allowed to verify the patient's identity more than once before starting to transfuse blood?  Does the nurse have to verify the patient's identity again before transfusing the next unit of blood?

  temporal logic is rigorous Linear Temporal Logic (LTL): ☐ ¬ order_received ∨ ☐ ¬ order_received ∨ ♢(order_received ∧ ♢(order_received ∧ (☐ ¬ blood_transfused ∨ (☐ ¬ blood_transfused ∨ ( ¬ blood_transfused U ( ¬ blood_transfused U identity_verified))) identity_verified)))   …but not accessible to most practitioners What is the problem? Specifications need to be… Accessible Accessible Rigorous Rigorous Precise/accurate Precise/accurate Consistent Consistent Analyzable Analyzable

Our Approach Provides property templates that explicitly shows options Provides property templates that explicitly shows options Extends property patterns (Dwyer, Avrunin, & Corbett 1998; 1999) Extends property patterns (Dwyer, Avrunin, & Corbett 1998; 1999) Provides multiple views of the property Provides multiple views of the property Views chosen to support precision, accessibility, and user guidance Views chosen to support precision, accessibility, and user guidance User can work with one or more of the views User can work with one or more of the views Changes made in one view are reflected in the others Changes made in one view are reflected in the others Formal view is rigorous enough to support verification Formal view is rigorous enough to support verification Implemented prototype tool, PROPEL Implemented prototype tool, PROPEL PROPerty Elucidation PROPerty Elucidation

Propel Templates Global Before end After start Between start and end SCOPESBEHAVIORS ResponseA results in B PrecedenceA enables B AbsenceA never occurs ExistenceA must occur Name Intent

Question Tree View Developed to help users select the appropriate pattern templates Developed to help users select the appropriate pattern templates One tree for scope and one for behavior One tree for scope and one for behavior Found to also be useful for resolving detailed options Found to also be useful for resolving detailed options

Example Property The patient’s identification must be verified prior to transfusing each unit of blood product. Must identify events of primary interest Must identify events of primary interest One or two events views use the actual parameter names if they are provided One or two events views use the actual parameter names if they are provided EVENT: transfuse-blood EVENT: verify-patient-ID

Question Tree View How many events of primary interest are there?  One: event verify-patient-ID  Two: events verify-patient-ID and transfuse-blood   After verify-patient-ID occurs, transfuse-blood is required to occur   transfuse-blood cannot occur until after verify-patient- ID has occurred

Precedence FSA Template  transfuse-blood or or  verify-patient-ID or or  (verify-patient-ID, transfuse-blood) transfuse-blood) or or verify-patient-ID transfuse-blood  (verify-patient-ID, transfuse-blood) transfuse-blood) verify-patient-ID  transfuse-blood or or  verify-patient-ID or or  (verify-patient-ID, transfuse-blood) transfuse-blood)

Precedence FSA Template  transfuse-blood or or  verify-patient-ID or or  (verify-patient-ID, transfuse-blood) transfuse-blood) or or verify-patient-ID transfuse-blood  (verify-patient-ID, transfuse-blood) transfuse-blood) verify-patient-ID  transfuse-blood or or  verify-patient-ID or or  (verify-patient-ID, transfuse-blood) transfuse-blood)

Precedence DNL Template

Example Behavior t ransfuse-blood cannot occur unless verify-patient-ID has already occurred. It is acceptable for verify-patient-ID to not occur, but if it does not occur then transfuse-blood can never occur. Even if verify-patient-ID does occur, transfuse-blood is not required to occur. Before the first verify-patient-ID occurs, the events in the alphabet of this property, other than transfuse-blood, can occur any number of times. After verify-patient-ID occurs and before the first subsequent transfuse-blood occurs: the events in the alphabet of this property, including verify-patient-ID but not transfuse-blood, can occur any number of times. After the first subsequent transfuse-blood occurs: the events in the alphabet of this property, other than verify-patient-ID or transfuse-blood, can occur any number of times; neither verify-patient-ID nor transfuse-blood can occur again.  (verify-patient-ID, transfuse-blood) transfuse-blood) transfuse-blood  (verify-patient-ID, transfuse-blood) transfuse-blood) verify-patient-ID  transfuse-blood

Observations about Specifying Properties Have used PROPEL (and FLAVERS) in the UMass Medical Safety Project Have used PROPEL (and FLAVERS) in the UMass Medical Safety Project 3 in-depth case studies 3 in-depth case studies Blood transfusion, Chemotherapy, Emergency Room Flow Blood transfusion, Chemotherapy, Emergency Room Flow Found (both) to be very effective Found (both) to be very effective Needed to add exceptional behavior to PROPEL Needed to add exceptional behavior to PROPEL Users frequently overlook the impact of exceptional behavior Users frequently overlook the impact of exceptional behavior Both computer scientists and non-computer scientists have found the approach useful Both computer scientists and non-computer scientists have found the approach useful

Conclusions Many areas for future research Many areas for future research Automatic model generation Automatic model generation Property specifications Property specifications Counter example generation Counter example generation Optimizations Optimizations Compositional analysis Compositional analysis Believe that model checking will become a standard technique in IDEs Believe that model checking will become a standard technique in IDEs Already available as a “hidden” optimization technique for some programming languages Already available as a “hidden” optimization technique for some programming languages Predefined properties Predefined properties Has tremendous potential in Model Based Development Has tremendous potential in Model Based Development Properties endure, perhaps with refinement, as models are elaborated Properties endure, perhaps with refinement, as models are elaborated