Verification of Evolving Software Natasha Sharygina Joint work with Sagar Chaki and Nishant Sinha Carnegie Mellon University

Motivation Component-based Software –Software modules shipped by separate developers –Undergo several updates/bug-fixes during their lifecycle Component assembly verification –Necessary on update of any component –High verification costs of global properties

Contribution Assembly A Component C Component C’ P ?

Contribution Automated substitutability check –Preserving all old behaviors –Allowing new behaviors Focus on components that have been modified Allow multiple upgrades simultaneously

Substitutability Check Given old and new components C i, C’ i and assembly A –For each i check if C’ i is substitutable for C i in A Two phases: –Containment check Ensures that behaviors of the old component are contained in its substitutable counterpart –Compatibility check Safety with respect to other components in assembly: all global specifications are satisfied Approach: –Obtain a finite behavioral model of all components by abstraction: Labeled Kripke Structures (LKS) –Use model checkers to solve both phases automatically

Abstraction into LKS Labeled Kripke Structures – State-event traces – Composition semantics –Synchronize on shared actions Two types of abstraction: –Predicate abstraction (over-approximations: existential transitions) –Modified predicate abstraction (under-approximations: universal transitions) p !q q    !p

Labelled Kripke Structures Directed graph with labels on edges and states, (S,Init,P,L,T, ,E) –Every state is labeled with a set of atomic propositions, P, true in the state –Every LKS comes with an alphabet of actions,  State labeling function : L: S  2 P Transition labeling function : E : T  (2  \ {}) –Assumption: LKSs are deadlock-free [cf. MEMOCODE 04] a bc b a 0,1 1,11,0 0,0

Surge Protector : State/Event State/Event model of the Surge Protector (example is given for m: [0..2], c: [0..2])

Surge Protector : State Only Kripke structure of the Surge Protector (example is given for m: [0..2], c: [0..2])

Abstraction into LKS Program Statements, States Control flow, Transitions Transitions depict observable behavior Automated

Predicate Abstraction into LKS L1 lock = 0 if (x < y) lock=1 x < y void OSSemPend(…) { L1: lock = 1; if (x < y) { L2: lock = 0; … } if (x >= y) { … L3: lock = 0; … } else { … } L2   L3   if (x >= y) lock = 0 x < y

Verification L1 if (P) L2 L3 unlock lock Model ERROR unlock lock if (!P) Specification

Predicate Abstraction into LKS L1 lock = 0 if (x < y) lock=1 x < y if (x >= y) lock = 0 x < y if (x < y) x >= y lock=1 void OSSemPend(…) { L1: lock = 1; if (x < y) { L2: lock = 0; … } if (x >= y) { … L3: lock = 0; … } else { … } L2  if (x >= y) x >= y   L3 

Verification L1 if (*) L2 L3 unlock lock Model ERROR unlock lock if (*) Specification

ComFoRT: Component Formal Verification Framework Verification Yes System OK Abstraction Model Counterexample Valid? Components Assembly Yes No Counterexample Counter Example Guided Abstraction Refinement Abstraction Refinement Improved Abstraction Guidance No Spurious Counterexample

CEGAR Process Abstraction / Model Creation Theorem Prover Verification Safety/Liveness Checker Yes System OK Counterexample Valid? ComponentsLKSs Yes Abstraction Refinement Abstraction Guidance No Candidate Counterexample

Containment Step 1. Construct LKSs M’ and M such that (C1) C µ M and (C2) M’ µ C’ Step 2. Verify if M µ M’ If yes then from (C1) and (C2) C µ C’ and we terminate Else CE is produced Step 3. Validate CE 1. Check if CE  C. If yes proceed to next step. Otherwise refine M and repeat from Step Check if CE  C’. If yes than CE  C \ C’, Feedback  CE and stop. Else refine M’ (by adding the valid CE) and repeat from Step 2.

Containment (contd.) In contrast to Refinement-based approaches –Containment ensures that all behaviors of M are present in M’ –However, containment allows new behaviors in M’ which could be possibly erroneous Containment check is not sufficient since new components –may not be safe with respect to other components –must satisfy the global behavioral specifications

L* learner Learning Regular languages: L* Forms the basis of the compatibility check L* proposed by D. Angluin Polynomial in the number of states and length of counterexample Minimally adequate Teacher IsMember( trace  ) IsCandidate( DFA D ) a b a b Unknown Regular Language ±Counterexample/ Yes Model checker Yes/No Minimum DFA

Compatibility check Recursive Assume-guarantee to verify assembly properties Generate a (smaller) environment assumption A –A: most general environment so that P holds –Constructed iteratively using L* and R 1, R 2 –Recursion for the R2 rule R 1 : M 1 || A ² P R 2 : M 2 || … || M n ² A M 1 || … || M n ² P

Compatibility check R 1 : M 1 || A i ² P R 2 : M 2 || … || M n ² A i true L* Assumption Generation AiAi true CE CE Analysis True CE False CE -CE for A i +CE for A i

Compatibility check (contd.) Generate a most general assumption for each M’ –M 1 = M’ –M 2 = Mn M (all other component LKSs) Membership queries: –M’ || CE µ P Candidate queries: –M’ || A µ P –M 2 µ A

Compatibility check (contd.) CE analysis: M’ || CE µ P –Yes ) False CE –No ) True CE Compatibility check infers –Either M’ is substitutable –Or counterexample CE CE may be spurious wrt. C’ due to the predicate abstraction –CE is present in component LKS M’ –Must refine M’ –Repeat substitutability check

Compatibility check (contd.) Dynamic compatibility check –Restarts learning using information from a previously learned candidate –Account for incremental changes between successive assumptions The dynamic nature is critical for 1) checking evolving software; 2) during abstraction refinement where a single component is updated at a time.

Feedback to Developers Feedback - collection of CEs  C identified during containment check Feedback Delivery: Given a trace   Feedback, and Rep(  ) – representation of  in terms of program control locations, predicate valuations and actions, the levels of feedback are 1) Rep(Prefix(  )) – to denote the divergence point of  2) Rep(Suffix(  )) – to show what exact behavior (exact code) of C is missing in C’ 3) Rep’(Pref(  )) – to denote location in Ci where changes are to be made

Experiments Prototype implementation in ComFoRT framework ABB IPC Module –Deployed by a world leader in robotics engineering systems – LOC –modified WriteMessageToQueue component –checked for substitutability inside the assembly of four components (read, write, queue, critical section) –Over 30 billion states after predicate abstraction Discovered synchronization bug –Process can incorrectly block while writing to a queue

Related Work Learning Assumptions: Cobleigh. et. al. –Do not consider state labeling of abstract models –Do not incorporate a CEGAR framework for AG Compatibility of Component Upgrades: Ernst et. al. –Do not consider temporal sequence of actions in generating invariants Interface Automata: Henzinger et. al. –Do not have CEGAR, AG

ComFoRT framework:

Questions?