Automated assume-guarantee reasoning for component verification Dimitra Giannakopoulou (RIACS), Corina Păsăreanu (Kestrel) Automated Software Engineering NASA Ames Research Center Moffett Field, CA, USA

System-Level Verification for Autonomous Systems Objectives Detect integration problems early Support module-based verification (for scalability) Design for verification Approach Software architecture design and verification Module verification with generated assumptions Design patterns / architectures for autonomous systems Verification is essential for autonomy insertion in missions commands plans executive

Compositional Verification M2M2 M1M1 A satisfies P? Decompose properties of system (M 1 || M 2 ) in properties of its components Does M 1 satisfy P? typically a component is designed to satisfy its requirements in specific contexts / environments Assume-guarantee reasoning: introduces assumption A representing M 1 ’s “context” Simplest assume-guarantee rule “discharge” the assumption 1.  A  M 1  P  2.  true  M 2  A  3.  true  M 1 || M 2  P 

Approach Infer assumptions automatically Two novel solutions developed 1.Algorithmic generation of assumption (controller); knowledge of environment is not required 2.Incremental assumption computation based on counterexamples, learning and knowledge of environment

Applications Verification produces more precise answers true – property satisfied in all environments false – property falsified in all environments assumption A – property true in environments where A is true Support for compositional verification Property decomposition Assumptions for assume-guarantee reasoning Runtime monitoring of environment assumption monitors actual environment during deployment may trigger recovery actions Component retrieval, sub-module construction, …

Implementation in LTSA Components modeled as labelled transition systems (LTS) an LTS M has a communicating alphabet  M internal actions represented by action “ τ ” LTSs assembled with parallel composition operator “||” –synchronizes shared actions, interleaves remaining actions A property P is a safety LTS (an LTS with no error states) P describes all legal behaviors with respect to  P P err – determinize & complete P; bad behaviors lead to “error” component M satisfies P iff error state unreachable in (M || P err ) Assume-guarantee reasoning assumptions and guarantees are safety LTSs  A  M  P  holds iff error state unreachable in (A || M || P err )

Example Input insend ack Output outsend ack Order err in out Order in out ||

Solution 1 in collaboration with Prof. Howard Barringer, Univ. of Manchester, UK Giannakopoulou, D., Păsăreanu, C., and Barringer, H., “Assumption Generation for Software Component Verification”, in Proc. of the 17th IEEE International Conference on Automated Software Engineering (ASE 2002). Awards: Best paper, ACM Distinguished Paper Award

The Problem Given component C, property P, and the interface of C with its environment, generate the weakest environment assumption A such that: assuming A, C ╞ P Weakest means that for all environments E: (E || C ╞ P) IFF E╞ A

Assumption Generation Step 1: composition, hiding, minimization Property true! (all environments) Step 2: backward propagation of error with  Property false! (all environments) Step 3: property extraction (subset construction and completion) Assumption

The problem 1.  A  M 1  P  2.  true  M 2  A  3.  true  M 1 || M 2  P  our goal: automate assume guarantee reasoning by generating appropriate assumptions for the rules developer encodes them explicitly abstractions of other modules how are assumptions obtained?

Framework, iteration i Model Checking 1.  A i  M 1  P  Learning real error? 2.  true  M 2  A i  AiAi counterexample – strengthen assumption counterexample – weaken assumption false true false YN P holds in M 1 ||M 2 P violated in M 1 ||M 2 Learning 1.  A i  M 1  P  real error? 2.  true  M 2  A i 

Learning with L* L* algorithm by Angluin, improved by Rivest & Schapire learning based on queries and counterexamples learns an unknown regular language (U over alphabet  ) and produces a DFA C such that L (C) = U constructs a sequence of DFAs C 1, C 2, … converging to C needs a Teacher to answer two types of questions: membership queries: is string s in U ? conjectures: for a candidate DFA C i, is L (C i ) = U ? –answers are (true) or (false + counterexample)

The L* Algorithm maintains a table that records whether strings in   belong to U makes membership queries to update it decides to make a conjecture – uses table to build a candidate C i if Teacher replies true, done! if Teacher replies false, uses counterexample to update the table General method: Characteristics: terminates with minimal automaton C for unknown language U each candidate C i is smallest any DFA consistent with table has at least as many states as C i |C 1 | < | C 2 | < … < |C| produces at most n candidates, where n = |C| # queries:  (kn 2 + n logm) m is size of largest counterexample, k is size of alphabet

Learning Assumptions L* learns the weakest assumption A w for M 1 for all environments E: ( M 1 || E satisfies P IFF E satisfies A w ) a string s   is in L (A w ) iff in the context of s, M 1 satisfies P  =  A w = (  M 1   P)   M 2 conjectures are intermediate assumptions A i framework may terminate before L* computes A w A w is not available – how do we implement the Teacher ? use model checking M2M2 M1M1 satisfies P? 1.  A  M 1  P  2.  true  M 2  A  3.  true  M 1 || M 2  P 

The Teacher Membership query: trace s Simulate s on M 1 || P err –If it leads to error state, reply false ( s  L (A w ) ) –Otherwise, reply true ( s  L (A w ) ) Conjecture: A i Oracle 1: performs (Step 1)  A i  M 1  P  –If false (with counterexample c): provide c  to L* (assumption too weak) –If true: forward A i to Oracle 2 Oracle 2: performs (Step 2)  true  M 2  A i  –If true: done! (P holds on M 1 || M 2 ) –If false (with counterexample c): forward c to analysis Counterexample analysis Simulate c  on M 1 || P err –If it leads to the error state: done! (P does not hold on M1 || M2) –Otherwise, provide c  to L* (assumption is too strong)

Output send ack out Order err in out in Example Input in ack send We check:  true  Input || Output  Order  M 1 = Input, M 2 = Output, P = Order Assumption’s alphabet: {send, out, ack}

Order err in out in Output send ack out Membership Queries Input in ack send E Table T S ? out S  S   ack out send out, ack out, out out, send S = set of prefixes E = set of suffixes  L (A w )? Simulate  on Input || Order err Yes!

Order err in out in Output send ack out Membership Queries Input in ack send E Table T S true out S  S   ack out send out, ack out, out out, send S = set of prefixes E = set of suffixes  L (A w )? Simulate  on Input || Order err Yes!

Order err in out in Output send ack out Membership Queries Input in ack send E Table T S true out ? S  S   ack out send out, ack out, out out, send S = set of prefixes E = set of suffixes  L (A w )? Simulate  on Input || Order err No!

Order err in out in Output send ack out Membership Queries Input in ack send E Table T S true outfalse S  S   ack out send out, ack out, out out, send S = set of prefixes E = set of suffixes  L (A w )? Simulate  on Input || Order err No!

Order err in out in Output send ack out Membership Queries Input in ack send E Table T S true outfalse S  S   ack out send out, ack out, out out, send S = set of prefixes E = set of suffixes true false true false

Order err in out in Output send ack out Candidate Construction Input in ack send E Table T S true outfalse S  S   ack out send out, ack out, out out, send S = set of prefixes E = set of suffixes ack send 2 states – error state not added to assumption Assumption A 1 true false true false

Order err in out in Output send ack out Conjectures Input in ack send ack send A1:A1: Oracle 1:  A 1  Input  Order  Counterexample: c =  in,send,ack,in  Return to L*: c   =  send,ack  Oracle 1:  A 2  Input  Order  True Oracle 2:  true  Output  A 2  True property Order holds on Input || Output ack send out, send A2:A2: Queries

Output send ack out send Order err in out in Another Example Input in ack send Oracle 2:  true  Output’  A 2  Counterexample: c=  send,send,out  ack,out,send ack send out ack send A4:A4: property Order holds on Input || Output’ Simulate c  =  send,send,out  on M 1 ||Order err Return c to L* not a real error! real error?

Ames K9 Rover Executive Executes flexible plans for autonomy branching on state / temporal conditions Multi-threaded system communication through shared variables synchronization through mutexes and condition variables Systematic translation of design documents into input language of LTSA Several synchronization issues If the Executive thread reads the value of variable savedWakeupStruct, the ExecCondChecker thread should not read this value unless the Executive clears it first. property P:

K9 Rover results iteration|A i |(#states, #trans)result 1 – Oracle 11(5, 24)too weak 2 – Oracle 12(268, 1408)too weak 3 – Oracle 13(235, 1209)too weak 4 – Oracle 15(464, 2500)not too weak Oracle 2(32, 197)false We check:  true  ExecCondChecker || Executive  P  M 1 = ExecCondChecker, M 2 = Executive 8.639secs Check M 1 || M 2 || P: (3630 sts, trans), secs

Related Work Assume-guarantee frameworks Jones 83; Pnueli 84; Clarke, Long& McMillan 89; Grumberg & Long 91; Xu, de Roever & He 97; Henzinger, Qadeer & Rajamani 98; … tool support: MOCHA; Calvin (static checking of Java); … Assumption generation Giannakopoulou, Păsăreanu & Barringer, 2002 Interfaces for Compositional Reachability Analysis Cheung & Kramer 1996, Krimm & Mounier 1997 Learning system models Groce, Peled, and Yannakakis, 2002 Ammons, Bodik, and Larus, 2002

Conclusions & Extensions framework for incremental assume-guarantee reasoning general – relies on standard features of model checkers extend framework to more than two components carry out more & larger case studies alternative approaches for generating assumptions combine with abstraction to deal with source code extend to liveness / fairness properties L  Algorithm, Maler and Pnueli, 1995 timed systems

part 2: work in progress

Symmetric Rules assumptions for both components at the same time early termination smaller assumptions take simplest symmetric rule the rule is sound and complete completeness is essential for automation 1.  A 1  M 1  P  2.  A 2  M 2  P  3.  true  M 2  A 1  OR  true  M 1  A 2   true  M 1 || M 2  P 

Framework learning  A 1  M 1  P  learning  A 2  M 2  P  A1A1 A2A2 false  true  M 2  A 1   true  M 1  A 2  OR counterexample analysis true false P holds in M 1 ||M 2 P violated in M 1 ||M 2 weaken strengthen true

Incorporation of other circular rules a variety of assume guarantee rules exist in the literature usually, they are sound but not complete we make them complete by adding premise 3 to the rule example: the rule is sound but not complete  A 1  M 1  P   A 2  M 2  P  coA 1 || coA 2 =   true  M 1 || M 2  P 

Framework learning  A 1  M 1  P  learning  A 2  M 2  P  A1A1 A2A2 false  true  M 2  A 1   true  M 1  A 2  OR true false P holds in M 1 ||M 2 P violated in M 1 ||M 2 weaken strengthen coA1 || coA2 =  false P holds in M 1 ||M 2 counterexample analysis counterexample analysis false P violated in M 1 ||M 2

Incorporation of other circular rules a variety of assume guarantee rules exist in the literature usually, they are sound but not complete we make them complete by adding premise 3 to the rule example – Chandy and Misra 1981: the rule is sound but not complete  A 1  M 1  P   A 2  M 2  P  M  A 1, P , M  A 2,P  P |= A 1, P |= A 2  true  M 1 || M 2  P 

Compositional verification for C C 1 ||C 2 |=P spurious C 1 ||C 2 |= P true false counterexample analysis learning framework predicate abstraction M1M1 C1C1 predicate abstraction M2M2 refine check C 1 ||C 2 |= P use as much information as you can from previous iterations C2C2 refine spurious

Compositional verification for C (details) learning  A 1  M 1  P  learning  A 2  M 2  P  A1A1 A2A2 false  true  M 2  A 1   true  M 1  A 2  OR counterexample analysis true false C 1 ||C 2 |=P weaken A 1 strengthen A 1 strengthen A 2 M1M1 predicate abstraction C1C1 predicate abstraction C2C2 weaken A 2 strengthen M 1 strengthen M 2 M2M2 false M1M1 M2M2