1. Towards a Compositional SPIN Corina Păsăreanu QSS, NASA Ames Research Center Dimitra Giannakopoulou RIACS/USRA, NASA Ames Research Center

2. Main objective: An integrated environment that supports software development and verification/validation throughout the lifecycle; detect integration problems early, prior to coding Approach: Compositional (“divide and conquer”) verification, for increased scalability, at design level Use design level artifacts to improve/aid coding, testing and fault containment Compositional Verification TestingDesignCodingRequirementsDeployment C1C1 C2C2 C1C1 C2C2 M1M1 M2M2 models implementations Cost of detecting/fixing defects increases Integration issues handled early Project Overview

3. Towards a Compositional SPIN Learning based compositional analysis for increased scalability [TACAS’03] – LTSA tool Contributions –Generic tool architecture for learning based assume guarantee reasoning Handles multiple components Uses SPIN to answer model checking questions Other tools can be used (e.g. Java PathFinder) –Heuristic for automated generation of interface specifications –Application to realistic resource arbiter for a spacecraft Significant memory gains over “traditional” non-compositional model checking

4. Outline Introduction Background: assume-guarantee reasoning and learning Tool architecture Implementation –Using SPIN for answering learning questions –Promela subset Case study: MER Arbiter model –Description, results, discussion Conclusions and future work

5. Compositional Verification M2M2 M1M1 A satisfies P? Check P on entire system: too many states! Use the natural decomposition of the system into its components to break-up the verification task Check components in isolation: Does M 1 satisfy P? –Typically a component is designed to satisfy its requirements in specific contexts / environments Assume-guarantee reasoning: introduces assumption A representing M 1 ’s “context” Does system made up of M 1 and M 2 satisfy property P?

6. Assume-Guarantee Rules M2M2 M1M1 A satisfies P? Simplest assume-guarantee rule We can also handle symmetric rules “discharge” the assumption 1.  A  M 1  P  2.  true  M 2  A  3.  true  M 1 || M 2  P  How do we come up with the assumption? (usually a difficult manual process) Reason about triples:  A  M  P  The formula is true if whenever M is part of a system that satisfies A, then the system must also guarantee P

7. Approaches Infer assumptions automatically Two solutions developed 1.Algorithmic generation of assumption (controller); knowledge of environment is not required [ASE’02] 2.Incremental assumption computation based on counterexamples, learning and knowledge of environment [TACAS’03, SAVCBS’03 (symmetric rules)]

8. Formalisms Components modeled as finite state machines (FSM) –FSMs assembled with parallel composition operator “||” Synchronizes shared actions, interleaves remaining actions A property P is a FSM –P describes all legal behaviors –P err – determinize & complete P; bad behaviors lead to “error” –Component M satisfies P iff error state unreachable in (M || P err ) Assume-guarantee reasoning –Assumptions and guarantees are FSMs –  A  M  P  holds iff error state unreachable in (A || M || P err ) –The alphabet of A,  A, contains all environment actions that appear in P

9. Example Input insend ack Output outsend ack Order err in out in out ||

10. Computing the Weakest Assumption Given component M, property P, and the interface of M with its environment, generate the weakest environment assumption A such that: assuming A, M ╞ P Weakest means that for all environments E: (E || M ╞ P) IFF E╞ A

11. Learning for Assume-guarantee Reasoning Use an off-the-shelf learning algorithm to build appropriate assumption for the rule Process is iterative Assumptions are generated by querying the system, and are gradually refined Queries are answered by model checking Refinement is based on counterexamples obtained by model checking Termination is guaranteed Extended with symmetric rules 1.  A  M 1  P  2.  true  M 2  A  3.  true  M 1 || M 2  P 

12. Learning with L* L* algorithm by Angluin, improved by Rivest & Schapire Learns an unknown regular language U (over alphabet  ) and produces a DFA A such that L(A) = U Uses a teacher to answer two types of questions Unknown regular language U L* conjecture: A i query: string s true false remove string t add string t output DFA A such that L(A) = U true false is s in U? is L(A i )=U?

13. Learning Assumptions Use L* to generate candidate assumptions  A = (  M 1   P)   M 2 L* query: string s true false  s  M 1  P  conjecture: A i  A i  M 1  P   true  M 2  A i  true error? true false true yes false no remove counterex. add counterex. P holds in M 1 || M 2 P violated 1.  A  M 1  P  2.  true  M 2  A  3.  true  M 1 || M 2  P  Model Checking

14. Characteristics Terminates with minimal automaton A for U Generates DFA candidates A i : |A 1 | < | A 2 | < … < |A| Produces at most n candidates, where n = |A| # queries:  (kn 2 + n logm), –m is size of largest counterexample, k is size of alphabet For n components (M 1 || M 2 || … || M n ) apply algorithm recursively: –  A i  M 1  P  as before –  true  M 2 || … || M n  A i  invokes the framework

15.  true  P  A i  Learning Interface Specifications Compute an assumption (as weak as possible) for a component M 1 to guarantee some property P, when environment is not available  A =  P L* query: string s true false  s  M 1  P  conjecture: A i  A i  M 1  P  true counterexample true false return A i counterexample not accurate with respect to !P || !M 1

16. Implementation in SPIN Stand-alone application –Invokes SPIN for queries and conjectures Handles only a Promela subset –Components are processes –Communication through rendezvous channels Safety properties –SPIN trace assertions Checking assume-guarantee triples –Encode assumptions into environment processes that run in parallel with components

17. Case Study Model derived from MER Resource Arbiter –Local management of resource contention between resource consumers (e.g. science instruments, communication systems) –Avoid simultaneous conflicts (e.g. driving while capturing a camera image are mutually incompatible) –Enforce orderly activity in accordance with predefined priorities encoded in a lookup table 5 resources –Comm, Drive, PanCam, Arm, Rat 5 User threads –Non-deterministically decide to use any of the 5 resources ~3000 LOC Checked several safety properties

18. Arbiter Architecture ARB U5U5 U4U4 Request, Cancel U3U3 U2U2 U1U1 Grant, Deny Rescind Property P Mutual exclusion between resources Comm and Drive can not be used at the same time

19. MER Arbiter Property Mutual exclusion between resources Comm and Drive can not be used at the same time PanCam and Arm can not be used at the same time Rat and Arm can not be used at the same time

20. Incremental Property Checking Compute A 1 … A 5 s.t.  A 1  U 1  P   A 2  U 2  A 1   A 3  U 3  A 2   A 4  U 4  A 3   A 5  U 5  A 4   true  ARB  A 5  Result P holds on U 1 ||.. || U 5 || ARB Two techniques –Recursive invocation –Interface specification Comparison with non-compositional analysis U1U1 U2U2 P A2A2 A1A1 ARB A1A1 U3U3 U4U4 A2A2 A4A4 A3A3 A3A3 U5U5 A5A5 A4A4 A5A5

21. AnalysisMemoryState SpaceTime t model t compile t run Assumption Size Monolithic544 MB3.9e+060.02s 0.8s 33.7sN/A Recursive2.6 MB10020.03s 1.1s 0.03s6.. 12 Interface2.6 MB29410.04s 1.3s 0.02s12 Analysis Results Results do not reflect the cost of generating the assumptions

22. Cost of Assumption Generation Analysisqueriesoracle1oracle2t SPIN +t Lear n Mem Learn t LTSA Mem LTSA Recursive48844815646.3s1743K42.8s20400K Interface (A 1 ) 852121818.2s508K3.07s4845K LTSA tool

23. Discussion Significant memory savings Serious time overhead Experimented with SPIN’s feature for separate compilation –Encode assumptions as never claims –Restrict search of the verifier New encoding for queries –Significant improvement –Cost of interface generation reduced from 818.213s to 185.185s

24. Conclusion and Future Work Tool architecture for compositional verification –Uses L* for automatic derivation of assumptions –Automated derivation of interface specification “Loose” integration with SPIN –Model checking for queries and conjectures Application to Arbiter case study –Significant memory gains –Serious time overhead  ; techniques to address this issue Future work –Tighter integration with SPIN –Parallel checks for queries –Investigate buffered message passing communication –Liveness (learning for infinitary regular sets) –Distinguish between read and write operations