Compositional Verification of Evolving Component-based Systems Nishant Sinha ECE, CMU PhD Proposal Presentation

1 Overview  Problem of Re-validating Evolving Systems  Our Solution : Substitutability Check Containment Compatibility  Assume-Guarantee for Simulation  Related Approaches  Proposed Research  Timeline

2 Evolving Systems/Designs  Evolution is inevitable in any real hardware/software system: Changing requirements, designs Bug fixes Product changes (e.g., underlying platform) Upgrades of vendor-provided components

3 Re-validation on Evolution Assembly A Component C (1.0) Component C’ (2.0) P ? added functionalities bug fixes

4 Re-validating Evolving Systems  Component assembly re-validation Necessary on upgrade of any component High costs of complete global verification  For example, a software system of 25 components (~20K LOC) and 100+ properties might take up to a month of verification effort  Require efficient formal techniques for re-validation upon evolution

5 Re-validation on Evolution  Pervasive problem in Software and Hardware Engineering communities  Evolution Upgrades, Bug Fixes, Re-designs..  Re-validation appears in numerous forms Regression checks, Upgrade compatibility checks, Re-design validation  Our proposed formulation/solution Substitutability Check

6 Subst. Check - Requirements  Substitutability Check for Upgrades Formal technique for statically re-validating upgrades  Local and Incremental nature Should avoid complete re-validation Local : focus on modified components Incremental : reuse previous validation results  A “make” for component-based systems …

7 Subst. Check - Our Solution  Our Substitutability Check procedure Focus on Static Re-validation of upgrades Local and Incremental Supports changes in Component Interfaces and Behaviors Supports multiple component upgrades

8 Component C Identical Behaviors New Behaviors Lost Behaviors Upgraded Component C’ Component Upgrade  Involves both addition and removal of behaviors Compatibility Check (global) Containment Check (local)  Two-phase Subst. Check Local : Containment Global : Compatibility

9 Substitutability Check

10 Substitutability Check  Containment Detects if any useful behaviors are lost Locally checks if behaviors (services) of the previous component are still present in new one C  C’  Compatibility Safety with respect to other components in assembly Detects if the added behaviors cause any assembly- level violations Does [ C’ 1 || C 2 ||.. || C n  P ] hold? [FM’05] [FM’05] Dynamic Component Substitutability Analysis, S. Chaki, E. Clarke, N. Sharygina, N. Sinha, Formal Methods Conference, 2005.

11 Subst. Check for Software Components  We have formalized Substitutability Check in context of Component-based Software Systems [FM’05] Components are communicating C programs Obtain a finite behavioral model of all components by abstraction: State-event communicating automata  Containment: Use both Under- and Over- approximations of components  Compatibility: Use Dynamic Assume-Guarantee Reasoning Focus of proposed research

12 State-event (SE) automata  Both state and actions labeled  Composition semantics Synchronize on shared actions  Represents abstractions p !q q    !p C Component Component SE Automaton abstraction Predicate Abstraction

13 Component Assembly  A set of communicating concurrent C programs No recursion, procedures inlined Communication is abstracted into interface actions C1C1 C2C2 C3C3 M1M1 M2M2 M3M3 Predicate Abstraction Component Assembly Abstract Assembly

14 Component Assembly Example  Inter-process communication module Lock serverQueue Write_to_queueRead_from_queue Enter_crit_section, Leave_crit_section ReadMQWriteMQ

15 Containment Check

16 Containment Check  Goal: Check C  C’ All behaviors retained after upgrade Cannot check directly: need approximations  Idea: Use both under- and over- approximations  Solution: Compute M: C  M Compute M’: M’  C’ Check for M  M’ C Containment Check Identical NewLost C’

17 Containment (contd.) CC’ M M’ over-approxunder-approx M  M’ ? True C  C’ False, CE CE  C ? False, Refine M CE  C’ ? True, Refine M’ ! ( C  C’), CE provided as feedback False True M C C’ M’

18 Containment (contd.)  Computing over- (may) approximation Conventional predicate abstraction A may (a,a’) =  s,  s’. [ R(s,s’)  a= h(s)  a’ = h(s’) ]  Computing under- (must) approximation Modified predicate abstraction Universal abstraction A must (a,a’) =  s,  s’. [ R(s,s’)  a= h(s)  a’ = h(s’) ]

19 Compatibility Check

20 Compatibility Check  Goal: Detect system-level spec violations due to added behavior  Approach: Dynamic Compositional Reasoning Use compositional reasoning to obtain verification proofs for the initial assembly Identify parts of the proof that need to be re- validated on upgrades Re-validate those parts while reusing as much of the previous proof as possible C Compatibility Check Identical NewLost C’

21 Compositional Reasoning  Assume-Guarantee Reasoning (AGR)  Automatically generate assumption A Cobleigh et. al. at NASA Ames  Use learning algorithm for regular languages, L* M 1 || A  P M 2  A M 1 || M 2  P AG - Non Circular

22 L* learner Learning Regular languages: L*  Proposed by D. Angluin, improved by Rivest et al. Learning regular sets from queries and counterexamples, Information and Computation, 75(2), Learns the minimal DFA corresponding to an unknown regular lang. Minimally adequate Teacher IsMember( trace  ) IsCandidate( DFA D ) a b a b Unknown Regular Language U ±Counterexample/ Yes Modelchecker Yes/No Minimum DFA Polynomial in the number of states and length of max counterexample

23 L* : Data Structures  Observation table : empty initially  Rows have access traces  Columns have distinguishing traces  Table entries show membership of traces in U  Iteratively updates the table using provided counterexample samples  Finally, builds a DFA from a closed table and presents it as a candidate  a ab  a b aa ab  q0q0 q1q1 a,b a b [q 0 ] [q 1 ] [q 0 ]

24 Automated AG Reasoning using L* R 1 : M 1 || A  P R 2 : M 2  A true L* Assumption Generation A CE CE Analysis Actual CE M 1 || M 2  P -CE for A +CE for A Teacher M 1 || M 2  P true [Cobleigh et al.]

25 Handling Multiple Components  AG-NC can be applied recursively (Cobleigh et al.) R 1 : M 1 || A  P R 2 : M 2  A M 1 || M 2  P M 1 || A 1  P M 2 || A 2  A 1 M 3  A 2 M 2 || M 3  A 1 M 1 || M 2 || M 3  P AG Proof Tree Each A i computed by a separate L* instantiation

26 Compatibility of Upgrades  Suppose assumptions are available from the old assembly  Re-validate an upgrade We propose Dynamic AGR : Incremental AGR technique Reuse previous verification results C Identical NewLost C’ Can we reuse previous assumptions directly? NO: upgrades may change the unknown U to be learned Therefore, Dynamic AGR needs Dynamic L* M 1 || A 1  P M2  A1 M2  A1 M 1 || M 2  P M’ 1 || A’ 1  P M 2  A’ 1 M’ 1 || M 2  P Upgrade Reuse? Violates L* invariant

27 Dynamic L*  Learn assumption A corresponding to U Corresponding Observation Table T  Unknown language U changes to U’  Goal: Continue learning from previous model A Start learning U’ by reusing table T  Central Idea: Re-validate T against U’ to obtain T’ T’ is valid with respect to U’

28 Dynamic L*  a ab  a b aa ab  q0q0 q1q1 a,b a b U’ = U\{b} Re-validation against U’  a ab  a b aa ab  q0q0 q1q1 a a b b Table valid for UTable valid for U’ Assumption A Assumption A’

29 Dynamic AG M 1 || A 1  P M 2  A 1 M 1 || M 2  P M’ 1 || A’ 1  P M 2  A’ 1 M’ 1 || M 2  P (Dynamic L*) Re-Validate! and Reuse Upgrade Multiple Components Heuristics for choosing order of re-validating A i ’s in proof tree

30 Implementation  ComFoRT framework for verifying concurrent C programs  Industrial benchmark ABB Inter-process Communication (IPC) software 4 main components – CriticalSection, IPCQueue, ReadMQ, WriteMQ  Evaluated on single and simultaneous upgrades WriteMQ and IPCQueue components  Properties P 1 : Write after obtaining CS lock P 2 : Correct protocol to write to IPCQueue

31 Experimental Results Upgrade# (Property) T orig (msec) (AGR from scratch) T ug (msec) (Dynamic AGR) Ipc 1 (P 1 ) Ipc 1 (P 2 ) Ipc 2 (P 1 ) Ipc 2 (P 2 ) Ipc 3 (P 1 ) Ipc 3 (P 2 ) Ipc 4 (P 1 )

32 Summary Till Now  Automated Substitutability Checking Containment and Compatibility Containment is a local check Compatibility is a global check and reuses previous verification results Handles multiple upgrades  Implementation ComFoRT framework Compatibility Check applied to IPC example

33 AGR for Simulation Conformance  Till now during AGR Notion of conformance was trace containment  Simulation conformance : local, non-det. M 1 || M 2 ≤ P  M 1 || A ≤ P; M 2 ≤ A Counterexamples are trees  We proposed automated AGR for simulation conformance of LTSs Learn from tree CEs instead of trace CEs Tree language of an LTS: set of trees that it simulates [CAV’05] [CAV’05] Automated Assume-Guarantee Reasoning for Simulation Conformance, S. Chaki, E. Clarke, N. Sinha, P. Thati, Conference on Computer-Aided Verification, 2005.

34 AGR for Simulation Conformance  Key contributions Formulate simulation check as regular tree language containment : M ≤ N  T(M)  T(N) Represent tree languages with det. tree automata Learning algorithm L T for an unknown tree lang. using teacher For AGR: Teacher uses tree language containment to answer queries  Experiments with C implementations of Secure Socket Layer (SSL) Protocol client and server

35 Results DirectAGGain Verif. Result T1T1 Mem 1 T2T2 Mem 2 Mem 1 / Mem 2 |A|MQ #CQ # Invalid * Valid * Valid * Valid * Valid * Invalid * Invalid * Invalid *

36 Related Work

37 Related Work  Behavioral Subtyping : Liskov,Wing [’01] Extend OO subtyping to behaviors Subtype Req: If  x) is provable about object x:T, then  (y) must hold for y:S, where S is a subtype of T Add invariant and history constraints to object specifications Local substitution check for an object, independent of its use by the environment

38 Related Work  Interface automata formalism: Henzinger et al. [’01-’03] Variant of Input/Output automata used to represent component abstractions and interfaces Composition and Refinement theory Composition is well-defined only if under some environment two automata don’t deadlock M refines N if M allows more inputs and exhibits less outputs than N Refinement is used as a notion of Substitution : does not capture environment use of a component

39 Related Work  Checking compatibility of Upgrades : McCamant, Ernst[’04] Compute observational summaries of components before upgrades using sample execution behaviors Internal and External summaries Pre-, Post-conditions to component method calls On an upgrade, check if new summaries refine old ones using a theorem prover Unsound component behavior abstraction No temporal information about component behaviors

40 Related Work  Automated Assume-Guarantee Reasoning (AGR) Weakest assumption [GPB02] Learning assumptions automatically [CGP03] Circular Rules for Automated AGR [BGP03] Predicate Abstraction and AGR [CCGP04] Symbolically learning assumptions [AMN04]

41 Related Work  Learning for Verification Black-box model checking [PVY99,GPY02] Learning for infinite-state model checking [HV04, VSVA04] Synthesizing interfaces of Java classes [ACMN05] Behavior-based model construction [HS04] Inductive learning for shape analysis [SLR05]

42 Proposed Research

43 Proposed Research  Substitutability Framework for SystemC  AGR for Liveness  Improving Substitutability Check  Learning-assisted CEGAR

44 SystemC  C++ library to simulate concurrency  SystemC constructs are macros in C++  Being adopted for System-level Design by industry Modular computation structure Has Communication primitives: Channels/Events Executable specifications Supports HW/SW Co-Design  Goal: Develop a Compatibility Check Framework for SystemC Designs

45 SystemC Example

46 SystemC Schematic

47 Program Structure program = process = ( comm | ctrl flow )* comm = event.notify() | event.notify_delayed() | signal.read() | signal.write() | channel. () | port.read() | port.write() | port. () ctrl flow = if (exp) then else | while (exp) variable types = bool | int | bitvector

48 Verifying SystemC Hardware Designs  Synchronous model of execution Based on Discrete-event simulation  Verification easier than C++ Finite number of concurrent processes No dynamic objects  Available open-source parsers PINAPA

49 Plan: Subst. Check for SystemC Flatten and Translate to Synch. Intermediate Language Flatten and Translate to Synch. Intermediate Language SystemC Design Build Verifier for Intermediate Language Build Verifier for Intermediate Language Build Compositional Verifier for Intermediate Language Build Compositional Verifier for Intermediate Language Perform Compatibility Check Delta-cycle semantics Events, Channels, Shared Vars, Processes Assumption automata over alphabet

50 AGR for Liveness  Till now, safety and simulation conformance Checking liveness specs involves reasoning about infinite system behaviors  Consider  -regular language containment M 1 || M 2   P  M 1 || A   P ; M 2   A “A” corresponds to an  -regular language Weakest assmp. language exists; is  -regular

51 AGR for Liveness  Develop a learning algorithm for general - regular languages Previous algorithms target a subclass Such languages have non-canonical finite representations Cannot extend L* directly; Myhill-Nerode congruence is insufficient Use a right-congruence theorem for  -regular languages (Maler and Staiger)

52 Improving Substitutability Check  Till now, only behavioral changes allowed during upgrades Interface ports of modules remain the same  Consider changes in alphabet during Dynamic Check Extend Dynamic L* to handle both unknown language change and alphabet change

53 Learning-assisted CEGAR Predicate Abstraction C1C2C1C2 M1M2M1M2 M 1 || A  P M 2  A M 1 || M 2  P Single Counterexample CE  Compositional Reasoning for Software Components C 1 and C 2  C 1 || C 2  P

54 Learning-assisted CEGAR Predicate Abstraction C1C2C1C2 M1M2M1M2 M 1 || A  P M 2  A M 1 || M 2  P Learner Spurious CE Constrains Provides Multiple Spurious CEs  Compositional Reasoning for Software Components C 1 and C 2  C 1 || C 2  P Builds spurious CE language automata