272: Software Engineering Fall 2008 Instructor: Tevfik Bultan Lectures 13: Modularity, Interfaces and Verification

Joint Work with My Students Action Language Verifier (ALV) –Tuba Yavuz-Kahveci, University of Florida, Gainesville Web Service Analysis Tool (WSAT) –Xiang Fu, Hofstra University (co-advised with Jianwen Su) Design for verification –Aysu Betin-Can, Middle East Technical University Interface Grammars –Graham Hughes, PhD candidate

Outline Motivation PART 1: Verification of Synchronization Policies in Concurrent Programs via Interfaces PART 2: Verification of Conversations among Web Services via Interfaces PART 3: Modular Verification with Interface Grammars Conclusions

Model Checking Software Model checking –An automated software verification technique –Exhaustive exploration of the state space of a program to find bugs Systematically explore all possible behaviors of a program –look for violations of the properties of interest assertion violations, deadlock Software model checkers: Verisoft, Java PathFinder (JPF), SLAM, BLAST, CBMC

Two Challenges in Model Checking State space explosion –Exponential increase in the state space with increasing number of variables and threads State space includes everything: threads, variables, control stack, heap Environment generation –Finding models for parts of software that are either not available for analysis, or are outside the scope of the model checker

Modular Verification Modularity is key to scalability of any verification technique –Moreover, it can help in isolating the behavior you wish to focus on, removing the parts that are beyond the scope of your verification technique Modularity is also a key concept for successful software design –The question is finding effective ways of exploiting the modularity in software during verification

Interfaces for Modularity How do we do modular verification? –Divide the software to a set of modules –Check each module in isolation How do we isolate a module during verification/testing? –Provide stubs representing other modules (environment) How do we get the stubs representing other modules? –Write interfaces Interfaces specify the behavior of a module from the viewpoint of other modules Generate stubs from the interfaces

Interfaces and Modularity: Basic Idea 1.Write interface specifications for the modules 2.Automatically generate stubs from the interface specifications 3.Automatically generated stubs provide the environment during modular verification

Three Applications I will talk about three different instantiations of this basic idea: 1.Verification of synchronization policies in concurrent programs using finite state interfaces 2.Verification of conversations among web services using finite state interfaces 3.Verification of sequential interactions using interface grammars

PART 1 Concurrency Controller Pattern for Synchronization

An Infinite State Model Checker Action Language Parser Model Checker OmegaLibraryCUDDPackage MONA Composite Symbolic Library PresburgerArithmeticManipulator BDDManipulatorAutomataManipulator Action Language Specification + CTL property Counter-exampleVerified Not sure Action Language Verifier (ALV)

What Can One Do with ALV? Check if a specification satisfies a CTL property For example, check a Read-Write lock implementation integer nr; boolean busy; initial: !busy and nr=0; r_enter: [!busy] nr := nr+1; r_exit: nr := nr-1; w_enter: [!busy && nr=0] busy := true; w_exit: busy := false;

Read-Write Lock in Action Language module main() integer nr; boolean busy; restrict: nr>=0; initial: nr=0 and !busy; module ReaderWriter() enumerated state {idle, reading, writing}; initial: state=idle; r_enter: state=idle and !busy and nr’=nr+1 and state’=reading; r_exit: state=reading and nr’=nr-1 and state’=idle; w_enter: state=idle and !busy and nr=0 busy’ and state’=writing; w_exit: state=writing and !busy’ and state’=idle; ReaderWriter: r_enter | r_exit | w_enter | w_exit; endmodule main: ReaderWriter*(); spec: invariant(busy => nr=0) spec: invariant(busy => eventually(!busy)) endmodule S : Cartesian product of variable domains defines variable domains defines the set of states the set of states I : Predicates defining the initial states the initial states R : Atomic actions of a single process a single process R : Transition relation of a process, defined as asynchronous composition of its atomic actions R : Transition relation of main, defined as asynchronous composition of finite but arbitrary number of reader-writer modules

Arbitrary Number of Threads? How do we check arbitrary number of threads? Counting abstraction –Create an integer variable for each thread state –Each variable counts the number of threads in a particular state –Generate updates and guards for these variables based on the specification Counting abstraction is automated

Parameterized Read-Write Lock module main() integer nr; boolean busy; parameterized integer numReaderWriter; restrict: nr>=0 and numReaderWriter>=1; initial: nr=0 and !busy; module ReaderWriter() integer idle, reading, writing; initial: idle=numReaderWriter; r_enter: idle>0 and !busy and nr’=nr+1 and idle’=idle-1 and reading’=reading+1; r_exit: reading>0 and nr’=nr-1 and reading’=reading-1 and idle’=idle+1; w_enter: idle>0 and !busy and nr=0 and busy’ and idle’=idle-1 and writing’=writing+1; w_exit: writing>0 and !busy’ and writing’=writing-1 and idle’=idle+1 ReaderWriter: r_enter | r_exit | w_enter | w_exit; endmodule main: ReaderWriter(); spec: invariant(busy => nr=0) spec: invariant(busy => eventually(!busy)) endmodule

Read-Write Lock Verification with ALV IntegersBooleansCons. Time (secs.) Ver. Time (secs.) Memory (Mbytes) RW RW RW RW RW RW-P

Read-Write Lock in Java class ReadWriteLock { private Object lockObj; private int totalReadLocksGiven; private boolean writeLockIssued; private int threadsWaitingForWriteLock; public ReadWriteLock() { lockObj = new Object(); writeLockIssued = false; } public void getReadLock() { synchronized (lockObj) { while ((writeLockIssued) || (threadsWaitingForWriteLock != 0)) { try { lockObj.wait(); } catch (InterruptedException e) { } } totalReadLocksGiven++; } } public void getWriteLock() { synchronized (lockObj) { threadsWaitingForWriteLock++; while ((totalReadLocksGiven != 0) || (writeLockIssued)) { try { lockObj.wait(); } catch (InterruptedException e) { // } } threadsWaitingForWriteLock--; writeLockIssued = true; } } public void done() { synchronized (lockObj) { //check for errors if ((totalReadLocksGiven == 0) && (!writeLockIssued)) { System.out.println(" Error: Invalid call to release the lock"); return; } if (writeLockIssued) writeLockIssued = false; else totalReadLocksGiven--; lockObj.notifyAll(); } } } How do we translate this to Action Language? Action Language Verifier Verification of Synchronization in Java Programs

A Design for Verification Approach Our design for verification approach is based on the following principles: 1.Use of design patterns that facilitate automated verification 2.Use of stateful, behavioral interfaces which isolate the behavior and enable modular verification 3.An assume-guarantee style modular verification strategy that separates verification of the behavior from the verification of the conformance to the interface specifications 4.A general model checking technique for interface verification 5.Domain specific and specialized verification techniques for behavior verification

Controller Shared Concurrency Controller Pattern ThreadA ThreadB StateMachine Controller -var1 -var2 +action1() +action2() Action +blocking() +nonblocking() -GuardedExecute SharedStub +a() +b() Shared +a() +b() GuardedCommand +guard() +update() int GuardedCommand ControllerStateMachine +action1() +action2() used at runtime used during interface verification used both times Helper classes

Concurrency Controller Pattern Avoids usage of error-prone Java synchronization primitives: synchronize, wait, notify Separates controller behavior from the threads that use the controller –Supports a modular verification approach that exploits this modularity for scalable verification

class Action{ protected final Object owner; … private boolean GuardedExecute(){ boolean result=false; for(int i=0; i<gcV.size(); i++) try{ if(((GuardedCommand)gcV.get(i)).guard()){ ((GuardedCommand)gcV.get(i)).update(); result=true; break; } }catch(Exception e){} return result; } public void blocking(){ synchronized(owner) { while(!GuardedExecute()) { try{owner.wait();} catch (Exception e){} } owner.notifyAll(); } } public boolean nonblocking(){ synchronized(owner) { boolean result=GuardedExecute(); if (result) owner.notifyAll(); return result; } } class RWController implements RWInterface{ int nR; boolean busy; final Action act_r_enter, act_r_exit; final Action act_w_enter, act_w_exit; RWController() {... gcs = new Vector(); gcs.add(new GuardedCommand() { public boolean guard(){ return (nR == 0 && !busy);} public void update(){busy = true;}} ); act_w_enter = new Action(this,gcs); } public void w_enter(){ act_w_enter.blocking();} public boolean w_exit(){ return act_w_exit.nonblocking();} public void r_enter(){ act_r_enter.blocking();} public boolean r_exit(){ return act_r_exit.nonblocking();} } Reader-Writer Controller This helper class is provided. No need to rewrite it!

Controller Interfaces A controller interface defines the acceptable call sequences for the threads that use the controller Interfaces are specified using finite state machines public class RWStateMachine implements RWInterface{ StateTable stateTable; final static int idle=0,reading=1,writing=2; public RWStateMachine(){... stateTable.insert("w_enter",idle,writing); } public void w_enter(){ stateTable.transition("w_enter"); }... } writing reading idle r_enter r_exit w_exit w_enter

Interface Machine Thread 1Thread 2Thread n Thread 1 Controller Shared Data Interface Machine Thread 2 Interface Machine Thread n Thread Modular Interface Verification Concurrent Program Controller Behavior Modular Behavior Verification Modular Design / Modular Verification Interface

Behavior Verification Analyzing properties (specified in CTL) of the synchronization policy encapsulated with a concurrency controller and its interface –Verify the controller properties assuming that the user threads adhere to the controller interface Behavior verification with Action Language Verifier –We wrote a translator which translates controller classes to Action Language –Using counting abstraction we can check concurrency controller classes for arbitrary number of threads

Interface Verification A thread is correct with respect to an interface if all the call sequences generated by the thread can also be generated by the interface machine –Checks if all the threads invoke controller methods in the order specified in the interfaces –Checks if the threads access shared data only at the correct interface states

Interface Verification Interface verification with Java PathFinder –Verify Java implementations of threads –Correctness criteria are specified as assertions Look for assertion violations Assertions are in the StateMachine and SharedStub –Performance improvement with thread Isolation thread modular verification

Thread Isolation: Part 1 Interaction among threads Threads can interact with each other in only two ways: –invoking controller actions –Invoking shared data methods To isolate the threads –Replace concurrency controllers with controller interface state machines –Replace shared data with shared stubs

Thread Isolation: Part 2 Interaction among a thread and its environment Modeling thread’s calls to its environment with stubs –File I/O, updating GUI components, socket operations, RMI call to another program Replace with pre-written or generated stubs Modeling the environment’s influence on threads with drivers –Thread initialization, RMI events, GUI events Enclose with drivers that generate all possible events that influence controller access

Concurrent Program Controller Classes Thread Classes Controller Interface Machine Controller Behavior Machine Java Path Finder Action Language Verifier Thread Isolation Thread Class Counting Abstraction Interface Verification Behavior Verification Verification Framework

A Case Study: TSAFE Tactical Separation Assisted Flight Environment (TSAFE) functionality: 1.Display aircraft position 2.Display aircraft planned route 3.Display aircraft future projected route trajectory 4.Show conformance problems between planned and projected route

Server Computation Flight Database Graphical Client > 21,057 lines of code with 87 classes Radar feed > User EventThread Feed Parser Timer TSAFE Architecture

Reengineering TSAFE Found all the synchronization statements in the code ( synchronize, wait, notify, notifyAll ) Identified 6 shared objects protected by these synchronization statements Used 2 instances of a reader-writer controller and 3 instances of a mutex controller for synchronization In the reengineered TSAFE code the synchronization statements appear only in the Action helper class provided by the concurrency controller pattern

Behavior Verification Performance RW Mutex Barrier BB-RW BB-Mutex ControllerTime(sec)Memory (MB)P-Time (sec)P-Memory (MB) P denotes parameterized verification for arbitrary number of threads

Interface Verification Performance ThreadTime (sec)Memory (MB) TServer-Main TServer-RMI TServer-Event TServer-Feed TClient-Main TClient-RMI TClient-Event

Effectiveness in Finding Faults Created 40 faulty versions of TSAFE by fault seeding Each version had at most one interface fault and at most one behavior fault –14 behavior and 26 interface faults Among 14 behavior faults ALV identified 12 of them –2 uncaught faults were spurious Among 26 interface faults JPF identified 21 of them –2 of the uncaught faults were spurious –3 of the uncaught faults were real faults that were not caught by JPF

Falsification Performance TServer-RMI TServer-Event TServer-Feed TClient-RMI TClient-Event ThreadTime (sec)Memory (MB) RW RW RW-P Mutex Mutex Mutex-p Concurrency ControllerTime (sec)Memory (MB)

Conclusions ALV performance –Cost of parameterized verification was somewhere between concrete instances with 8 and 16 threads –Falsification performance was better than verification Completeness of the controller properties –Effectiveness of behavior verification by ALV critically depends on the completeness of the specified properties Concrete vs. parameterized behavior verification –When no faults are found, the result obtained with parameterized verification is stronger –However for falsification we observed that concrete instances were as effective as parameterized instances

Conclusions JPF performance –Typically falsification performance is better than verification performance –In some cases faults caused execution of new code causing the falsification performance to be worse than verification performance Thread isolation –Automatic environment generation for threads result in too much non-determinism and JPF runs out of memory –Dependency analysis was crucial for mitigating this Deep faults were difficult to catch using JPF –Three uncaught faults were created to test this

Conclusions Unknown shared objects –The presented approach does not handle this problem –Using escape analysis may help We could not find a scalable and precise escape analysis tool Environment generation –This is the crucial problem in scalability of the interface verification –Using a design for verification approach for environment generation may help

PART 2 Peer Controller Pattern for Web Services

Web Services Loosely coupled, interaction through standardized interfaces Standardized data transmission via XML Asynchronous messaging Platform independent (.NET, J2EE) Data Type Service Composition Message WSBPEL Web Service Standards Implementation Platforms Microsoft.Net, Sun J2EE WSDL SOAP XML Schema XML WSCDL Interaction

Web Service Conversations A composite web service consists of –a finite set of peers –and a finite set of message classes The messages among the peers are exchanged using reliable and asynchronous messaging –FIFO and unbounded message queues A conversation is a sequence of messages generated by the peers during an execution

Properties of Conversations The notion of conversation enables us to reason about temporal properties of the composite web services LTL framework extends naturally to conversations –LTL temporal operators X (neXt), U (Until), G (Globally), F (Future) –Atomic properties Predicates on message classes (or contents) Example: G ( payment  F receipt ) Model checking problem: Given an LTL property, does the conversation set satisfy the property?

Conversation Protocol query suggest query suggest confirm G(query  F(confirm)) ? ?query !suggest Agent !query ?suggest !reserve !query ?suggest Customer Hotel AgentCustomerHotel query suggest reserve confirm Input Queue... Conversation reserve ?confirm !confirm ? G(query  F(confirm)) ?query !suggest ?reserve Top-Down vs. Bottom-Up

Realizability Analysis Conversation protocol specifies the global communication behavior –How do we implement the peers? How do we obtain the contracts that peers have to obey from the global contract specified by the conversation protocol? Project the global protocol to each peer –By dropping unrelated messages for each peer A conversation protocol is realizable if the following are equal: –Conversations specified by the conversation protocol –Conversations generated by the projected services

Synchronizability Analysis A composite web service is synchronizable, if its conversation set does not change –when asynchronous communication is replaced with synchronous communication If a composite web service is synchronizable we can check its properties about its conversations using synchronous communication semantics –For finite state peers this is a finite state model checking problem

BPEL to GFSA Guarded automata GFSA to Promela (bounded queue) BPEL Web Services Promela Synchronizability Analysis GFSA to Promela (synchronous communication) Intermediate Representation Conversation Protocol Front End Realizability Analysis Guarded automaton skip GFSA parser success fail GFSA to Promela (single process, no communication) success fail AnalysisBack End (bottom-up) (top-down) Verification Languages Web Service Analysis Tool (WSAT)

Checking Service Implementations There are some problems: People write web service implementations using programming languages such as Java, C#, etc. Synchronizability analysis works on state machine models How do we generate the state machines from the Java code? Synchronizability Analysis Checking Service Implementations

Design for Verification Approach Use the same principles: 1.Use of design patterns that facilitate automated verification 2.Use of stateful, behavioral interfaces which isolate the behavior and enable modular verification 3.An assume-guarantee style modular verification strategy that separates verification of the behavior from the verification of the conformance to the interface specifications 4.A general model checking technique for interface verification 5.Domain specific and specialized verification techniques for behavior verification

ApplicationThreadCommunicationInterface StateMachine CommunicatorCommunicationController PeerServletThreadContainer sessionId Peer Controller Pattern used at runtime used during interface verification used both times Red Bordered classes are the ones the user has to implement

Peer Controller Pattern Eases the development of web services Uses Java API for XML messaging (JAXM) –Asynchronous communication among peers Supported by a modular verification technique –Behavior Verification: Checks properties of conversations of a web service composed of multiple peers assuming that peers behave according to their interfaces –Interface Verification: Checks if each peer behaves according to its interface

Interface Machine Peer 1 Peer 2Peer n Peer 1 Interface Machine Peer 2 Interface Machine Peer n Peer Modular Interface Verification Composite Service Conversation Behavior Modular Conversation Verification Modular Design / Modular Verification interface

Composite Service Peer State Machine Promela Java Path Finder Spin Peer Code Interface Verification Verification Framework Thread Peer State Machines WSAT Synchronizability Analysis Conversation Verification Promela Translation

Behavior Verification Uses WSAT for synchronizability analysis Uses Spin model checker for conversation verification –Automated translation to Promela using WSAT Spin is a finite state model checker –We have to bound the channel sizes, session numbers, message types Synchronizability analysis –Enables us to verify web services efficiently by replacing communication channels with channels of size 0 (i.e., synchronous communication) –The verification results hold for unbounded channels

Interface Verification If the call sequence to the Communicator class is accepted by the state machine specifying the interface, then the peer implementation conforms to the behavior in the contract Uses JPF model checker

Interface Verification Isolated check of individual peer implementations –CommunicationController is replaced with CommunicatorInterface –Drivers simulating other peers are automatically generated State Space reduction –Usage of stubs –Each session is independent just need to check each peer for one session

Examples We used this approach to implement several simple web services –Travel agency –Loan approver –Product ordering Performance of both interface and behavior verification were reasonable

Interface Verification Interface Verification with JPF for Loan Approver ThreadsT (sec)M (MB) Customer Loan Approver Risk Assesor

Behavior Verification Sample Property: Whenever a request with a small amount is sent, eventually an approval message accepting the loan request will be sent. Loan Approval system has 154 reachable states – because queue lengths never exceeds 1 Behavior verification used <1 sec and 1.49 MB SPIN requires restricted domains –Have to bound the channel sizes  bounded message queues In general there is no guarantee these results will hold for other queue sizes –Using synchronizability analysis we use queues of size 0 and still guarantee that the verification results hold for unbounded queues!

Conclusions We were able to use our design for verification approach based on design patterns and behavioral interfaces in different domains Use of domain specific behavior verification techniques has been very effective Interface verification is challenging Model checking research resulted in various verification techniques and tools which can be customized for specific classes of software systems Automated verification techniques can scale to realistic software systems using design for verification approach

Conclusions Once the behavior is isolated (using concurrency controller or peer controller patterns) behavior verification was quite efficient Interface verification was very hard It is necessary to find effective behavioral interface specification and verification techniques

PART 3 Interface Grammars

Interface Grammars Component A Component B Interface Grammar Interface Compiler Component B Stub Component A Model Checker Interface Grammar

An Example An interface grammar for transactions –Specifies the appropriate ordering for method calls to a transaction manager –Method calls are the terminal symbols of the interface grammar S tart → Base Base →begin Tail Base |ε |ε Tail →commit |rollback |rollback

An Example Consider the call sequence begin rollback begin commit Here is a derivation: Start  Base  begin Tail Base  begin rollback Base  begin rollback begin Tail Base  begin rollback begin commit Base  begin rollback begin commit S tart → Base Base →begin Tail Base |ε |ε Tail →commit |rollback |rollback

Another Example This interface can also be specified as a Finite State Machine (FSM) However, the following grammar, which specifies nested transactions, cannot be specified as a FSM Start → Base Base →begin Base Tail Base |ε |ε Tail →commit |rollback |rollback begin commit rollback

Yet Another Example setrollbackonly which forces all the pending transactions to finish with rollback instead of commitLet’s add another method called setrollbackonly which forces all the pending transactions to finish with rollback instead of commit We achieve this by extending the interface grammars with semantic predicates and semantic actionsWe achieve this by extending the interface grammars with semantic predicates and semantic actions Start →«r:=false; l:=0» Base Base →begin «l:=l+1» Base Tail «l:=l-1; if l=0 then r:=false» Base «l:=l-1; if l=0 then r:=false» Base |setrollbackonly «r:=true» Base |setrollbackonly «r:=true» Base |ε |ε Tail →«r=false» commit |rollback |rollback

Interface Grammar Translation Our interface compiler translates interface grammars to executable code: –the generated code is the stub for the component The generated code is a parser that –parses the incoming method calls –while making sure that the incoming method calls conform to the interface grammar specification

Verification with Interface Grammars Interface Grammar Interface Compiler Program Model Checker Top-down parser parse table semantic predicates and semantic actions parser stack Component Stub method invocation (lookahead)

A Case Study We wrote an interface grammar for the EJB 3.0 Persistence API – This is an API specification for mapping Java object graphs to a relational database – Hibernate is an implementation of this API Hibernate distribution contains several example test clients that are designed to fail and test exceptional behavior by violating the interface specification

A Case Study, Continued We used these simple clients to check the fidelity of the stub generated from our interface specification – We used the JPF software model checker None of these examples can run under JPF directly Time taken to develop the interface was dominated by the need to understand EJB Persistence first – about a couple of hours

Experiments: Falsification sec. # obj.# iter Client 1Client 2Client 3 Client 4

Experiments: Verification sec. # obj.# iter Client 1Client 2Client 3 Client 4

A Case Study, Continued For these simple clients, interface violations can be detected by JPF in a couple of seconds using the EJB stub generated by our interface compiler –Falsification time does not increase with the number of operations executed or the number of objects created by the clients When we fix the errors, JPF is able to verify the absence of interface violations –Verification time increases with the number of operations executed or the number of objects created by the clients

Interface Grammars: Uni/Bidirectional Interface grammars can be –Unidirectional: No callbacks –Bidirectional: Need to handle Callbacks CallerCallee Interface Comp AComp B Interface

Interface Grammars: Client/Server Interface grammars can be used for –Client verification: Generate a stub for the server –Server verification: Generate a driver for the server Interface Compiler Interface Client Stub Server Driver

Semantic Elements in JML To handle both client and server side verification –We need to generate stubs and drivers from the same specification Semantic predicate in one direction becomes a semantic action in the other direction and visa versa We focused on a subset of Java Modeling Language –A restricted subset that is reversable –Semantic predicates and actions are written in this subset –Interface compiler automatically generates code from them both for client and server side verification

Interface Grammars and Data A crucial part of the interface specification is specifying the allowable values for the method arguments and generating allowable return values Approach 1: These can be specified in the semantic actions and semantic predicates of the interface grammars Approach 2: Can we specify the constraints about the arguments and return values using the grammar rules? –Yes, grammar productions can be used to specify the structure of most recursive data structures.

Checking Arguments A crucial part of the interface specification is specifying the allowable values for the method arguments and generating allowable return values In what I discussed so far all these are done in the semantic actions and semantic predicates The question is, can we specify the constraints about the arguments and return values using the grammar rules –Recursive data structures are especially good candidates for this!

Shape Types Shape types [Fradet, Metayer, POPL 97] provide a formalism for specifying recursive data structures It is a specification formalism based on graph grammars Shape types can be used to specify the connections among the heap allocated objects Objects become the parameters of the nonterminals and the constraints on the connections among the objects are specified on the right-hand-sides of the grammar rules (similar to semantic predicates)

Shape Type for Doubly Linked List 12 next prev p 3 next prev 4 next prev next prev Doubly →p x, prev x null, L x L x →next x y, prev y x, L y L x →next x null p 1, prev 1 null, L 1 Doubly  p 1, prev 1 null, L 1  next 1 2, prev 2 1, L 2  next 2 3, prev 3 2, L 3  next 3 4, prev 4 3, L 4  next 4 null

Shape Type for Binary Tree 1 right left p right left right left 4 right left Bintree →p x, B x B x →left x y, right x z, B y, B z B x →left x null, right x null

Extension to Interface Grammars In order to support shape types we extend the interface grammars as follows: –We allow nonterminals with parameters This extension is sufficient since the constraints about the connections among the objects can be stated using semantics predicates and semantic actions

Interface Grammars + Shape Types Doubly →p x, prev x null, L x L x →next x y, prev y x, L y L x →next x null Doubly[x] → ensure x == \new(Node) && x.getPrev() == null; L[x] L[x] → Node y; ensure y == \new(Node) && x.getNext() == y && y.getPrev() == x; L[y] | ensure x.getNext() == null;

Interface Grammars + Shape Types Bintree →p x, B x B x →left x y, right x z, B y, B z B x →left x null, right x null Bintree[x] → ensure x == \new(Node); B[x] B[x] → Node y, z; ensure y == \new(Node) && z == \new(Node) && x.getLeft() == y && x.getRight() == z ; B[y] B[z] | ensure x.getLeft() == null && x.getRight() == null;

Objection Generation vs. Validation The use of shape types in interface grammars has two purposes –For the objects that are passed as method arguments we need to check that their shape is allowed by the shape type We call this object validation –For the objects that are returned by the component we need to generate an object that is allowed by the shape type We call this object generation

Object Generation vs. Validation Object generation and validation tasks are broadly symmetric –The set of nonterminals and productions used for object generation and validation are the same and are dictated by the shape type specification –In object generation semantic actions are used to set the fields of objects to appropriate values dictated by the shape type specification –In object validation these are constraints that are checked using semantic predicates specified as guards Given the semantic elements specified in JML, our interface compile generates code both for object generation and validation

Object Generation vs. Validation There is a minor problem with object validation In shape type specifications, the assumption is that there is no aliasing among the objects unless it is explicitly specified This assumption is easy to enforce during object generation since every new statement creates a new object that has nothing else pointing to it In order to enforce the same constraint during object validation we need to make sure that there is no unspecified aliasing –This can be enforced by using a hash-set for storing and propagating all the observed objects

Modular Verification of Web Services We applied our modular verification approach based on interface grammars to both client and server side verification of Web services

Interface Grammars and Web Services Our approach: 1.A WSDL-to-interface grammar translator automatically generates grammar productions that generate and/or validate XML arguments and return values 2.User adds control flow constraints by modifying the grammar 3.Interface compiler automatically generates a stub for client side verification and a driver for server-side verification

Interface Grammars for Web Services

Another Case Study: AWS-ECS We tested the Amazon E-Commerce Web Service (AWS- ECS) using our approach AWS-ECS WSDL specification lists 40 operations –that provide differing ways of searching Amazon’s product database We focus on the core operations: –ItemSearch, CartCreate, CartAdd, CartModify, CartGet, CartClear

Client-side Verification For client verification we used a demonstration client provided by Amazon This client does not check any constraints such as –You should not try to insert an item to a shopping cart before creating a shopping cart When such requests are sent to AWS-ECS they would return an error message Using our approach we can easily check if the client allows such erroneous requests Falsification time changes with the type of faults we are looking for (data or control errors), changes from 10 to 60 seconds

AWS-ECS: Server Verification Our interface compiler automatically generates a driver that sends sequences of requests to AWS-ECS server and checks that the return values conform to the interface specification The driver is a sentence generator –It generates sequences of SOAP requests based on the interface specification We used two algorithms for sentence generation: –A random sentence generation algorithm –Purdom’s algorithm: A directed sentence generation algorithm that guarantees production coverage

Directed Sentence Generation Number of sentences generated: 5 Average derivation length: 24 Average number of SOAP requests/responses: 3.8 Verification time: 20 seconds

Random Sentence Algorithm Number of sentences generated: 100 Average derivation length: 17.5 Average number of SOAP requests/responses: 3.2

Server-side verification We found two errors during server side verification –Errors were discovered within 10 seconds These errors indicate a mismatch between the interface specification and the server implementation It may mean that we misunderstood the description of the Web service interface It may also mean that there is an error in the service implementation

Conclusions Modular verification is a necessity Interfaces are crucial for modular verification Interface grammars provide a new specification mechanism for interfaces Interface grammars can be used for automated stub and driver generation leading to modular verification

Conclusions Behavioral interfaces can be useful both –From software design perspective by enabling better modular designs –From verification perspective by enabling more efficient modular verification The challenge is to find behavioral interface specification mechanisms that serve both of these goals

Related Work: Modular Verification Clarke, Long, McMillan, Compositional Model Checking Henzinger, Qadeer, Rajamani, Assume Guarantee Reasoning in Model Checking Flanagan, Qadeer, Thread-Modular Model Checking Krishnamurthi, Fisler, Modular Verification of Feature Oriented Programs

Related Work: Design for Verification Meyer, Design by Contract Flanagan, Leino, et al. ESC Java Mehlitz, Penix, Design for Verification Using Design Patterns Sarna-Starosta, Stirewalt, Dillon, Design for Verification for Synchronization

Related Work: Interfaces L. de Alfaro and T. A. Henzinger. Interface automata. O. Tkachuk, M. B. Dwyer, and C. Pasareanu. Automated environment generation for software model checking. T. Ball and S. K. Rajamani. SLAM interface specification language. G. T. Leavens et al.: JML

Related: Grammar-based Testing A. G. Duncan, J. S. Hurchinson: Using attributed grammars to test designs and implementations P. M. Maurer: Generating test data with enhanced context free grammars P. M. Maurer: The design and implementation of a grammar-based data generator E. G. Sirer and B. N. Bershad: Using production grammars in software testing