1. Modular Alternatives to Testing
Contract Checking –
Modular Alternatives to Testing
Murali Sitaraman
Computer Science
Clemson University
Clemson, SC 29634
This research is funded in part the U. S. National Science Foundation
grant CCR

2. What Are Contracts? Contracts are specifications.
They specify requirements for the component users.
They specify guarantees of the component implementers (providers) if the requirements are met by the users.

3. Component-Based Software Using Design-By-Contract Paradigm
uses
implements
uses
implements
implements
uses

4. Why Check Contracts? To pinpoint errors in a modular fashion.
Overall system testing may reveal no errors even if there are contractual errors.
When a requirement is violated, a component may have benign behavior for the particular input choices.
Contractual errors may remain hidden when the chosen component implementation requires less than specified in its contract.

5. Runtime Contract Checking
Idea is to check contracts at run-time.
Wrappers for C++ components (Edwards, Hollingsworth, Sitaraman, and Weide)
Run-time assertion checking using JML (Cheon, Edwards, Leavens, and Sitaraman)
Several others.
Contractual errors may still remain hidden, for example, when a component implementation delivers more than in its contract.
Performance ramifications.

6. DEET Project Overview

7. The DEET Project
DEET is Best Bug Repellent – New England Journal of Medicine, 2002.
DEET is Detecting Errors Efficiently without Testing.
Collaborative effort involving Clemson University, Ohio State University, and University of Tubingen, Germany.

8. Correctness Problem and Well-Known Approaches
Problem: Does the program do what is specified to do?
Formal verification objective: Prove that it does using static analysis.
Testing (and runtime checking) objective: Find errors, i.e., find mismatches between specified intent and program behavior through execution.

9. DEET vs. Verification vs. Testing
DEET is a static analysis approach like formal verification.
DEET is intended for error detection like testing.

10. DEET Goals
Other than the need for assertions, the technique should be automatic and require no manual input selection.
Should not raise false alarms, though there is no need to find all errors.
Should serve as a cost-effective and efficient prelude to both testing and verification.

11. Scalability of DEET
DEET is designed to find errors in components and systems developed under the design-by-contract paradigm.
The analysis is modular, i.e., it can analyze one component at a time.
No particular level of completeness in specifications is necessary. The error detection ability is to dependent on the quality of assertions.

12. An Example

13. Part I: Interface Contract Specification

14. Abstraction in Specification
Think of a List as a pair of mathematical strings:
A string of entries that are to the left of the current position and
A string of entries to the right
Initially, both strings are empty.

15. View of a List of Trees with Abstraction,
> , <
> )
S2 = ( <
Left
Right

16. View After Insert (T, S2)
S2 = ( < ,
> , <
> )
T =

17. Mathematical Modeling
Concept List_Template (type Entry);
uses String_Theory, …;
Type List is modeled by (
Left: String(Entry);
Right: String(Entry) );
exemplar S;
initialization ensures
S.Left = empty_string and
S.Right = empty_string;
...
end List_Template;

18. List Operations Concept List_Template (type Entry); uses …
Type List is modeled by …
Oper Insert(E: Entry; S: List);
Oper Remove(E: Entry; S: List);
Oper Advance(S: List);
Oper Reset(S: List);
Oper Advance_To_End(S: List);
Oper Left_Length(S: List): Integer;
Oper Right_Length(S: List): Integer;
Oper Swap_Rights(S1, S2: List);
end List_Template;

19. Design & Specification of Operations
Operation Insert(clears E: Entry; updates S: List);
Ensures S.Left = #S.Left and
S.Right = <#E> ° #S.Right;
Operation Remove(replaces E: Entry; updates S: List);
Requires |S.Right| > 0;
#S.Right = <E> ° S.Right;

20. Design & Specification of Operations
Operation Advance(updates S: List);
Requires |S.Right| > 0;
Ensures S.Left ° S.Right = #S.Left ° #S.Right and
|S.Left| = |#S.Left| + 1;
Operation Reset(updates S: List);
Ensures S.Left = empty_string and
S.Right = #S.Left ° #S.Right;
Operation Right_Length(restores S: List);
Ensures Right_Length = (|S.Right|);

21. Part II: Erroneous Code Example

22. A Specification of List Reverse
Operation Reverse(updates S: List);
Requires |S.Left| = 0;
Ensures S.Left = #S.RightRev and
S.Right = empty_string;

23. Example Behavior of Reverse
Left
Right
#S = ( < > , < ,
> )
S = ( < ,
> , <
> )

24. An Erroneous Implementation
Procedure Reverse (updates S: List);
decreasing |S.Right|;
Var E: Entry;
If Right_Length(S) > 0 then
Remove(E, S);
Reverse(S);
Insert(E, S);
end;
end Reverse;

25. DEET Steps for Error Detection

26. Step 1: Verification Condition Generation
What do we need to prove that the code is correct?
What can we assume?
What do we need to confirm?

27. Step 1: Verification Condition Generation
Procedure Reverse (updates S: List);
decreasing |S.Right|;
Var E: Entry;
0 Assume: |S0.Left| = 0;
If Right_Length(S) > 0 then
Remove(E, S);
Reverse(S);
Insert(E, S);
end;
5 Confirm: S5.Left = S0.RightRev and S5.Right = empty_string
end Reverse;

28. Step 1: Verification Condition Generation
State Path Assume Confirm
Condition
0 |S0.Left| = 0
If Right_Length(S) > 0 then
1 |S0.Right| > 0 S1 = S0 |S1.Right| > 0
Remove(E, S);
2 |S0.Right| > 0 S2.Left = S1 .Left and
S1.Right = <E2> ° S2 .Right
|S0.Left| = 0 and
|S2.Right| < |S0.Right|
Reverse(S);

29. Step 2: Error Hypothesis Generation
Conjunct assumptions and negation of what needs to be confirmed.
Search for a counterexample.

30. Step 2: Error Hypothesis Generation
(|P0.Right| > 0)  I
(|P0.Left| = 0)  II
(P1 = P0)  III
(P2.Left = P1.Left  P1.Right = <E2> * P2.Right)  IV
(E3 = E2  P3.Left = Rev(P2.Right)  |P3.Right| = 0)  V
(P4.Left = P3.Left  P4.Right = <E3> * P3.Right)  VI
(P5 = P4)  VII
( (P5.Left = Rev(P0.Right)  |P5.Right| = 0)) VIII

31. Step 3: Efficient Searching for Counterexamples Using Scope Restriction
Restrict the scopes of participating variables, by restricting mathematical spaces.
For variables of type Entry, suppose the scope is restricted to be 1.
Entry scope becomes: {Z0}
For variables of type Str(Entry), suppose that the length is restricted to be at most 1.
The scope of String of Entries becomes:
{Str_Empty, Str_Z0}

32. Step 3: Use Scope Restriction to Generate a Boolean Formula
Boolean formula that corresponds to P1 = P0:
((P1_Left_equals_Str_Empty  P0_Left_equals_Str_Empty)  (P1_Left_equals_Str_Z0 
P0_Left_equals_Str_Z0)) 
((P1_Right_equals_Str_Empty  P0_Right_equals_Str_Empty)  (P1_Right_equals_Str_Z0  P0_Right_equals_Str_Z0))

33. Step 3: Additional Boolean Conjuncts That Are Necessary
Unique Values (example: P0.Right)
( P0_Right_equals_Str_Empty   P0_Right_equals_Str_Z0)
String Length theorems (example: P0.Right)
(Len_P0_Right_equals_Zero  P0_Right_equals_Str_Empty)

34. Step 4: Employ a SAT Solver to find a Solution
Set these to True
P0_Left_equals_Str_Empty
P0_Right_equals_Str_Z0 …
P5_Left_equals_Str_Empty
P5_Right_equals_Str_Z0
Set these to False
P0_Left_equals_Str_Z0
P0_Right_equals_Str_Empty

35. How Fast Is DEET?
We used Sinz/Kuechlin solver that can handle non-CNF formulae easily.
It took the solver less than a fraction of a second to find the counterexample.
We tried it on an example with 2000 statements and 6000 variables. It took the solver less than 2 seconds to find two counterexamples on a 1.2MHz Athlon PC .

36. Error in the Example Can Be Revealed with Simpler Specs
Type List is modeled by (
Left_Len, Right_Len: N; );
exemplar S;
initialization ensures
S.Left_Len = 0 and
S.Right_Len = 0;
Operation Insert(clears E: Entry; updates S: List);
Ensures S.Left_Len = #S.Left_Len and
S.Right_Len = #S.Right_Len + 1;
-- other operations

37. But Simpler Spec Cannot Reveal the Error in this Code
Procedure Reverse (updates S: List);
decreasing |S.Right|;
Var E, F: Entry;
If Right_Length(S) > 0 then
Remove(E, S);
Reverse(S);
Insert(F, S);
Advance(S);
end;
end Reverse;

38. Related Work
Jackson’s Alloy work and his hypothesis: A significant number of errors can be found by examining the software within a “small scope of inputs” [Jackson & Vaziri 00, …]
Model checking and symbolic execution work
Others

39. Status and Future Directions
Our thesis: DEET can be an efficient and cost-effective prelude to more exhaustive testing or verification
Its scalability and utility for error detection needs to be shown through practical experimentation

40. Other Work Language development (RESOLVE, DirectJava, others)
Tool/environment development
Software component specification, design, development, and verification
Performance constraint specification and verification
Undergraduate CS education

41. Modular Alternatives to Testing
Contract Checking –
Modular Alternatives to Testing
Murali Sitaraman
Computer Science
Clemson University
Clemson, SC 29634
This research is funded in part the U. S. National Science Foundation
grant CCR