Download presentation
Presentation is loading. Please wait.
Published byAnna Grant Modified over 9 years ago
1
Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen
2
Software Testing Testing accounts for 50% of software development cost Software failure costs USA $60 billion annually –Improvement in software testing infrastructure can save one-third of this cost “The economic impacts of inadequate infrastructure for software testing”, NIST, May, 2002 Currently, software testing is mostly done manually
3
Simple C code int double(int x) { return 2 * x; } void test_me(int x, int y){ int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); }
4
Automated testing What do we need to do if we want to automatically test a piece of code (i.e., automatic unit testing)? –Figure out the environment of the application What are the inputs? What are the interfaces for interacting with other components –Automatically generate an environment for the application Automatically generate the values that come from the environment
5
Automatic Extraction of Interface Automatically determine (code parsing) –inputs to the program arguments to the entry function –variables: whose value depends on environment external objects –function calls: return value depends on the environment external function calls For simple C code –want to unit test the function test_me –int x and int y : passed as an argument to test_me forms the external environment
6
Automated Random Testing Generate a test driver automatically to simulate random environment of the extracted interface –most general environment –C – code Compile the program along with the test driver to create a closed executable. Run the executable several times to see if assertion is violated
7
Random test-driver main(){ int tmp1 = randomInt(); int tmp2 = randomInt(); test_me(tmp1,tmp2); } int double(int x) { return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Random Test Driver
8
Random test-driver main(){ int tmp1 = randomInt(); int tmp2 = randomInt(); test_me(tmp1,tmp2); } int double(int x) { return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Random Test Driver Probability of reaching abort() is extremely low
9
Limitations Hard to hit the assertion violated with random values of x and y –there is an extremely low probability of hitting assertion violation Can we do better? –Directed Automated Random Testing White box testing
10
DART (Directed Automated Random Testing) Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); }
11
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=36t1=m concrete statesymbolic stateconstraints
12
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=36, t2=-7t1=m, t2=n concrete statesymbolic stateconstraints
13
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=36, t2=-7t1=m, t2=n concrete statesymbolic stateconstraints
14
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7x=m, y=n concrete statesymbolic stateconstraints
15
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m concrete statesymbolic stateconstraints
16
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m 2m != n concrete statesymbolic stateconstraints
17
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m 2m != n concrete statesymbolic stateconstraints
18
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m 2m != n solve: 2m = n m=1, n=2 concrete statesymbolic stateconstraints
19
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=1t1=m concrete statesymbolic stateconstraints
20
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=1, t2=2t1=m, t2=n concrete statesymbolic stateconstraints
21
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=1, t2=2t1=m, t2=n concrete statesymbolic stateconstraints
22
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2x=m, y=n concrete statesymbolic stateconstraints
23
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m concrete statesymbolic stateconstraints
24
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n concrete statesymbolic stateconstraints
25
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n m != n+10 concrete statesymbolic stateconstraints
26
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n m != n+10 concrete statesymbolic stateconstraints
27
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n m != n+10 concrete statesymbolic stateconstraints
28
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n m != n+10 solve: 2m = n and m=n+10 m= -10, n= -20 concrete statesymbolic stateconstraints
29
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=-10t1=m concrete statesymbolic stateconstraints
30
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=-10, t2=-20t1=m, t2=n concrete statesymbolic stateconstraints
31
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=-10, t2=-20t1=m, t2=n concrete statesymbolic stateconstraints
32
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=-10, y=-20x=m, y=n concrete statesymbolic stateconstraints
33
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=-10, y=-20, z=- 20 x=m, y=n, z=2m concrete statesymbolic stateconstraints
34
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=-10, y=-20, z=- 20 x=m, y=n, z=2m 2m = n concrete statesymbolic stateconstraints
35
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution 2m = n m = n+10 x=-10, y=-20, z=- 20 x=m, y=n, z=2m concrete statesymbolic stateconstraints
36
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution 2m = n m = n+10 x=-10, y=-20, z=- 20 x=m, y=n, z=2m Program Error concrete statesymbolic stateconstraints
37
DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } z==y x!=y+10 NY NY Error
38
DART in a Nutshell Dynamically observe random execution and generate new test inputs to drive the next execution along an alternative path –do dynamic analysis on a random execution –collect symbolic constraints at branch points –negate one constraint at a branch point (say b) –call constraint solver to generate new test inputs –use the new test inputs for next execution to take alternative path at branch b –(Check that branch b is indeed taken next)
39
More details Instrument the C program to do both –Concrete Execution Actual Execution –Symbolic Execution and Lightweight theorem proving (path constraint solving) Dynamic symbolic analysis Interacts with concrete execution Instrumentation also checks whether the next execution matches the last prediction.
40
Experiments Tested a C implementation of a security protocol (Needham-Schroeder) with a known attack –406 lines of code –Took less than 26 minutes on a 2GHz machine to discover middle-man attack In contrast, a software model-checker (VeriSoft) and a hand-written nondeterministic model of the attacker took hours to discover the attack
41
Larger Experiment oSIP (open-source session initiation protocol) –http://www.gnu.org/software/osip/osip.html –30,000 lines of C code (version 2.0.9) –600 externally visible functions Results –crashed 65% of the externally visible functions within 1000 iterations –no nullity check for pointers Focused on oSIP parser –can externally crash oSIP server –osip_message_parse() : pass a buffer of size 2.5 MB with no 0 or “|” character –tries to copy the packet to stack using alloca(size) this fails: returns NULL pointer –this NULL pointer passed to another function does not check for nullity and crashes
42
Advantage of Dynamic Analysis over Static Analysis struct foo { int i; char c; } bar (struct foo *a) { if (a->c == 0) { *((char *)a + sizeof(int)) = 1; if (a->c != 0) { abort(); } Reasoning about dynamic data is easy Due to limitation of alias analysis “static analyzers” cannot determine that “a->c” has been rewritten –Software model checker BLAST would infer that the program is safe DART finds the error
43
Further advantages 1 foobar(int x, int y){ 2 if (x*x*x > 0){ 3 if (x>0 && y==10){ 4 abort(); 5 } 6 } else { 7 if (x>0 && y==20){ 8 abort(); 9 } 10 } 11 } static analysis based model- checkers would consider both branches –both abort() statements are reachable –false alarm Symbolic execution gets stuck at line number 2 DART finds the only error
44
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver
45
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y
46
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y solve x*x*x+ 3*x*x+9 = y to take else branch Don’t know how to solve !! –Stuck ?
47
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y solve x*x*x+ 3*x*x+9 = y to take else branch Don’t know how to solve !! –Stuck ? –NO : DART handles this smartly
48
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver
49
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z
50
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 (randomly chose a value) and proceed
51
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 and proceed take then branch with constraint 9 != y
52
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 and proceed take then branch with constraint 9 != y solve 9 = y to take else branch execute next run with x = -3 and y= 9 –got error (reaches abort)
53
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 and proceed take then branch with constraint 9 != y solve 9 = y to take else branch execute next run with x = -3 and y= 9 –got error (reaches abort) Replace symbolic expression by concrete value when symbolic expression becomes unmanageable (i.e. non-linear)
54
Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } void again_test_me(int x,int y){ z = black_box_fun(x); if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); }
55
Discussion In comparison to existing testing tools, DART is –light-weight –dynamic analysis (compare with static analysis) ensures no false alarms –concrete execution and symbolic execution run simultaneously symbolic execution consults concrete execution whenever dynamic analysis becomes intractable –real tool that works on real C programs completely automatic Software model-checkers using abstraction (SLAM, BLAST) –starts with an abstraction with more behaviors – gradually refines –static analysis approach – false alarms –DART: executes program systematically to explore feasible paths
56
Another Name for this Approach: Concolic Execution Combine concrete and symbolic execution for unit testing –Concrete + Symbolic = Concolic Concolic Execution –Use concrete execution over a concrete input to guide symbolic execution –Concrete execution helps Symbolic execution to simplify complex and unmanageable symbolic expressions by replacing symbolic values by concrete values Achieves Scalability –Higher branch coverage than random testing –No false positives or scalability issue like in symbolic execution based testing
57
CUTE: A concolic execution tool CUTE: A Concolic Unit Testing Engine –For C and Java –Handle pointers Can test data-structures Can handle heap –Supports bounded depth search –Use static analysis to find branches that can lead to assertion violation use this info to prune search space
58
CUTE Case study: SGLIB SGLIB: popular library for C data-structures Used in Xrefactory a commercial tool for refactoring C/C++ programs Found two bugs in sglib 1.0.1 –reported them to authors –fixed in sglib 1.0.2 Bug 1: –doubly-linked list library segmentation fault occurs when a non-zero length list is concatenated with a zero-length list discovered in 140 iterations ( < 1second) Bug 2: –hash-table an infinite loop in hash table is-member function –193 iterations (1 second)
59
DART vs CUTE DART handles only arithmetic constraints CUTE –Supports C with pointers, data-structures –Highly efficient constraint solver 100 -1000 times faster –arithmetic, pointers –Provides Bounded Depth-First Search and Random Search strategies –Publicly available tool that works on ALL C programs
60
Example typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Random Test Driver: random memory graph reachable from p random value for x Probability of reaching abort( ) is extremely low
61
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0 p, x=236 NULL
62
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0 p, x=236 NULL
63
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL
64
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL !(p 0 !=NULL)
65
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL p 0 =NULL solve: x 0 >0 and p 0 NULL
66
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL p 0 =NULL solve: x 0 >0 and p 0 NULL x 0 =236, p 0 634 NULL
67
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634
68
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0
69
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0 NULL
70
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0 NULL 2x 0 +1 v 0
71
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0 NULL 2x 0 +1 v 0
72
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0 NULL 2x 0 +1 v 0 solve: x 0 >0 and p 0 NULL and 2x 0 +1=v 0
73
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0 NULL 2x 0 +1 v 0 solve: x 0 >0 and p 0 NULL and 2x 0 +1=v 0 x 0 =1, p 0 3 NULL
74
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3
75
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0
76
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0 NULL
77
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0 NULL 2x 0 +1=v 0
78
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0 NULL 2x 0 +1=v 0 n0p0n0p0
79
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0 NULL 2x 0 +1=v 0 n0p0n0p0
80
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints solve: x 0 >0 and p 0 NULL and 2x 0 +1=v 0 and n 0 =p 0. p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0 NULL 2x 0 +1=v 0 n0p0n0p0
81
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints solve: x 0 >0 and p 0 NULL and 2x 0 +1=v 0 and n 0 =p 0 x 0 =1, p 0 3 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0 NULL 2x 0 +1=v 0 n0p0n0p0
82
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3
83
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3
84
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p 0 NULL p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3
85
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p 0 NULL 2x 0 +1=v 0 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3
86
CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p 0 NULL 2x 0 +1=v 0 n0=p0n0=p0 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3 Program Error
87
Pointer Inputs: Input Graph typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; }
88
Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1
89
Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1
90
Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1
91
Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1
92
Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1
93
Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1
94
Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1
95
Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1
96
CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path
97
CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically
98
CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically –Both cooperate with each other concrete execution guides the symbolic execution
99
CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically –Both cooperate with each other concrete execution guides the symbolic execution concrete execution enables symbolic execution to overcome incompleteness of theorem prover –replace symbolic expressions by concrete values if symbolic expressions become complex –resolve aliases for pointer using concrete values –handle arrays naturally
101
CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically –Both cooperate with each other concrete execution guides the symbolic execution concrete execution enables symbolic execution to overcome incompleteness of theorem prover –replace symbolic expressions by concrete values if symbolic expressions become complex –resolve aliases for pointer using concrete values –handle arrays naturally symbolic execution helps to generate concrete input for next execution –increases coverage
102
Testing Data-structures of CUTE itself Unit tested several non-standard data-structures implemented for the CUTE tool –cu_depend (used to determine dependency during constraint solving using graph algorithm) –cu_linear (linear symbolic expressions) –cu_pointer (pointer symbolic expressions) Discovered a few memory leaks and a couple of segmentation faults –these errors did not show up in other uses of CUTE –for memory leaks we used CUTE in conjunction with Valgrind
103
SGLIB: popular library for C data-structures Used in Xrefactory a commercial tool for refactoring C/C++ programs Found two bugs in sglib 1.0.1 –reported them to authors –fixed in sglib 1.0.2 Bug 1: –doubly-linked list library segmentation fault occurs when a non-zero length list is concatenated with a zero-length list discovered in 140 iterations ( < 1second) Bug 2: –hash-table an infinite loop in hash table is-member function –193 iterations (1 second)
104
Summary “DART: Directed Automated Random Testing” by Patrice Godefroid, Nils Klarlund, and Koushik Sen (PLDI’05) –handles only arithmetic constraints CUTE –Supports C with pointers, data-structures –Highly efficient constraint solver 100 -1000 times faster –arithmetic, pointers –Provides Bounded Depth-First Search and Random Search strategies –Publicly available tool that works on ALL C programs
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.