Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen.

Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen

Software Testing Testing accounts for 50% of software development cost Software failure costs USA $60 billion annually –Improvement in software testing infrastructure can save one-third of this cost “The economic impacts of inadequate infrastructure for software testing”, NIST, May, 2002 Currently, software testing is mostly done manually

Simple C code int double(int x) { return 2 * x; } void test_me(int x, int y){ int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); }

Automated testing What do we need to do if we want to automatically test a piece of code (i.e., automatic unit testing)? –Figure out the environment of the application What are the inputs? What are the interfaces for interacting with other components –Automatically generate an environment for the application Automatically generate the values that come from the environment

Automatic Extraction of Interface Automatically determine (code parsing) –inputs to the program arguments to the entry function –variables: whose value depends on environment external objects –function calls: return value depends on the environment external function calls For simple C code –want to unit test the function test_me –int x and int y : passed as an argument to test_me forms the external environment

Automated Random Testing Generate a test driver automatically to simulate random environment of the extracted interface –most general environment –C – code Compile the program along with the test driver to create a closed executable. Run the executable several times to see if assertion is violated

Random test-driver main(){ int tmp1 = randomInt(); int tmp2 = randomInt(); test_me(tmp1,tmp2); } int double(int x) { return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Random Test Driver

Random test-driver main(){ int tmp1 = randomInt(); int tmp2 = randomInt(); test_me(tmp1,tmp2); } int double(int x) { return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Random Test Driver Probability of reaching abort() is extremely low

Limitations Hard to hit the assertion violated with random values of x and y –there is an extremely low probability of hitting assertion violation Can we do better? –Directed Automated Random Testing White box testing

DART (Directed Automated Random Testing) Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); }

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=36t1=m concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=36, t2=-7t1=m, t2=n concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7x=m, y=n concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m 2m != n concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m 2m != n solve: 2m = n m=1, n=2 concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=1t1=m concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=1, t2=2t1=m, t2=n concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2x=m, y=n concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n m != n+10 concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n m != n+10 solve: 2m = n and m=n+10 m= -10, n= -20 concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=-10t1=m concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=-10, t2=-20t1=m, t2=n concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=-10, y=-20x=m, y=n concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=-10, y=-20, z=- 20 x=m, y=n, z=2m concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=-10, y=-20, z=- 20 x=m, y=n, z=2m 2m = n concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution 2m = n m = n+10 x=-10, y=-20, z=- 20 x=m, y=n, z=2m concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution 2m = n m = n+10 x=-10, y=-20, z=- 20 x=m, y=n, z=2m Program Error concrete statesymbolic stateconstraints

DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } z==y x!=y+10 NY NY Error

DART in a Nutshell Dynamically observe random execution and generate new test inputs to drive the next execution along an alternative path –do dynamic analysis on a random execution –collect symbolic constraints at branch points –negate one constraint at a branch point (say b) –call constraint solver to generate new test inputs –use the new test inputs for next execution to take alternative path at branch b –(Check that branch b is indeed taken next)

More details Instrument the C program to do both –Concrete Execution Actual Execution –Symbolic Execution and Lightweight theorem proving (path constraint solving) Dynamic symbolic analysis Interacts with concrete execution Instrumentation also checks whether the next execution matches the last prediction.

Experiments Tested a C implementation of a security protocol (Needham-Schroeder) with a known attack –406 lines of code –Took less than 26 minutes on a 2GHz machine to discover middle-man attack In contrast, a software model-checker (VeriSoft) and a hand-written nondeterministic model of the attacker took hours to discover the attack

Larger Experiment oSIP (open-source session initiation protocol) –http://www.gnu.org/software/osip/osip.html –30,000 lines of C code (version 2.0.9) –600 externally visible functions Results –crashed 65% of the externally visible functions within 1000 iterations –no nullity check for pointers Focused on oSIP parser –can externally crash oSIP server –osip_message_parse() : pass a buffer of size 2.5 MB with no 0 or “|” character –tries to copy the packet to stack using alloca(size) this fails: returns NULL pointer –this NULL pointer passed to another function does not check for nullity and crashes

Advantage of Dynamic Analysis over Static Analysis struct foo { int i; char c; } bar (struct foo *a) { if (a->c == 0) { *((char *)a + sizeof(int)) = 1; if (a->c != 0) { abort(); } Reasoning about dynamic data is easy Due to limitation of alias analysis “static analyzers” cannot determine that “a->c” has been rewritten –Software model checker BLAST would infer that the program is safe DART finds the error

Further advantages 1 foobar(int x, int y){ 2 if (x*x*x > 0){ 3 if (x>0 && y==10){ 4 abort(); 5 } 6 } else { 7 if (x>0 && y==20){ 8 abort(); 9 } 10 } 11 } static analysis based model- checkers would consider both branches –both abort() statements are reachable –false alarm Symbolic execution gets stuck at line number 2 DART finds the only error

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y solve x*x*x+ 3*x*x+9 = y to take else branch Don’t know how to solve !! –Stuck ?

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y solve x*x*x+ 3*x*x+9 = y to take else branch Don’t know how to solve !! –Stuck ? –NO : DART handles this smartly

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 (randomly chose a value) and proceed

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 and proceed take then branch with constraint 9 != y

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 and proceed take then branch with constraint 9 != y solve 9 = y to take else branch execute next run with x = -3 and y= 9 –got error (reaches abort)

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 and proceed take then branch with constraint 9 != y solve 9 = y to take else branch execute next run with x = -3 and y= 9 –got error (reaches abort) Replace symbolic expression by concrete value when symbolic expression becomes unmanageable (i.e. non-linear)

Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } void again_test_me(int x,int y){ z = black_box_fun(x); if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); }

Discussion In comparison to existing testing tools, DART is –light-weight –dynamic analysis (compare with static analysis) ensures no false alarms –concrete execution and symbolic execution run simultaneously symbolic execution consults concrete execution whenever dynamic analysis becomes intractable –real tool that works on real C programs completely automatic Software model-checkers using abstraction (SLAM, BLAST) –starts with an abstraction with more behaviors – gradually refines –static analysis approach – false alarms –DART: executes program systematically to explore feasible paths

Another Name for this Approach: Concolic Execution Combine concrete and symbolic execution for unit testing –Concrete + Symbolic = Concolic Concolic Execution –Use concrete execution over a concrete input to guide symbolic execution –Concrete execution helps Symbolic execution to simplify complex and unmanageable symbolic expressions by replacing symbolic values by concrete values Achieves Scalability –Higher branch coverage than random testing –No false positives or scalability issue like in symbolic execution based testing

CUTE: A concolic execution tool CUTE: A Concolic Unit Testing Engine –For C and Java –Handle pointers Can test data-structures Can handle heap –Supports bounded depth search –Use static analysis to find branches that can lead to assertion violation use this info to prune search space

CUTE Case study: SGLIB SGLIB: popular library for C data-structures Used in Xrefactory a commercial tool for refactoring C/C++ programs Found two bugs in sglib 1.0.1 –reported them to authors –fixed in sglib 1.0.2 Bug 1: –doubly-linked list library segmentation fault occurs when a non-zero length list is concatenated with a zero-length list discovered in 140 iterations ( < 1second) Bug 2: –hash-table an infinite loop in hash table is-member function –193 iterations (1 second)

DART vs CUTE DART handles only arithmetic constraints CUTE –Supports C with pointers, data-structures –Highly efficient constraint solver 100 -1000 times faster –arithmetic, pointers –Provides Bounded Depth-First Search and Random Search strategies –Publicly available tool that works on ALL C programs

Example typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Random Test Driver: random memory graph reachable from p random value for x Probability of reaching abort( ) is extremely low

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0 p, x=236 NULL

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL !(p 0 !=NULL)

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL p 0 =NULL solve: x 0 >0 and p 0  NULL

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL p 0 =NULL solve: x 0 >0 and p 0  NULL x 0 =236, p 0 634 NULL

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0  NULL

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0  NULL 2x 0 +1  v 0

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0  NULL 2x 0 +1  v 0 solve: x 0 >0 and p 0  NULL and 2x 0 +1=v 0

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0  NULL 2x 0 +1  v 0 solve: x 0 >0 and p 0  NULL and 2x 0 +1=v 0 x 0 =1, p 0 3 NULL

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL 2x 0 +1=v 0

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL 2x 0 +1=v 0 n0p0n0p0

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints solve: x 0 >0 and p 0  NULL and 2x 0 +1=v 0 and n 0 =p 0. p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL 2x 0 +1=v 0 n0p0n0p0

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints solve: x 0 >0 and p 0  NULL and 2x 0 +1=v 0 and n 0 =p 0 x 0 =1, p 0 3 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL 2x 0 +1=v 0 n0p0n0p0

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p 0  NULL p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p 0  NULL 2x 0 +1=v 0 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3

CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p 0  NULL 2x 0 +1=v 0 n0=p0n0=p0 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3 Program Error

Pointer Inputs: Input Graph typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; }

Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1

CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path

CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically

CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically –Both cooperate with each other concrete execution guides the symbolic execution

CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically –Both cooperate with each other concrete execution guides the symbolic execution concrete execution enables symbolic execution to overcome incompleteness of theorem prover –replace symbolic expressions by concrete values if symbolic expressions become complex –resolve aliases for pointer using concrete values –handle arrays naturally

CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically –Both cooperate with each other concrete execution guides the symbolic execution concrete execution enables symbolic execution to overcome incompleteness of theorem prover –replace symbolic expressions by concrete values if symbolic expressions become complex –resolve aliases for pointer using concrete values –handle arrays naturally symbolic execution helps to generate concrete input for next execution –increases coverage

Testing Data-structures of CUTE itself Unit tested several non-standard data-structures implemented for the CUTE tool –cu_depend (used to determine dependency during constraint solving using graph algorithm) –cu_linear (linear symbolic expressions) –cu_pointer (pointer symbolic expressions) Discovered a few memory leaks and a couple of segmentation faults –these errors did not show up in other uses of CUTE –for memory leaks we used CUTE in conjunction with Valgrind

SGLIB: popular library for C data-structures Used in Xrefactory a commercial tool for refactoring C/C++ programs Found two bugs in sglib 1.0.1 –reported them to authors –fixed in sglib 1.0.2 Bug 1: –doubly-linked list library segmentation fault occurs when a non-zero length list is concatenated with a zero-length list discovered in 140 iterations ( < 1second) Bug 2: –hash-table an infinite loop in hash table is-member function –193 iterations (1 second)

Summary “DART: Directed Automated Random Testing” by Patrice Godefroid, Nils Klarlund, and Koushik Sen (PLDI’05) –handles only arithmetic constraints CUTE –Supports C with pointers, data-structures –Highly efficient constraint solver 100 -1000 times faster –arithmetic, pointers –Provides Bounded Depth-First Search and Random Search strategies –Publicly available tool that works on ALL C programs

Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen.

Similar presentations

Presentation on theme: "Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen.

Similar presentations

Presentation on theme: "Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen."— Presentation transcript:

Similar presentations

About project

Feedback