Presentation is loading. Please wait.

Presentation is loading. Please wait.

Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen.

Similar presentations


Presentation on theme: "Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen."— Presentation transcript:

1 Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen

2 Software Testing Testing accounts for 50% of software development cost Software failure costs USA $60 billion annually –Improvement in software testing infrastructure can save one-third of this cost “The economic impacts of inadequate infrastructure for software testing”, NIST, May, 2002 Currently, software testing is mostly done manually

3 Simple C code int double(int x) { return 2 * x; } void test_me(int x, int y){ int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); }

4 Automated testing What do we need to do if we want to automatically test a piece of code (i.e., automatic unit testing)? –Figure out the environment of the application What are the inputs? What are the interfaces for interacting with other components –Automatically generate an environment for the application Automatically generate the values that come from the environment

5 Automatic Extraction of Interface Automatically determine (code parsing) –inputs to the program arguments to the entry function –variables: whose value depends on environment external objects –function calls: return value depends on the environment external function calls For simple C code –want to unit test the function test_me –int x and int y : passed as an argument to test_me forms the external environment

6 Automated Random Testing Generate a test driver automatically to simulate random environment of the extracted interface –most general environment –C – code Compile the program along with the test driver to create a closed executable. Run the executable several times to see if assertion is violated

7 Random test-driver main(){ int tmp1 = randomInt(); int tmp2 = randomInt(); test_me(tmp1,tmp2); } int double(int x) { return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Random Test Driver

8 Random test-driver main(){ int tmp1 = randomInt(); int tmp2 = randomInt(); test_me(tmp1,tmp2); } int double(int x) { return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Random Test Driver Probability of reaching abort() is extremely low

9 Limitations Hard to hit the assertion violated with random values of x and y –there is an extremely low probability of hitting assertion violation Can we do better? –Directed Automated Random Testing White box testing

10 DART (Directed Automated Random Testing) Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); }

11 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=36t1=m concrete statesymbolic stateconstraints

12 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=36, t2=-7t1=m, t2=n concrete statesymbolic stateconstraints

13 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=36, t2=-7t1=m, t2=n concrete statesymbolic stateconstraints

14 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7x=m, y=n concrete statesymbolic stateconstraints

15 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m concrete statesymbolic stateconstraints

16 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m 2m != n concrete statesymbolic stateconstraints

17 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m 2m != n concrete statesymbolic stateconstraints

18 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=36, y=-7, z=72x=m, y=n, z=2m 2m != n solve: 2m = n m=1, n=2 concrete statesymbolic stateconstraints

19 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=1t1=m concrete statesymbolic stateconstraints

20 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=1, t2=2t1=m, t2=n concrete statesymbolic stateconstraints

21 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=1, t2=2t1=m, t2=n concrete statesymbolic stateconstraints

22 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2x=m, y=n concrete statesymbolic stateconstraints

23 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m concrete statesymbolic stateconstraints

24 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n concrete statesymbolic stateconstraints

25 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n m != n+10 concrete statesymbolic stateconstraints

26 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n m != n+10 concrete statesymbolic stateconstraints

27 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n m != n+10 concrete statesymbolic stateconstraints

28 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=1, y=2, z=2x=m, y=n, z=2m 2m = n m != n+10 solve: 2m = n and m=n+10 m= -10, n= -20 concrete statesymbolic stateconstraints

29 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=-10t1=m concrete statesymbolic stateconstraints

30 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=-10, t2=-20t1=m, t2=n concrete statesymbolic stateconstraints

31 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution t1=-10, t2=-20t1=m, t2=n concrete statesymbolic stateconstraints

32 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=-10, y=-20x=m, y=n concrete statesymbolic stateconstraints

33 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=-10, y=-20, z=- 20 x=m, y=n, z=2m concrete statesymbolic stateconstraints

34 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution x=-10, y=-20, z=- 20 x=m, y=n, z=2m 2m = n concrete statesymbolic stateconstraints

35 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution 2m = n m = n+10 x=-10, y=-20, z=- 20 x=m, y=n, z=2m concrete statesymbolic stateconstraints

36 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } Concrete Execution Symbolic Execution 2m = n m = n+10 x=-10, y=-20, z=- 20 x=m, y=n, z=2m Program Error concrete statesymbolic stateconstraints

37 DART Approach main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if(z==y){ if(x != y+10){ printf(“I am fine here”); } else { printf(“I should not reach here”); abort(); } z==y x!=y+10 NY NY Error

38 DART in a Nutshell Dynamically observe random execution and generate new test inputs to drive the next execution along an alternative path –do dynamic analysis on a random execution –collect symbolic constraints at branch points –negate one constraint at a branch point (say b) –call constraint solver to generate new test inputs –use the new test inputs for next execution to take alternative path at branch b –(Check that branch b is indeed taken next)

39 More details Instrument the C program to do both –Concrete Execution Actual Execution –Symbolic Execution and Lightweight theorem proving (path constraint solving) Dynamic symbolic analysis Interacts with concrete execution Instrumentation also checks whether the next execution matches the last prediction.

40 Experiments Tested a C implementation of a security protocol (Needham-Schroeder) with a known attack –406 lines of code –Took less than 26 minutes on a 2GHz machine to discover middle-man attack In contrast, a software model-checker (VeriSoft) and a hand-written nondeterministic model of the attacker took hours to discover the attack

41 Larger Experiment oSIP (open-source session initiation protocol) –http://www.gnu.org/software/osip/osip.html –30,000 lines of C code (version 2.0.9) –600 externally visible functions Results –crashed 65% of the externally visible functions within 1000 iterations –no nullity check for pointers Focused on oSIP parser –can externally crash oSIP server –osip_message_parse() : pass a buffer of size 2.5 MB with no 0 or “|” character –tries to copy the packet to stack using alloca(size) this fails: returns NULL pointer –this NULL pointer passed to another function does not check for nullity and crashes

42 Advantage of Dynamic Analysis over Static Analysis struct foo { int i; char c; } bar (struct foo *a) { if (a->c == 0) { *((char *)a + sizeof(int)) = 1; if (a->c != 0) { abort(); } Reasoning about dynamic data is easy Due to limitation of alias analysis “static analyzers” cannot determine that “a->c” has been rewritten –Software model checker BLAST would infer that the program is safe DART finds the error

43 Further advantages 1 foobar(int x, int y){ 2 if (x*x*x > 0){ 3 if (x>0 && y==10){ 4 abort(); 5 } 6 } else { 7 if (x>0 && y==20){ 8 abort(); 9 } 10 } 11 } static analysis based model- checkers would consider both branches –both abort() statements are reachable –false alarm Symbolic execution gets stuck at line number 2 DART finds the only error

44 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver

45 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y

46 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y solve x*x*x+ 3*x*x+9 = y to take else branch Don’t know how to solve !! –Stuck ?

47 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y solve x*x*x+ 3*x*x+9 = y to take else branch Don’t know how to solve !! –Stuck ? –NO : DART handles this smartly

48 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver

49 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z

50 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 (randomly chose a value) and proceed

51 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 and proceed take then branch with constraint 9 != y

52 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 and proceed take then branch with constraint 9 != y solve 9 = y to take else branch execute next run with x = -3 and y= 9 –got error (reaches abort)

53 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 –cannot handle symbolic value of z –make symbolic z = 9 and proceed take then branch with constraint 9 != y solve 9 = y to take else branch execute next run with x = -3 and y= 9 –got error (reaches abort) Replace symbolic expression by concrete value when symbolic expression becomes unmanageable (i.e. non-linear)

54 Simultaneous Symbolic & Concrete Execution void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } void again_test_me(int x,int y){ z = black_box_fun(x); if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); }

55 Discussion In comparison to existing testing tools, DART is –light-weight –dynamic analysis (compare with static analysis) ensures no false alarms –concrete execution and symbolic execution run simultaneously symbolic execution consults concrete execution whenever dynamic analysis becomes intractable –real tool that works on real C programs completely automatic Software model-checkers using abstraction (SLAM, BLAST) –starts with an abstraction with more behaviors – gradually refines –static analysis approach – false alarms –DART: executes program systematically to explore feasible paths

56 Another Name for this Approach: Concolic Execution Combine concrete and symbolic execution for unit testing –Concrete + Symbolic = Concolic Concolic Execution –Use concrete execution over a concrete input to guide symbolic execution –Concrete execution helps Symbolic execution to simplify complex and unmanageable symbolic expressions by replacing symbolic values by concrete values Achieves Scalability –Higher branch coverage than random testing –No false positives or scalability issue like in symbolic execution based testing

57 CUTE: A concolic execution tool CUTE: A Concolic Unit Testing Engine –For C and Java –Handle pointers Can test data-structures Can handle heap –Supports bounded depth search –Use static analysis to find branches that can lead to assertion violation use this info to prune search space

58 CUTE Case study: SGLIB SGLIB: popular library for C data-structures Used in Xrefactory a commercial tool for refactoring C/C++ programs Found two bugs in sglib 1.0.1 –reported them to authors –fixed in sglib 1.0.2 Bug 1: –doubly-linked list library segmentation fault occurs when a non-zero length list is concatenated with a zero-length list discovered in 140 iterations ( < 1second) Bug 2: –hash-table an infinite loop in hash table is-member function –193 iterations (1 second)

59 DART vs CUTE DART handles only arithmetic constraints CUTE –Supports C with pointers, data-structures –Highly efficient constraint solver 100 -1000 times faster –arithmetic, pointers –Provides Bounded Depth-First Search and Random Search strategies –Publicly available tool that works on ALL C programs

60 Example typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Random Test Driver: random memory graph reachable from p random value for x Probability of reaching abort( ) is extremely low

61 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0 p, x=236 NULL

62 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0 p, x=236 NULL

63 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL

64 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL !(p 0 !=NULL)

65 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL p 0 =NULL solve: x 0 >0 and p 0  NULL

66 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0 p, x=236 NULL p 0 =NULL solve: x 0 >0 and p 0  NULL x 0 =236, p 0 634 NULL

67 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634

68 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0

69 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0  NULL

70 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0  NULL 2x 0 +1  v 0

71 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0  NULL 2x 0 +1  v 0

72 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0  NULL 2x 0 +1  v 0 solve: x 0 >0 and p 0  NULL and 2x 0 +1=v 0

73 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=236 634 x 0 >0 p 0  NULL 2x 0 +1  v 0 solve: x 0 >0 and p 0  NULL and 2x 0 +1=v 0 x 0 =1, p 0 3 NULL

74 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3

75 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0

76 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL

77 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL 2x 0 +1=v 0

78 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL 2x 0 +1=v 0 n0p0n0p0

79 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL 2x 0 +1=v 0 n0p0n0p0

80 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints solve: x 0 >0 and p 0  NULL and 2x 0 +1=v 0 and n 0 =p 0. p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL 2x 0 +1=v 0 n0p0n0p0

81 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints solve: x 0 >0 and p 0  NULL and 2x 0 +1=v 0 and n 0 =p 0 x 0 =1, p 0 3 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p NULL, x=1 3 x 0 >0 p 0  NULL 2x 0 +1=v 0 n0p0n0p0

82 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3

83 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3

84 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p 0  NULL p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3

85 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p 0  NULL 2x 0 +1=v 0 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3

86 CUTE Approach typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Concrete Execution Symbolic Execution concrete statesymbolic stateconstraints x 0 >0 p 0  NULL 2x 0 +1=v 0 n0=p0n0=p0 p=p 0, x=x 0, p->v =v 0, p->next=n 0 p, x=1 3 Program Error

87 Pointer Inputs: Input Graph typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; }

88 Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1

89 Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1

90 Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1

91 Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1

92 Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1

93 Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1

94 Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1

95 Explicit Path (not State) Model Checking Traverse all execution paths one by one to detect errors –check for assertion violations –check for program crash –combine with valgrind to discover memory leaks –detect invariants 0 1 00 0 0 1 1 1 1 1 1

96 CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path

97 CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically

98 CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically –Both cooperate with each other concrete execution guides the symbolic execution

99 CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically –Both cooperate with each other concrete execution guides the symbolic execution concrete execution enables symbolic execution to overcome incompleteness of theorem prover –replace symbolic expressions by concrete values if symbolic expressions become complex –resolve aliases for pointer using concrete values –handle arrays naturally

100

101 CUTE in a Nutshell Generate concrete inputs one by one –each input leads program along a different path On each input execute program both concretely and symbolically –Both cooperate with each other concrete execution guides the symbolic execution concrete execution enables symbolic execution to overcome incompleteness of theorem prover –replace symbolic expressions by concrete values if symbolic expressions become complex –resolve aliases for pointer using concrete values –handle arrays naturally symbolic execution helps to generate concrete input for next execution –increases coverage

102 Testing Data-structures of CUTE itself Unit tested several non-standard data-structures implemented for the CUTE tool –cu_depend (used to determine dependency during constraint solving using graph algorithm) –cu_linear (linear symbolic expressions) –cu_pointer (pointer symbolic expressions) Discovered a few memory leaks and a couple of segmentation faults –these errors did not show up in other uses of CUTE –for memory leaks we used CUTE in conjunction with Valgrind

103 SGLIB: popular library for C data-structures Used in Xrefactory a commercial tool for refactoring C/C++ programs Found two bugs in sglib 1.0.1 –reported them to authors –fixed in sglib 1.0.2 Bug 1: –doubly-linked list library segmentation fault occurs when a non-zero length list is concatenated with a zero-length list discovered in 140 iterations ( < 1second) Bug 2: –hash-table an infinite loop in hash table is-member function –193 iterations (1 second)

104 Summary “DART: Directed Automated Random Testing” by Patrice Godefroid, Nils Klarlund, and Koushik Sen (PLDI’05) –handles only arithmetic constraints CUTE –Supports C with pointers, data-structures –Highly efficient constraint solver 100 -1000 times faster –arithmetic, pointers –Provides Bounded Depth-First Search and Random Search strategies –Publicly available tool that works on ALL C programs


Download ppt "Dynamic Symbolic Execution (aka, directed automated random testing, aka concolic execution) Slides by Koushik Sen."

Similar presentations


Ads by Google