Generating Tests from Counterexamples Jinseong Jeon ARCS, KAIST

CS750b, KAIST 2/16 Actual Anxiety Counterexample! What makes him panic? What makes C.E. happen makes him panic!

CS750b, KAIST 3/16 Greater Goals Counterexample! How can it happen?  a test vector Any other cases?  a test suite

CS750b, KAIST 4/16 The Greatest Goal Automated Debugger! Program Target Pred. Test Driver Generator Test Suite Generator Test Driver Test Suite Testing

CS750b, KAIST 5/16 Contents Program Target Pred. Test Driver Generator Test Suite Generator Test Driver Test Suite Testing How to generate a test vector? How to generate a test suite? How to generate a test driver?

CS750b, KAIST 6/16 from Trace to Test (1/2) Example() { if (y == x) y++; if (z <= x) y++; a = y – z; if (a < x) LOC: } [ Program ] assume (y = x) y = y + 1 assume !(z <= x) a = y – z assume (a < x) [ Trace ] = = + 1 :  z,0> · = - [ Trace formula ] p, ( pc  LOC )

CS750b, KAIST 7/16 from Trace to Test (2/2) = = + 1 :  z,0> · = - [ Trace formula ]  0  y,1>  1  2  -1 [ Assignment ]  0  2 [ Test vector ] integer linear programming (ILP) solver

CS750b, KAIST 8/16 Linear Programming object function –maximize c 1 x 1 + c 2 x 2 problem constraints –a 11 x 1 + a 12 x 2 · b 1 –a 12 x 1 + a 22 x 2 · b 2 Algorithms –Simplex, Branch and Bound, etc. feasible region

CS750b, KAIST 9/16 An ILP Application Buffer Overrun Detection using Liner Programming and Static Analysis int main() { char header[2048], buf[1024], *cc1, *cc2, *ptr; int i; FILE *fp;... ptr = fgets(header, 2048, fp); cc1 = copy_buffer(header); for (i = 0; i < 10; i++) { ptr = fgets(buf, 1024, fp); cc2 = copy_buffer(buf); } header!alloc!max · 2048 header!alloc!min ¸ 0... header!used!max · 2048 header!used!min ¸ 1 cc1!used!max ¸ header!used!max cc1!used!min · header!used!min... i’!max ¸ i!max + 1 I’!min · i!min

CS750b, KAIST 10/16 Test Suite Gen. worklist à all locations in decreasing order of d.f.numbering Reach. Tree à a single node, the root test suite à ; while (worklist  ; ) { q à pop(worklist) p-trace à MC(Reach. Tree, p, q) if Reach. Tree is complete then worklist à ; else test suite à test suite [ { test_vector(p-trace) } } return test suite

CS750b, KAIST 11/16 Heuristics A test vector can cover several locations.  we can remove those locations from the worklist. MC’s unfolding (visiting) order  uncovered first, covered last Time-out option

CS750b, KAIST 12/16 Test Driver Gen. Original code Lib. call User input Test suite Test-feeding func. Test Driver

CS750b, KAIST 13/16 A Security Example (1/2) int saved_uid, saved_euid; work_and_drop_priv() { L5: FILE *fp = fopen(FILENAME,”w”); L6: if (!fp) { L7: return; } L8: // work L9: seteuid(saved_uid); } int get_root_privileges() { L1: if (saved_euid == 0) { L2: return -1; } L3: seteuid(0); L4: return 0; } int main(int argc, char *argv[]) { L10: saved_uid = getuid(); L11: saved_euid = geteuid(); L12: seteuid(saved_uid); L13: // work under normal mode L14: if (get_root_privileges() == 0 ) { L15: work_and_drop_priv(); } L16: execv(argv[1], argv+1); }

CS750b, KAIST 14/16 A Security Example (2/2) L10: saved_uid = getuid(); L11: saved_euid = geteuid(); L12: seteuid(saved_uid); L14: tmp = get_root_privileges(); L1: if (saved_euid != 0) /* fails */ L3: seteuid(saved_euid); L4: return 0; L14: if (tmp == 0) /* succeeds */ L15: work_and_drop_priv(); L5: fp = fopen(FILENAME, “w”); L6: if (!fp) /* succeeds */ L7: return; L16: /* uid = 0 */ [ A trace generated by BLAST ]

CS750b, KAIST 15/16 Experiments kbfiltr, floppy, cdaudio, parport, parclass  Microsoft Windows device drivers ping  an implementation of the ping utility ftpd  a Linux port of the ftp daemon

CS750b, KAIST 16/16 Conclusions pros –generate a test suite using model-checker –generate an automated debugger dead code detection, safety verification cons –only integer variables –what is really affected by BLAST?