KISS: K EEP I T S IMPLE AND S EQUENTIAL Guy Martin, OSLab, GNU09 Shaz Qadeer Microsoft Research One Microsoft Way Redmond, WA Dinghao Wu Department of Computer Science Princeton University Princeton, NJ [QaWu04]

A GENDA Background KISS, Keep It Simple and Sequential An Overview Instrumentation Procedure The Architecture Abstract SLAM, an overview A Parallel Language for KISS KISS, Program Transformation Race Detection KISS, Evaluation Related Work Conclusion 2 [QaWu04]

B ACKGROUND Thesis: Existing Model Checking methods for bugs detection in concurrent programs have high computational complexity. Why? Because they are thread interleaving-based. Exploration of all possible thread interleaving which increases their complexity since the number of thread interleaving also increases exponentially with the number of threads. Solution: KISS, translates a concurrent program P into a sequential program P’ that simulates the execution of a large subset of the behaviors of P. 3 [QaWu04]

KISS, AN OVERVIEW T1T1 T2T2 TnTn 4 [QaWu04] T1T1 T2T2 TnTn Multithreaded Program Each Thread has its own Execution Stack Sequential Program: Single Stack The frame of the stack is partitioned into contiguous blocks: one block per thread. The generated sequential program simulates the original concurrent program and runs under the control of a nondeterministic scheduler.

KISS, AN OVERVIEW 2-functions based nondeterministic scheduler Terminate or resume a thread execution Call another thread by its starting function Stack-based algorithm with 2 main ideas Nondeterministic termination of a thread at any time during its execution  usage of a Boolean raise 1 Schedule a nondeterministic chosen number of existing threads at any time during the executing time  usage of variable set ts 2 The number of simulated behaviors of P depends deeply of the size of the global state space of P’ 5 [QaWu04] 1 Encodes the effect of raising an exception, initialized to false and precedes every statement in a function. 2 Is a bounded-size multiset of the starting functions of threads that have been forked but not scheduled.

KISS, I NSTRUMENTATION PROCEDURE Whenever the concurrent program invokes the function f asynchronously, f is added to ts if ts is not full. The execution of f happens later If the set ts is full, then the asynchronous call to f is replaced with a synchronous call to f, thus executing it immediately.  The instrumentation code nondeterministically chooses a function from the set ts, invokes it, and sets raise to false when it returns. 6 [QaWu04]

KISS, T HE A RCHITECTURE Instrumentation SLAM Concurrent C Program Sequential C Program Error TraceNo Bug Found 7 [QaWu04]

KISS, A BSTRACT KISS reduces parallel analysis into sequential analysis By translating a multithreaded program P into a sequential program P’ P’ simulates a subset of parallelism in P Simulation by P’ uses its own stack-based nondeterministic scheduler The generated sequential P’ program is analyzed in the SLAM model checker. 8 [QaWu04]

SLAM, AN OVERVIEW The goal of the SLAM 1 project is to check whether or not a program obeys "API usage rules" that specify what it means to be a good client of an API. The SLAM toolkit statically analyzes a C program to determine whether or not it violates given usage rules. The toolkit has two unique aspects: it does not require the programmer to annotate the source program (invariants are inferred); it minimizes noise (false error messages) through a process known as “counterexample-driven refinement“ SLAM exploits and extends results from program analysis, model checking and automated deduction. 1 Cf. [BaRa02] 9 [QaWu04]

A PARALLEL LANGUAGE FOR KISS async vo() creates a new thread whose starting function is the value of vo asume(vo) blocks until variable vo becomes true atomic{s} executes just like s except that the execution may not be interrupted in the middle by other thread Lock procedures lock_acquire(l) = atomic{assume(*l==0);*l=1;} lock_release(l) = atomic{*l=0;} Loop statements if (v) s1 else s2 = choice{assume(v); s1 [] assume( ￢ v); s2} while (v) s def = iter{assume(v); s}; assume( ￢ v) 10 function names f ::= f0 | f1 |... integers i ::=... | −1 | 0 | 1 |... boolean constants b ::= true | false constants c ::= i | b | f primitives op ::= + | − | × | == variables v ::= v0 | v1 |... values u ::= v | c statements s ::= v0 = c | v0 = &v1 | v0 = ∗ v1 | ∗ v0 = v1 | v0 = v1 op v2 | assert (v0) | assume(v0) | atomic{s} | v = v0() | async v0() | return | s1; s2 | choice{s1 []... [] sn} | iter{s} def [QaWu04]

KISS, P ROGRAM T RANSFORMATION Definitions A concurrent program is one expressible in the previous parallel language A sequential program is one expressible in the previous parallel language without using asynchronous function calls and atomic statements. [[.]] is used to translate a concurrent program into a sequential program For any statement s, [[ s ]] is a statement without any asynchronous function calls and synchronization statements. For any function f with body s, a new function [[ f ]] is introduced with body [[ s ]] 11 [QaWu04]

KISS, P ROGRAM TRANSFORMATION 12 [[ v 0 = c ]] = schedule(); choice{skip [] RAISE}; v 0 = c [[ v 0 = &v 1 ]] = schedule(); choice{skip [] RAISE}; v 0 = &v 1 [[ v 0 = ∗ v 1 ]] = schedule(); choice{skip [] RAISE}; v 0 = ∗ v 1 [[ ∗ v 0 = v 1 ]] = schedule(); choice{skip [] RAISE}; ∗ v 0 = v 1 [[ v 0 = v 1 op v 2 ]] = schedule(); choice{skip [] RAISE}; v 0 = v 1 op v 2 [[ assert (v 0 ) ]] = schedule(); choice{skip [] RAISE}; assert (v 0 ) [[ assume(v 0 ) ]] = schedule(); choice{skip [] RAISE}; assume(v 0 ) [[ atomic{s} ]] = schedule(); choice{skip [] RAISE}; s [[ v = v 0 () ]] = schedule(); choice{skip [] RAISE}; v = [[ v 0 ]]() ; if (raise) return [[ async v 0 () ]] = schedule(); choice{skip [] RAISE}; if (size() < MAX) put(v 0 ) else { [[ v 0 ]]() ; raise = false} [[ return ]] = schedule(); return [[ s1; s2 ]] = [[ s1 ]]; [[ s2 ]] [[ choice{s1 []...[] sn} ]] = choice{ [[ s1 ]] []... [] [[ sn ]] } [[ iter{s} ]] = iter{ [[ s ]] } Instrumentation for assertion checking [QaWu04]

KISS, P ROGRAM TRANSFORMATION The maximum size of ts determines the number of possible executions of the original program that can be simulated by the translated program. Check(s) = raise = false; ts = ∅ ; [[ s ]] ; schedule(); Complexity 1 for Model Checking O(|C|.2 g+l ) Theorem 1 (Completeness). Suppose the multiset ts is unbounded. If a balanced execution of a concurrent program s goes wrong by failing an assertion, then the sequential program Check(s) also goes wrong, and vice versa. 13 [QaWu04] def 1 |C|size of the control-flow graph, g is the number of global variables and l max number of local variables.

R ACE D ETECTION News variable & functions are added Access equals 0: no access, 1: read access, 2: write access check r (x) checks that there is no race on r due to a read of the address contained in x check w (x) checks that there is no race on r due to a write to the address contained in x. An assertion in one of these calls is violated only if there are conflicting (read/write or write/write) accesses by two different threads. Check(s) = raise = false; ts = ∅ ; access = 0; [[ s ]] ; schedule(); If an assertion is violated in Check(s), there is an execution of s in which either an assertion is violated or there is a race condition on r. 14 [QaWu04] def

KISS, E VALUATION Experiments are performed on a 2.2GHz PC running Windows XP Many spurious alarms and benign data races reported Flexible Solution Implementation Can support many synchronization mechanisms Environment Modeling Flexible mechanisms for writing experiments Specification General assertion checker for concurrent program which can check other properties than races freedom 15 [QaWu04] DriverKLOCFieldsRaces# Races tracedrv Moufiltr Kbfiltr Imca Startio Toaster/toastmon Diskperf diag vedv Fakemodem Gameenum Toaster/bus Serenum Toaster/func Mouclass Kbdclass Mouser Fdc Total DriverRaces Moufiltr0 Kbfiltr0 Imca1 Toaster/toastmon1 Diskperf0 1394diag1 1394vedv1 Fakemodem6 Gameenum1 Serenum2 Toaster/func5 Mouclass1 Kbdclass1 Mouser1 Fdc9 Total 30 First evaluation: Many false alarms and benign Data races reported. Second evaluation: The number of reported data races went down. Some useful information from developers have been used.

R ELATED W ORK Model Checking: Systematic exploration of the state space of a model of the concurrent program(SPIN, JPF, Bandera, Bogor, etc.)  Exponential number of state to analyze Static Analysis: tools typically based on type systems and dataflow analysis (Warlock, RccJava, ESC/Java, etc.)  Less precise but more scalable than model checkers Dynamic Analysis: Instrumentation & execution of the program(Eraser, happens-before tools, etc.)  Small coverage of the program and high overhead Others: static analysis focusing on the synchronous message-passing mechanism( Bouajjani et al )  Not automated verification method 16 [QaWu04]

C ONCLUSION 17 [QaWu04] New technique which transform a concurrent program into a sequential program which simulates a large subset of the behaviors of the concurrent program Implementation in KISS, an automated checker for multithreaded C program built on top of SLAM Further work Adaptation on other tools like PREfix, MC, ESP and BLAST Solve the problem of benign races by allowing programmers to annotate an access like benign or by using the ideas behind the type system for atomicity for automatic pruning of such benign race conditions.

KISS, L IMITATIONS Analysis of a subset of the concurrent program behavior The number of simulated behaviors of P depends deeply of the size of the global state space of P’. The stack size of the generated sequential program increase with the number of concurrent threads. Hard limit on number of explored contexts e.g., two context switches for 2-threaded concurrent program Many spurious alarms and benign data races reported The model checking’s complexity which can be very high O(|C|.2 g+l ) 18 [QaWu04]

THANK YOU FOR YOUR ATTENTION 19 [QaWu04]