RADAR: Dataflow Analysis for Concurrent Programs using Datarace Detection Ravi Chugh, Jan Voung, Ranjit Jhala, Sorin Lerner {rchugh, jvoung, jhala, cs.ucsd.edu UC San Diego

Studying Concurrency is Important

A “counting argument” > wget > grep –i “concurr” program_schedule.html | wc –l 6

All Jokes Aside… Architectural Trends Devilish Bugs Pressing Issue

Our Approach Leverage sequential work Sequential Dataflow Analysis Concurrent Dataflow Analysis

Sequential Reasoning is Incorrect if (p != null) { use(p); p = null; } p = new_or_die(); p != null x = compute(); p != null *p = x;

Sequential Optimization is Incorrect g = compute(); flag = 1; flag = 0; while (flag == 0) { /* spin */ } use(g); 0 Constant propagation would be incorrect Compilers forced to be conservative Optimization opportunities missed

Our Approach Sequential Dataflow Analysis Concurrent Dataflow Analysis RADAR

Concurrency Analysis Race Detector Our Approach Concurrent Dataflow Analysis Sequential Dataflow Analysis RADAR

Race Detector Modular Framework Sequential Non-Null Analysis Sequential Constant Analysis Sequential Dataflow Analysis Concurrent Dataflow Analysis RADAR Concurrent Non-Null Analysis Concurrent Constant Analysis

Modular Framework RADAR Race Detector Sequential Constant Analysis Concurrent Constant Analysis precisescalable

RADAR: Results Concurrent non-null analysis Scales to: –Apache (~130 KLOC) –OpenSSL (~210 KLOC) –subset of Linux kernel (~830 KLOC)

Example with Locks lock(l); if (p != null) { use(p); p = null; } unlock(l); lock(l); p = new_or_die(); x = compute(); *p = x; unlock(l);

lock(l); if (p != null) { use(p); p = null; } unlock(l); Can this be Optimized? lock(l); p = new_or_die(); unlock(l); x = compute(); lock(l); *p = x; unlock(l);

Optimized Version is Race-free lock(l); if (p != null) { use(p); p = null; } unlock(l); lock(l); p = new_or_die(); unlock(l); x = compute(); lock(l); *p = x; unlock(l);

lock(l); if (p != null) { use(p); p = null; } unlock(l); Oops… lock(l); p = new_or_die(); unlock(l); x = compute(); lock(l); *p = x; unlock(l);

lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; p != null unlock(l); Sequential Non-Null Analysis lock(l); if (p != null) { p != null use(p); p != null p = null; p != null } unlock(l);

RADAR on Example lock(l); if (p != null) { p != null use(p); p != null p = null; p != null } unlock(l); lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; p != null unlock(l);

lock(l); p = new_or_die(); p != null unlock(l); x = compute(); lock(l); *p = x; unlock(l); RADAR on Example Can fact be invalidated by concurrent thread? Can p be written by a concurrent thread?

lock(l); p = new_or_die(); p != null unlock(l); x = compute(); lock(l); *p = x; unlock(l); RADAR on Example pseudo-read(p) Race Detector Can p be written by a concurrent thread? Race No Race

lock(l); p = new_or_die(); p != null unlock(l); x = compute(); lock(l); *p = x; unlock(l); RADAR on Example pseudo-read(p) Can p be written by a concurrent thread? Race No Race Race Detector

lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); lock(l); *p = x; unlock(l); RADAR on Example pseudo-read(p) Race Detector Race No Race

lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; unlock(l); RADAR on Example UNSAFE

RADAR on Safe Example lock(l); p = new_or_die(); p != null x = compute(); *p = x; unlock(l); Race Detector Race No Race pseudo-read(p)

RADAR on Safe Example lock(l); p = new_or_die(); p != null x = compute(); p != null *p = x; unlock(l); Race Detector Race No Race pseudo-read(p)

RADAR on Safe Example lock(l); p = new_or_die(); p != null x = compute(); p != null *p = x; unlock(l); SAFE

More on RADAR 1.Round trip queries to race detector 2.Inter-procedural analysis

(1) Round-trip Queries lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; p != null unlock(l); Possible Race Allow sequential analysis to run pseudo-read(p) Race Detector Race No Race Get superset of (concurrent) facts pseudo-read(p)

lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; p != null unlock(l); (1) Round-trip Queries Insert all pseudo-reads at once pseudo-read(p) Race Detector Send whole program to race detector

(1) Round-trip Queries lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; p != null unlock(l); pseudo-read(p) Get results back from Race Detector

(1) Round-trip Queries lock(l); p = new_or_die(); unlock(l); x = compute(); lock(l); *p = x; unlock(l); pseudo-read(p) Get results back from Race Detector Rerun analysis using race results

(1) Round-trip Queries lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); lock(l); *p = x; unlock(l); Rerun analysis using race results

(1) Round-trip Queries lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; unlock(l); UNSAFE

(2) Handling Procedures Unlock in foo allows interference lock(l); if (p != null) { foo(); *p = 10; } unlock(l); void foo () { if (*) { unlock(l); compute(); lock(l); } lock(l); p = null; unlock(l); Want to summarize effect of calling foo

(2) Handling Procedures lock(l); if (p != null) { foo(); *p = 10; } unlock(l); void foo () { if (*) { unlock(l); compute(); lock(l); } lock(l); p = null; unlock(l);

lock(l); if (p != null) { foo(); *p = 10; } unlock(l); (2) Handling Procedures pseudo-unlock(l); pseudo-read(p); pseudo-lock(l); In the caller, RADAR inserts: pseudo-unlock for every unlock in foo pseudo-reads

Evaluation Is RADAR scalable? Is RADAR precise? Where can we do better?

Experiments: Benchmarks Apache (130 KLOC) –worker threads + modules (e.g., caches) OpenSSL 0.9.8g (210 KLOC) –model a multi-threaded client Linux (830 KLOC) –subset from RELAY experiments

SequentialRace Detector += Non-Null Never race Optimistic (sequential) Non-Null Escapes ⇒ race Conservative

GAP SequentialRace Detector += Non-Null Never race Optimistic (sequential) Escapes ⇒ race Conservative Shared ⇒ race No locks ⇒ race

SequentialRace Detector += Non-Null Optimistic (sequential) Conservative Never race Escapes ⇒ race Shared ⇒ race No locks ⇒ race

Sources of Imprecision Alias analysis –affects sequential dataflow and race detection Lockset-based race analysis –ignores fork, join, condition variables RADAR framework –pseudo-read (for non-null fact) races with –“ x = NonNullAddress; ”

Related Work Programming-model-based approaches –[Knoop et al 96], [Grunwald et al 93], … –par-begin / par-end –handles introduction of facts between threads Thread-modular –[Owicki et al 76], [Jones 83], [Flanagan et al 03], … –more precise (use environment assumption) –inference not as scalable

Conclusion RADAR Race Detector Sequential Dataflow Analysis Concurrent Dataflow Analysis precisescalable

THANKS!

What is filtered for non-null? if (a->f != null) { b->f = null; deref(a->f); //filter if warned deref(b->f); //don’t filter }