Public Deployment of Cooperative Bug Isolation Ben Liblit, Mayur Naik, Alice Zheng, Alex Aiken, and Michael Jordan UC Berkeley, Stanford University and University of Wisconsin (pending) Motivate research Review instrumentation strategy Engineering issues exposed working toward public deployment Eye candy Screen shots Deployment status Sneak peak at analysis results

Our Goal: Measure Reality We measure bridges, airplanes, cars… Where is ight data recorder for software? Users are a vast, untapped resource 96,000 new Kazaa users during this workshop Users know what matters most Opportunity for reality-directed debugging Implicit bug triage for an imperfect world Users define what matters most. Office XP: 60 million licenses in first year; nearly 2 per second Kazaa: “over 1.9 million downloaded last week”; more than 3 per second

Bug Isolation Architecture Guesses Shipping Application Program Source Sampler  € ƒ ‚ Compiler Note: no programmer annotation required! Just pick which instrumentation schemes to activate. Review of instrumentation strategy Engineering challenges & solutions Deployment status Preliminary results Statistical Debugging Prole & J/L Top bugs with likely causes

Predicates on Program Behavior Guess what might be interesting Branches: Left? Right? Function returns: Negative? Zero? Positive? Pairs of variables: Less? Equal? Greater? Reference counts: Alive? Dead? Count how often guesses are true Feedback: vector of counts + outcome label These are just examples; we have several more schemes too.

Sampling the Bernoulli Way “Next sample” countdown Geometric distribution Split into acyclic regions Finite threshold weight Martin Hirzel & Trishul Chilimbi: optimal threshold check placement is NP-hard optimality = minimal set of check nodes such that every cycle contains at least one by reduction from Vertex-Cover

Sampling the Bernoulli Way “Next sample” countdown Geometric distribution Split into acyclic regions Finite threshold weight 4 3 1 2 1 2 1 Martin Hirzel & Trishul Chilimbi: optimal threshold check placement is NP-hard optimality = minimal set of check nodes such that every cycle contains at least one by reduction from Vertex-Cover 1

Sampling the Bernoulli Way “Next sample” countdown Geometric distribution Split into acyclic regions Finite threshold weight Clone acyclic regions “Fast” & “slow” variants Choose at run time Result: Subset of dynamic behavior Statistically fair sample >4? Related work: Arnold & Ryder [2001]

Multithreaded Programs Global next-sample countdown High contention, small footprint Want to use registers for performance Thread-local: one countdown per thread Global predicate counters Low contention, large footprint Optimistic atomic increment

Multi-Module Programs Forget about global static analysis Plug-ins, shared libraries Instrumented & uninstrumented code Self-management at compile time Locally derive identifying object signature Embed static site information within object le Self-management at run time Report feedback state on normal object unload Signal handlers walk global object registry

Native Compiler Integration Instrumentor must mimic native compiler You don’t have time to port & annotate by hand Our approach: source-to-source, then native Hooks for GCC: Stage wrapping via scripts Flag management via specles Compiler Sampler Guesses Shipping Application Program Source Compiler Tool Chain

Keeping the User In Control

Public Deployment, To Date

Public Deployment, To Date

Sneak Peak: Data Exploration Entire band: total number of runs in which the predicate is observed to be true Black segment: P(fail | line of code on which the predicate occurs is reached) White segment: P(succeed | predicate is observed to be true) Red + pink segments: increase in P(crash | predicate is observed to be true) beyond P(crash line of code on which the predicate occurs is reached) Red segment only: increase for which we are 95% confident

Conclusions Public deployment is challenging Real world code pushes tools to their limits Large user communities take time to build But the results are worth it: “Thanks to Ben Liblit and the Cooperative Bug Isolation Project, this version of Rhythmbox should be the most stable yet.”

The Cooperative Bug Isolation Project Join the Cause! The Cooperative Bug Isolation Project http://www.cs.berkeley.edu/~liblit/sampler/