Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010

2 Computer Annoyances Program bugsSlow computers

3 Unit Testing Test program parts, not the whole program –Test parts individually, then together –Smaller parts  larger parts Unit tests are programs –Compare expected result to actual result: assertEquals(12, multiply(3,4));

4 Moore’s Law Adopted from Sutter 2009

5 Existing Approaches Fail Simple Unit Testing Frameworks –Example: JUnit –Assume program is single-threaded –Many errors go undetected detected not detected Main thread: Other thread:

6 Problem: Non-Determinism Synchronization points –Information exchanged between threads –Order of threads reaching sync. points is non-deterministic

7 Problem: Non-Determinism Synchronization points –Information exchanged between threads –Order of threads reaching sync. points is non-deterministic –Program could succeed in one order but fail in another

8 Comprehensive Tools: Costly Model checking or schedule-based execution –Examples: JPF, ExitBlock –Test all arrangements –Too costly t= # of threads s= # of sync. points N= # of arrangements Longer than the universe exists! (if N=1  1 minute)

9 Insert random delays before synchronization points to modify the order Run tests many times with different delays Probabilistic Approach

10 Probabilistic Approach Insert random delays before synchronization points to modify the order Run tests many times with different delays delay

11 (Proposed) Contributions Probabilistic Approach –Random delays at synchronization points Call Graph Analysis –Determine which parts of the program may interact –Only insert delays where necessary Improved JUnit Framework –Exception handler for all threads –Ensures that all errors in all threads are detected

12 Evaluation Tested DrJava with Improved JUnit –Number of unit tests:900 –Previously unknown problems:20 –Slowdown: ~1 percent Other parts still under development

13 Conclusion Unit testing of concurrent programs will become more important Concutest helps… –by improving JUnit’s abilities –by using probabilistic techniques –by using call graph analysis to reduce the number of synchronization points

14 Image Attribution 1.Left image on Computer Annoyances: Adapted from MicrosoftComputer AnnoyancesMicrosoft 2.Graph on Moore’s Law: Adapted from Herb Sutter 2009Moore’s LawHerb Sutter Top image on Multi-Cores and Concurrency: Adapted from Brian Goetz et al. 2006, Addison WesleyMulti-Cores and ConcurrencyBrian Goetz et al. 2006, Addison Wesley 4.Bottom image on Multi-Cores and Concurrency: Caption FridaysMulti-Cores and Concurrency Caption Fridays

15 Extra Slides

16 Multi-Cores and Concurrency Recent trends –Clock speed stagnated –Transistors still increasing –Reason: multi-core CPUs Implication –Must use concurrency to benefit from modern CPUs –Concurrent programming is difficult

17 Number of schedules (N) –t: # of threads, s: # of slices per thread –Note: s is # of slices (between synchronization points) Number of Arrangements

18 Product of s-combinations For thread 1: choose s out of ts time slices For thread 2: choose s out of ts-s time slices … For thread t-1: choose s out of 2s time slices For thread t-1: choose s out of s time slices Writing s-combinations using factorial Cancel out terms in denominator and next numerator Left with (ts)! in numerator and t numerators with s! Number of Arrangements