1. Learning from Mistakes: A Comprehensive Study on Real-World Concurrency Bug Characteristics
Ben Shelton

2. Motivation Concurrent programming is necessary
Multicore revolution
Concurrent programming is difficult, and programmers need tools to help them write more reliable concurrent programs
Concurrency testing / static analysis tools
Narrowing the search space
Languages and features that make it easier for programmers to express concurrency
Transactional Memory?

3. Methodology Four open-source applications:
MySQL
Apache
Mozilla
OpenOffice
Search bug databases for keywords involving concurrency and randomly choose 500 bug reports
Check that each bug is caused by concurrency error
105 concurrency bugs in final sample set

4. Main Findings
97% of non-deadlock bugs are atomicity-violation or order-violation.
66% of non-deadlock bugs involve concurrent accesses to only one variable.
97% of deadlock bugs involve two threads in contention for two or fewer resources.
73% of non-deadlock bugs were fixed by techniques other than adding/changing locks.
Transactional memory could help in about one third of the bugs examined, but additional language features for concurrency are recommended.

5. Atomicity Violations

6. Order Violations

7. Order Violations, continued…

8. Low-Hanging Fruit
22% of deadlock bugs are caused by a thread trying to acquire a resource it already has
Almost trivial to check for
66% of non-deadlock bugs involve concurrent accesses to only one variable
This is the assumption that current concurrency testing software makes – turns out to be valid

9. Slightly-Higher-Hanging Fruit
96% of the bugs can be reproduced every time with a certain partial order between only two threads
Pairwise testing (computationally tractable) will catch most of these bugs!
92% of the bugs can be reproduced every time with a certain partial order between no more than four memory accesses
Cuts down search space from exponential to polynomial

10. Locks Are (Often) Not The Answer
Adding or changing locks was only used to fix 20 out of 74 non-deadlock bugs.
Instead:
Condition check
Change ordering of program statements
Algorithm redesign
Authors state: “Just telling programmers that certain conflicting accesses are not protected by the same lock is not enough to fix concurrency bugs.”
What about Eraser and tools like it?

11. The Fix Isn’t Always Right The First Time
~30% of the Mozilla bugs had at least one incorrect patch posted before the final correct fix.
When fixing deadlock bugs by allowing one thread to give up on acquiring a resource, programmers sometimes introduced non-deadlock bugs.
Takeaways:
This stuff is hard!
Programmers need help
Language support
Better debugging and analysis tools

12. Transactional Memory This bit of code looks like a transaction!
Programmer can rely on TM library to provide desired functionality rather than trying to reimplement it herself

13. Transactional Memory, continued…
Not suitable for I/O operations
HW-based TM may not work for large critical sections
Still isn’t able to express order in many cases
Even with extensions to TM, wouldn’t work for 20 out of 105 bugs in this study
But 85 out of 105 is still pretty good!

14. Paper Conclusions
By making simplifying assumptions, we can build practical concurrency testing tools that will catch the majority of expected concurrency bugs.
Transactional memory is a useful tool, and would be even more useful with expansions/modifications.
Programming languages need a better way to express order relationships between thread actions, and concurrency testing tools need to do a better job of illustrating bugs involving these relationships.

15. Is The Data Set Representative?
Authors claim that the data set is representative of two main classes of concurrent programs:
Server-based
Client-based
Data set isn’t evenly distributed between applications
57 bugs in Mozilla; only 8 bugs in OpenOffice

16. TM Enhancements
(From Microsoft Research [5])
Authors support TM enhancements like watch/retry, retry/orElse

17. TM Enhancements, continued…
Neither GCC’s new support for STM, nor Intel’s STM compiler for C++, nor the Deuce runtime for Java support these primitives
STM is seeing wider adoption – reasonable to think they’ll be available soon

18. Improved Programming Tools
Grace: Safe Multithreading – a potential solution for fork/join parallelism [4]
Run each thread as a process
Each thread executes optimistically but locally
Check for collision at commit
Threads run concurrently but commit in program order
Programmer can reason as if program were serial
Reduces the likelihood of order violations

19. Improved Debugging Tools
CTrigger – follow-up work from the authors [2]
Test improbable thread interleavings to do a faster job of detecting/characterizing atomicity violations
Execution Synthesis [3]
Tool gives the programmer a set of initial inputs and an execution order that is guaranteed to reproduce the bug
Better than “watch this variable”

20. References
[1] Shan Lu, Soyeon Park, Eunsoo Seo, and Yuanyuan Zhou Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. SIGARCH Comput. Archit. News 36, 1 (March 2008),
[2] Soyeon Park, Shan Lu, and Yuanyuan Zhou CTrigger: exposing atomicity violation bugs from their hiding places. In Proceedings of the 14th international conference on Architectural support for programming languages and operating systems (ASPLOS '09). ACM, New York, NY, USA,
[3] Cristian Zamfir and George Candea Execution synthesis: a technique for automated software debugging. In Proceedings of the 5th European conference on Computer systems(EuroSys '10). ACM, New York, NY, USA,
[4] Emery D. Berger, Ting Yang, Tongping Liu, and Gene Novark Grace: safe multithreaded programming for C/C++. SIGPLAN Not. 44, 10 (October 2009),
[5] Simon Peyton Jones (Microsoft Research Cambridge). Transactional Memory and Concurrency. OSCON ‘07. us/um/people/simonpj/papers/stm/STM-OSCON.pdf