2. Motivation  Parallel programming is difficult  Culprit: Non-determinism Interleaving of parallel threads But required to harness parallelism  Sequential programs produce deterministic results  To make parallel programming easy We want determinism Same input => semantically same output.

3. Determinism Efforts  Language design [DPJ, X10,Yada] Deterministic by design: Types and annotations Constrains programming May need non-determinism for better performance  Deterministic runtime [DMP, Kendo]  Race detection Absence of data races does not imply determinism Data races could be benign and could help in improving performance  Determinism Checker [SingleTrack, Torrellas et al.] Dynamic determinism checker.  Determinism with respect to abstraction [Rinard et al.]

4. Our Goal  Provide a framework that can express relevant determinism directly and easily Separate from functional correctness specification Allow data races and healthy non- determinism Language independent Useful for QA

5. Determinism specification: A sweet spot? Lightweight, but precise. Determinism specification: A sweet spot? Lightweight, but precise. Our Goal  How to specify correctness of parallelism? Implicit: No sources of non-determinism (no data races) Implicit: No sources of non-determinism (no data races) Explicit: Full functional correctness. Explicit: Full functional correctness.

6. Outline  Motivation  Deterministic Specification [FSE’09]  Checking: Active Testing  Experimental Evaluation  Future Work + Conclusions

7. Deterministic Specification  Goal: Specify deterministic behavior. Same initial parameters => same image. Non-determinism is internal. // Parallel fractal render mandelbrot(params, img); // Parallel fractal render mandelbrot(params, img);

8. Deterministic Specification  Program:, Initial State:, Schedules:  Specifies: Two runs from same initial program state have same result state for any pair of schedules deterministic { // Parallel fractal render mandelbrot(params, img); } deterministic { // Parallel fractal render mandelbrot(params, img); }

9. double A[][], b[], x[];... deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x); } double A[][], b[], x[];... deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x); } Deterministic Specification  Too restrictive – different schedules may give slightly different floating-point results.

10. set t = new RedBlackTreeSet(); deterministic { t.add(3) || t.add(5); } set t = new RedBlackTreeSet(); deterministic { t.add(3) || t.add(5); } Deterministic Specification  Too restrictive – internal structure of set may differ depending on order of adds.

11. deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data); } deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data); } Deterministic Specification  Too restrictive – search can correctly return any tree with optimal cost.

12. Semantic Determinism  Too strict to require every interleaving to give exact same program state: deterministic { P } deterministic { P }

13. Semantic Determinism  Too strict to require every interleaving to give exact same program state: deterministic { P } deterministic { P } Predicate! Should be user-defined. Predicate! Should be user-defined.

14. Semantic Determinism  Too strict to require every interleaving to give exact same program state:  Specifies: Final states are equivalent. deterministic { P } assert Post(s 1,s 1 ’) deterministic { P } assert Post(s 1,s 1 ’)

15. double A[][], b[], x[];... deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x); } assert (|x – x’| < ε) double A[][], b[], x[];... deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x); } assert (|x – x’| < ε) Semantic Determinism “Bridge” predicate

16.  Resulting sets are semantically equal. set t = new RedBlackTreeSet(); deterministic { t.add(3) || t.add(5); } assert (t.equals(t’)) set t = new RedBlackTreeSet(); deterministic { t.add(3) || t.add(5); } assert (t.equals(t’)) Semantic Determinism

17. deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data); } assert (t.cost == t’.cost()) deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data); } assert (t.cost == t’.cost()) Semantic Determinism

18.  Too strict – initial states must be identical Not compositional. Preconditions for Determinism set t = … deterministic { t.add(3) || t.add(5); } assert (t.equals(t’)) … deterministic { t.add(4) || t.add(6); } assert (t.equals(t’)) set t = … deterministic { t.add(3) || t.add(5); } assert (t.equals(t’)) … deterministic { t.add(4) || t.add(6); } assert (t.equals(t’))

19. Preconditions for Determinism  Too strict to require identical initial states: deterministic { P } assert Post(s 1,s 1 ’) deterministic { P } assert Post(s 1,s 1 ’)

20. Preconditions for Determinism  Too strict to require identical initial states: deterministic assume (s 0 = s 0 ’) { P } assert Post(s 1,s 1 ’) deterministic assume (s 0 = s 0 ’) { P } assert Post(s 1,s 1 ’)

21. Preconditions for Determinism  Too strict to require identical initial states: deterministic assume (s 0 = s 0 ’) { P } assert Post(s 1,s 1 ’) deterministic assume (s 0 = s 0 ’) { P } assert Post(s 1,s 1 ’) Predicate! Should be user-defined. Predicate! Should be user-defined.

22. Preconditions for Determinism  Too strict to require identical initial states:  Specifies: deterministic assume Pre(s 0,s 0 ’) { P } assert Post(s 1,s 1 ’) deterministic assume Pre(s 0,s 0 ’) { P } assert Post(s 1,s 1 ’)

23. deterministic assume Pre(s 0,s 0 ’) { P } assert Post(s 1,s 1 ’) deterministic assume Pre(s 0,s 0 ’) { P } assert Post(s 1,s 1 ’) Bridge predicates/assertions “Bridge” predicate “Bridge” assertion

24. set t =... deterministic assume (t.equals(t’) { t.add(4) || t.add(6); } assert (t.equals(t’)) set t =... deterministic assume (t.equals(t’) { t.add(4) || t.add(6); } assert (t.equals(t’))  Specifies: Semantically equal sets yield semantically equal sets. Preconditions for Determinism

25. Advantage  Separately check parallelism and functional correctness. Show parallelism is outwardly deterministic. Reason about correctness sequentially. Decompose correctness proof!  Example: Write Cilk program and prove (or test) sequential correctness. Add parallelism, answers should not change

26. Determinism vs. Atomicity  Internal vs. external parallelism/non-determinism Complementary notions Atomic Deterministic “Closed program” “Open program”

27. Other Testing Use sequential program as spec deterministic Pre(s0,s0’) { if (*) { SequentialPgm; } else { ParallelPgm; } } assert Post(s,s’); Regression testing deterministic Pre(s0,s0’) { if (*) { PgmVersion1; } else { PgmVersion2; } } assert Post(s,s’);

28. Outline  Motivation  Deterministic Specification [FSE’09]  Checking: Active Testing [PLDI 08,09, FSE 08, CAV 09]  Experimental Evaluation  Future Work + Conclusions

29. Checking: Active Testing Predicting and Exploring “interesting” Schedules

30. Active Testing: Predict and Test Potential Bugs  Predict potential bugs: Data races: Eraser or lockset based Atomicity violations: cycle in transactions and happens-before relation Deadlocks: cycle in resource acquisition graph Memory model bugs: cycle in happens- before relation  Test schedules to create those bugs [ASE 07, PLDI 08, FSE 08, PLDI 09, FSE 09, CAV 09]

31. Active Testing Cartoon: Phase I 31 Potential Collision 1 2 1 2 3

32. Active Testing Cartoon: Phase II 32 1 2 1 2 3

33. Outline  Motivation  Deterministic Specification  Checking: Active Testing  Experimental Evaluation  Future Work + Conclusions

34. Ease of Asserting Determinism  Implemented a deterministic assertion library for Java.  Manually added deterministic assertions to 13 Java benchmarks with 200 – 4k LoC  Typically 5-10 minutes per benchmark Functional correctness very difficult.

35. Ease of Use: Example Deterministic.open(); Predicate eq = new Equals(); Deterministic.assume(width, eq); … (9 parameters total) … Deterministic.assume(gamma, eq); // Compute fractal in threads int matrix[][] = …; Deterministic.assert(matrix, eq); Deterministic.close();

36. Effectiveness in Finding Bugs  13 Java benchmarks of 200 – 4k LoC  Ran benchmarks on ~100 schedules Schedules with data races and other “interesting” interleavings (active testing)  For every pair of executions of deterministic Pre { P } Post: check that:

37. Experiments: Java Grande Forum BenchmarkLoC Data Races Found | Errors High-Level Races Found | Errors sor 300 2000 moldyn 1.3k 2000 lufact 1.5k 1000 raytracer 1.9k 3100 montecarlo 3.6k 1020

38. Experiments: Parallel Java Lib BenchmarkLoC Data Races Found | Errors High-Level Races Found | Errors pi 150 901+1 keysearch3 200 300+0 mandelbrot 250 900+0 phylogeny 4.4k 400+0 tsp* 700 6020

39. Experimental Evaluation  Across 13 benchmarks:  Found 40 data races. 1 violates deterministic assertions.

40. Experimental Evaluation  Across 13 benchmarks:  Found 40 data races. 1 violates deterministic assertions.  Found many interesting interleavings (non-atomic methods, lock races, etc.) 1 violates deterministic assertions.

41. Determinism Violation  Pair of calls to nextDouble() must be atomic. deterministic { // N trials in parallel. foreach (n = 0; n < N; n++) { x = Random.nextDouble(); y = Random.nextDouble(); … } } assert (|pi - pi’| < 1e-10) deterministic { // N trials in parallel. foreach (n = 0; n < N; n++) { x = Random.nextDouble(); y = Random.nextDouble(); … } } assert (|pi - pi’| < 1e-10)

42. Outline  Motivation  Deterministic Specification  Checking: Active Testing  Experimental Evaluation  Future Work + Conclusions

43. How to verify: Essential Rule deterministic assume(Φ 1 ) A 11 ;A 12 ;A 13 || A 21 ;A 22 ;A 23 || A 31 ;A 32 ;A 33 } assert (Φ 2 )

44. How to verify: Essential Rule deterministic assume(Φ 1 ) A 11 ;A 12 ;A 13 || A 21 ;A 22 ;A 23 || A 31 ;A 32 ;A 33 } assert (Φ 2 ) Find an invariant Φ and prove that for all i, j, l, m deterministic assume(Φ) A ij || A lm } assert (Φ) and Φ 1 → Φ and Φ → Φ 2

45. Prove for the following deterministic assume(true) { parallel while (!wq.is_empty()) { [work = wq.get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } } assert(best=best’)

46. Decompose the proof Let us prove that P ≈ S That is prove that deterministic assume(Pre(S 0,S’ 0 )) { if (*) P else S } assert (Post(S,S’))

47. Decompose the proof Let us prove that P ≈ S Create a non deterministic sequential program NS [Galois] P ≡ NS ≈ S

48. Decompose the proof Let us prove that P ≈ S Create a non deterministic sequential program NS P ≡ NS ≈ S deterministic assume(S 0 =S’ 0 ) { if (*) P else NS } assert (S=S’) deterministic assume(Pre(S 0,S’ 0 )) { if (*) NS else S } assert (Post(S,S’))

49. How to verify? while (!wq.is_empty()) { [work = wq.nondet_get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } deterministic assume(true) { parallel while (!wq.is_empty()) { [work = wq.get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } } assert(best=best’) Not deterministic sequential program

50. How to verify? while (!wq.is_empty()) { [work = wq.nondet_get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } Barrier() deterministic assume(true) { parallel while (!wq.is_empty()) { [work = wq.get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } } assert(best=best’) Not deterministic sequential program

51. How to verify? while (!wq.is_empty()) { [work = wq.nondet_get();] if (bound(work) >= best && non_det) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } Barrier() deterministic assume(true) { parallel while (!wq.is_empty()) { [work = wq.get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } } assert(best=best’) Not deterministic sequential program

52. Verifying Determinism  Verify determinism of each piece.  No need to consider cross product of all interleavings. P P P P P P

53. Verifying Determinism  Compositional reasoning for determinism? P Q Q Q Q Q Q P P

54. Summary  “Bridge” predicates and assertions Simple to assert natural determinism Semantic determinism  Active Testing Direct search based on imprecise bug reports  Verify/prove determinism Prove exact equivalence with non deterministic sequential programs Prove semantic equivalence between non deterministic sequential programs and sequential programs

57. Deterministic Assertion Library  Implemented assertion library for Java:  Records set to check: eq.apply(set 0,set 0 ’) => eq.apply(set,set’) Predicate eq = new Equals(); Deterministic.open(); Deterministic.assume(set, eq);... Deterministic.assert(set, eq); Deterministic.close(); Predicate eq = new Equals(); Deterministic.open(); Deterministic.assume(set, eq);... Deterministic.assert(set, eq); Deterministic.close();

58. Checking Determinism  Run P on some number of schedules.  For every pair and of executions of P: deterministic assume Pre(s 0,s 0 ’) { P } assert Post(s 1,s 1 ’) deterministic assume Pre(s 0,s 0 ’) { P } assert Post(s 1,s 1 ’)

59. Outline  Motivation (3 min)  Deterministic Specification (6 min)  Experimental Evaluation (5 min)  Related Work (3 min)  Future Work + Conclusions (3 min)