1. Java PathRelaxer: Extending JPF for JMM-Aware Model Checking Huafeng Jin, Tuba Yavuz-Kahveci, and Beverly Sanders Computer and Information Science and Engineering University of Florida

2. Contents  Memory Model  The Java Memory Model  Algorithm  Implementation  Experience  Conclusion

3. Contents  Memory Model  The Java Memory Model  Algorithm  Implementation  Experience  Conclusion

4.  Specifies “which value each read of a memory location may return”.  Sequentially consistent (SC) memory model  Memory actions must execute one at a time in a single total single order  Read always see the value of the most recent write to that memory location.  Relaxed memory models  PSO, TSO, Java Memory Model (JMM), etc. Memory Model

5.  Specifies “which value each read of a memory location may return”.  Sequentially consistent (SC) memory model  Memory actions must execute one at a time in a single total single order  Read always see the value of the most recent write to that memory location.  Relaxed memory models  PSO, TSO, Java Memory Model (JMM), etc. Memory Model JPF assumes SC memory model

6. Example Intially, x = 0, done = false Intially, x = 0, done = false SCMM r == 1 Thread-1Thread-2 x = 1; done = true; while (!done){/*spin*/} r = x;

7. Example Intially, x = 0, done = false Intially, x = 0, done = false Thread-1Thread-2 x = 1; done = true; while (!done){/*spin*/} r = x; SCMM r == 1 JMM r == 0 ˅ r == 1

8. Java’s String class public final class String{ private final char value[]; private final int offset; private final int count; private int hash; //default 0 … public int hashCode(){ int h = hash, len = count; //read of hash if (h == 0 && len > 0){ … /*calculate hash code locally and assign to h*/ hash = h; //write of hash } return h; }  Data race is benign in both SC MM and JMM

9. Another Version public final class String{ private final char value[]; private final int offset; private final int count; private int hash; //default 0 … public int hashCode(){ int h = hash, len = count; //read of hash if (h == 0 && len > 0){ … /*calculate hash code locally and assign to h*/ hash = h; //write of hash } h = hash; //read of hash return h; }  Benign in SC MM but not benign in JMM

10. Another Version public final class String{ private final char value[]; private final int offset; private final int count; private int hash; //default 0 … public int hashCode(){ int h = hash, len = count; //read of hash if (h == 0 && len > 0){ … /*calculate hash code locally and assign to h*/ hash = h; //write of hash } h = hash; //read of hash return h; }  Benign in SC MM but not benign in JMM Return hash code or 0

11.  JPF: generates executions under SC memory model.  JPR: generates executions under an overapproximation of JMM. Extending JPF

12. Contents  Memory Model  The Java Memory Model  Algorithm  Implementation  Experience  Conclusion

13.  SC memory model:  Read sees most recent write to that location.  Java memory model:  Read sees any write (past/future) to that location provided the execution is  Well-formed  Meets causality constraints Overview of JMM

14.  Action (memory related)  Action (memory related)  Non-synchronization actions:  non-volatile write, non-volatile read  Synchronization actions:  volatile write, volatile read, lock, unlock, thread start, thread join, … JMM Action tThread ID kAction kind (volatile read/write, non-volatile read/write, lock/unlock, thread start, thread join …) vVariable/monitor uUnique action ID

15.  Execution E  Execution E JMM Execution AFinite set of actions PProgram ≤ po Program order, a partial order over A based on each thread’s sequence. ≤ so Synchronization order, a total order over all the synchronization actions in A WWrite-seen function, maps each read action to the write action it sees VValue-written function, maps each write action to the value it writes

16.  A partial order over actions with regard to ≤ so Synchronizes-with Order ≤ sw unlock(x) ≤ sw subsequent lock(x) volatile write(x)subsequent volatile read(x) start thread t1 st action of thread t Write of default value1 st action in each thread

17.  A partial order over actions by taking transitive closure of ≤ po and ≤ sw Initially, x == 0 ⋀ done == false, done is volatile Happens-before Order ≤ hb Thread-1Thread-2 x = 1; done = true while (!done){/*spin*/} r = x; ≤ po ≤ sw

18. Thread-1Thread-2 x = 1; done = true while (!done){/*spin*/} r = x;  A partial order over actions by taking transitive closure of ≤ po and ≤ sw Initially, x == 0 ⋀ done == false, done is volatile Happens-before Order ≤ hb ≤ po ≤ sw ≤ hb

19.  In an execution Data Race Thread-1: … Write … Thread-2: … Read … x ≤ hb

20.  A program: If all the SC executions are free of data races, it is Data-Race-Free program (DRF). If all the SC executions are free of data races, it is Data-Race-Free program (DRF).  DRF Guarantee:  Any legal execution of DRF program is SC. Data Race Free

21.  For all reads r of variable v, it cannot be  r ≤ hb W(r)  W(r) ≤ hb w ≤ hb r (w writes to v) Well-formed Execution

22.  r can only be 1, not 0 Initially, x == 0 ⋀ done == false, done is volatile Example Thread-1Thread-2 x = 1; done = true while(!done) {/*spin*/} r = x; ≤ po ≤ sw If read x = 0, then there is an interleaving write x = 1.

23. An execution E with ≤ hb is legal if there is a finite sequence of set of actions C i and well- formed executions E i with ≤ hbi and ≤ swi such that C 0 = ∅, C i ⊆ C i-1 for all i > 0, ∪ C i = A, and for each i > 0 the following rules are satisfied: An execution E with ≤ hb is legal if there is a finite sequence of set of actions C i and well- formed executions E i with ≤ hbi and ≤ swi such that C 0 = ∅, C i ⊆ C i-1 for all i > 0, ∪ C i = A, and for each i > 0 the following rules are satisfied: Causality Rules (complicated)

24. An execution E with ≤ hb is legal if there is a finite sequence of set of actions C i and well- formed executions E i with ≤ hbi and ≤ swi such that C 0 = ∅, C i ⊆ C i-1 for all i > 0, ∪ C i = A, and for each i > 0 the following rules are satisfied: An execution E with ≤ hb is legal if there is a finite sequence of set of actions C i and well- formed executions E i with ≤ hbi and ≤ swi such that C 0 = ∅, C i ⊆ C i-1 for all i > 0, ∪ C i = A, and for each i > 0 the following rules are satisfied: Causality Rules (complicated) E → E 1 → E 2 → … → E i ∅ C1C1 C2C2 C i-1 Justify

25.  Causality Rules:  Rules out out-of-thin-air values  Example: Initially, x == y == 0, x and y are non-volatile r1 == r2 == 42 is out-of-thin-air value r1 == r2 == 42 is out-of-thin-air value Out-of-thin-air Value Thread-1Thread-2 r1 = x;r2 = y; y = r1;x = r2;

26. Contents  Memory Model  The Java Memory Model  Algorithm  Implementation  Experience  Conclusion

27.  Fixed-point semantics  Overapproximation of JMM  WriteSet JPR Overview WriteSet Write: add values to Read: Pick value from JPF

28. Structure of JPR JPR Driver JPFJMMListener WriteSet old WriteSet new Events Iterative calls Bytecode of the target program

29.  JPF’s state representation is extended with the following metadata: Metadata WriteSetMemLoc → 2 Aid × Val Collect write values ActionSet2 Action Current set of actions HBSet2 Aid × Aid Collect ≤ hb relations ImposeSet2 Aid × Val Rule out some out-of-thin-air values ReadAid → Aid × ValRecord W(r) and V(W(r)) WriteAid → ValRecord V(w)

30. Initially, x == y == 0, x and y are non-volatile. Under JMM, r1 == 1 ⋀ r2 == 1 is possible. Example Thread-1Thread-2 r1 = x;r2 = y; y = 1;x = 1;

31. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = { }, WS(y) = { } IS = ∅ 1 st iteration GWS = ∅ init

32. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = { }, WS(y) = { } IS = ∅ R(A1) =, legal past read A1; r1 = x; 1 st iteration initA1

33. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = { }, WS(y) = {, } IS = ∅, R(A1) = A1; r1 = x; A2: y = 1; 1 st iteration initA1A2

34. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = { }, WS(y) = {, } IS = ∅, R(A1) = A1; r1 = x; A2: y = 1; B1: r2 = y; 1 st iteration init A1A2 B1

35. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = { }, WS(y) = {, } IS = ∅, R(A1) =, R(B1) = legal past read A1; r1 = x; A2: y = 1; B1: r2 = y; 0 1 st iteration init A1A2 B1

36. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = {, }, WS(y) = {, } IS = ∅, R(A1) =, R(B1) = A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; r1 = 0, r2 = 0 1 st iteration init A1A2 B1B2

37. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = {, }, WS(y) = {, } IS = ∅, R(A1) =, R(B1) = legal past read A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; 1 1 st iteration init A1A2 B1

38. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; r1 = 0, r2 = 1 1 WS(x) = {, }, WS(y) = {, } IS = ∅, R(A1) =, R(B1) = 1 st iteration init A1A2 B1B2

39. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; 1 B1: r2 = y; WS(x) = { }, WS(y) = { } IS = ∅, R(A1) =, R(B1) = 0 legal past read 1 st iteration init A1 B1

40. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; 1 B1: r2 = y; A2: y = 1; WS(x) = { }, WS(y) = {, } IS = ∅, R(A1) =, R(B1) = 1 st iteration init A1A2 B1

41. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; 1 B1: r2 = y; A2: y = 1; B2: x = 1; WS(x) = {, }, WS(y) = {, } IS = ∅, R(A1) =, R(B1) = r1 = 0, r2 = 0 1 st iteration init A1A2 B1B2

42. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; 1 B1: r2 = y; A2: y = 1; B2: x = 1; A2: y = 1; WS(x) = {, }, WS(y) = {, } IS = ∅, R(A1) =, R(B1) = r1 = 0, r2 = 0 1 st iteration init A1A2 B1B2

43. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; 1 B1: r2 = y; A2: y = 1; B2: x = 1; A2: y = 1; B1: r2 = y; WS(x) = { }, WS(y) = { } IS = ∅ R(B1) = legal past read 1 st iteration initB1

44. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; 1 B1: r2 = y; A2: y = 1; B2: x = 1; A2: y = 1; B1: r2 = y; A1; r1 = x; WS(x) = { }, WS(y) = { } IS = ∅ R(B1) =, R(A1) = legal past read 1 st iteration init A1 B1

45. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; 1 B1: r2 = y; A2: y = 1; B2: x = 1; A2: y = 1; B1: r2 = y; A1; r1 = x; A2: y = 1; B2: x = 1; A2: y = 1; r1 = 0, r2 = 0 1 st iteration

46. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 A1; r1 = x; A2: y = 1; B1: r2 = y; 0 B2: x = 1; 1 B1: r2 = y; A2: y = 1; B2: x = 1; A2: y = 1; B1: r2 = y; A1; r1 = x; A2: y = 1; B2: x = 1; A2: y = 1; B2: x = 1; A1; r1 = x; 0 A2: y = 1; 1 r1 = 0, r2 = 1 1 st iteration

47. r1 = 0, r2 = 1 The WriteSet collected after 1 st iteration is GWS(x) = {, } GWS(y) = {, } It is passed to the 2 nd iteration init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1;

48. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = {, }, WS(y) = {, } IS = ∅ 2 nd iteration GWS(x) = {, } GWS(y) = {, } init

49. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = {, }, WS(y) = {, } IS = ∅ 2 nd iteration A1: r1 = x; initA1

50. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = {, }, WS(y) = {, } IS = { }, R(A1) = potential future read 2 nd iteration A1: r1 = x; 0 1 … initA1

51. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = {, }, WS(y) = {, } IS = { }, R(A1) = 2 nd iteration A1: r1 = x; 0 1 A2: y = 1; … initA1A2

52. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = {, }, WS(y) = {, } IS = { }, R(A1) = 2 nd iteration A1: r1 = x; 0 1 A2: y = 1; B1: r2 = y; … init A1A2 B1

53. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = {, }, WS(y) = {, } IS = { }, R(A1) =, R(B1) = 2 nd iteration A1: r1 = x; 0 1 A2: y = 1; B1: r2 = y; 01 … … init A1A2 B1

54. init: x = 0, y = 0; Thread-1Thread-2 A1: r1 = x;B1: r2 = y; A2: y = 1;B2: x = 1; init: x = 0, y = 0 WS(x) = {, }, WS(y) = {, } IS = { }, justified R(A1) =, R(B1) = 2 nd iteration A1: r1 = x; 0 1 A2: y = 1; B1: r2 = y; 01 B2: x = 1; … … r1 = 1, r2 = 1 init A1A2 B1B2

55.  3 rd iteration generates the same global WriteSet as 2 nd iteration, so a fixed-point is reached.  Possible outcomes running JPR:  r1 == 0 ⋀ r2 == 0  r1 == 0 ⋀ r2 == 1  r1 == 1 ⋀ r2 == 0  r1 == 1 ⋀ r2 == 1 Example

56. Contents  Memory Model  The Java Memory Model  Algorithm  Implementation  Experience  Conclusion

57.  JRF (Java Racefinder) is a JPF extension used to precisely detect data races.  Kyunghee Kim, Eric Mercer, Neha Rungta, Tuba Yavuz-Kahveci, Beverly Sanders  http://babelfish.arc.nasa.gov/trac/jpf/wiki/proje cts/jpf-racefinder Working with JRF

58.  Data Race Free (DRF) Guarantee  For DRF programs, model checking under SC memory model is enough.  JPF is sufficient, no need to run JPR. Working with JRF

59. JRF JPF JPR DRF? Y DRF? N

60. Contents  Memory Model  The Java Memory Model  Algorithm  Implementation  Experience  Conclusion

61.  Group 1  tc1 – tc20 from JMM causality test cases http://www.cs.umd.edu/~pugh/java/memoryModel/unified Proposal/testcases.html  Group 2  Benign data races (hash code, is prime)  Group 3  Harmful data races (dcl, peterson, dekker) Testing Suites

62. Experiment Results Test Cases Time (milliseconds)

63. Experiment Results Test Cases Time (milliseconds) JPR takes much longer time than JPF:  Iterations  Data choice generators

64. Experiment Results Test Cases Number of states

65. Contents  Memory Model  The Java Memory Model  Algorithm  Implementation  Experience  Conclusion

66.  JPR:  Applies a fixed-point based semantic  Adds non-SC behaviors into JPF  Generates an overapproximiation of JMM Conclusion