Prescient Memory: Exposing Weak Memory Model Behavior by Looking into the Future MAN CAO JAKE ROEMER ARITRA SENGUPTA MICHAEL D. BOND 1

Parallel Programming is Hard 2

Shared-memory 3 Main Memory CPU Cache CPU Cache CPU Cache CPU Cache

Parallel Programming is Hard Shared-memory Difficult to be both correct and scalable 4 Main Memory CPU Cache CPU Cache CPU Cache CPU Cache

Parallel Programming is Hard Shared-memory Difficult to be both correct and scalable Data race 5 Main Memory CPU Cache CPU Cache CPU Cache CPU Cache

Parallel Programming is Hard Shared-memory Difficult to be both correct and scalable Data race -Fundamentally, lacks strong semantic guarantees 6 Main Memory CPU Cache CPU Cache CPU Cache CPU Cache

Example #1: Weak Semantics 7 T1 T2 data = new Foo(); flag = true; if (flag) data.bar(); Foo data = null; boolean flag= false;

Example #1: Weak Semantics 8 Null pointer exception! T1 T2 data = new Foo(); flag = true; if (flag) data.bar(); Foo data = null; boolean flag= false;

Example #1: Weak Semantics 9 No data dependence T1 T2 data = new Foo(); flag = true; Foo data = null; boolean flag= false; Null pointer exception! if (flag) data.bar();

Exposing Behaviors of Data Races Existing Approaches Dynamic analyses Model checkers 10

Exposing Behaviors of Data Races Existing Approaches Dynamic analyses - Limitation: coverage Model checkers - Limitation: scalability 11

Exposing Behaviors of Data Races Existing Approaches Dynamic analyses - Limitation: coverage Model checkers - Limitation: scalability Prescient Memory (PM) Dynamic analysis with better coverage 12

Outline Memory Models and Behaviors of Data Races Design Prescient Memory (PM) PM-profiler PM Workflow Evaluation 13

Memory Model 14 Defines possible values that a load can return

Memory Model Defines possible values that a load can return 15 Sequential Consistency (SC) Impractical to enforce Strong

Memory Model Defines possible values that a load can return 16 Sequential Consistency (SC) Impractical to enforce Strong Enables compiler & hardware optimizations DRF0, C++11, Java Weak

Behaviors Allowed by Memory Models 17 DRF0 Memory Model Java Memory Model (JMM)

Behaviors Allowed by Memory Models 18 DRF0 Memory Model Java Memory Model (JMM) Racy execution No semantics Data-race-free execution Strong semantics (SC)

Behaviors Allowed by Memory Models 19 DRF0 Memory Model Java Memory Model (JMM) Racy execution No semantics Data-race-free execution Strong semantics (SC) Racy execution Weak semantics Data-race-free execution Strong semantics (SC)

Behaviors Allowed by Memory Models 20 DRF0 Memory Model Java Memory Model (JMM) Racy execution No semantics Data-race-free execution Strong semantics (SC) Racy execution Weak semantics Data-race-free execution Strong semantics (SC) Racy execution can still lead to surprising behaviors!

Behaviors Allowed in JMM #1: Revisit 21 T1 T2 data = new Foo(); flag = true; if (flag) data.bar(); Foo data = null; boolean flag= false;

Behaviors Allowed in JMM #1: Revisit 22 T1 T2 data = new Foo(); flag = true; if (flag) data.bar(); Foo data = null; boolean flag= false; stale value latest value

Behaviors Allowed in JMM #1: Revisit 23 Null pointer exception! T1 T2 data = new Foo(); flag = true; if (flag) data.bar(); Foo data = null; boolean flag= false; stale value latest value

Behaviors Allowed in JMM #1: Revisit 24 T1 T2 data = new Foo(); flag = true; if (flag) data.bar(); Foo data = null; boolean flag= false; Returning stale value can trigger the exception

Behaviors Allowed in JMM #2 25 r = data; flag = 1; T1T2 int data = flag = 0; while (flag == 0) {} data = 1; assert r == 0;

Behaviors Allowed in JMM #2 26 r = data; flag = 1; T1T2 int data = flag = 0; while (flag == 0) {} data = 1; assert r == 0;

Behaviors Allowed in JMM #2 27 r = data; flag = 1; T1T2 int data = flag = 0; while (flag == 0) {} data = 1; assert r == 0; latest value future value

Behaviors Allowed in JMM #2 28 r = data; flag = 1; T1T2 int data = flag = 0; while (flag == 0) {} data = 1; assert r == 0; latest value future value Valid due to lack of happens-before ordering

Behaviors Allowed in JMM #2 29 r = data; flag = 1; T1T2 int data = flag = 0; while (flag == 0) {} data = 1; assert r == 0; latest value future value Assertion failure!

Behaviors Allowed in JMM #2 30 r = data; flag = 1; T1T2 int data = flag = 0; while (flag == 0) {} data = 1; assert r == 0; Assertion failure!

Behaviors Allowed in JMM #2 31 r = data; flag = 1; T1T2 int data = flag = 0; while (flag == 0) {} data = 1; assert r == 0; Requires returning future value or reordering to trigger the assertion failure

Example #3 32 r1 = x; y = r1; T1T2 r2 = y; if (r2 == 1) { r3 = y; x = r3; } else x = 1; assert r2 == 0; int x = y = 0;

Example #3 33 r1 = x; y = r1; T1T2 r2 = y; if (r2 == 1) { r3 = y; x = r3; } else x = 1; assert r2 == 0; int x = y = 0; JMM disallows r2 == 1 because of causality requirements – Ševčík and Aspinall, ECOOP, 2008

Example #3 34 r1 = x; y = r1; T1T2 r2 = y; if (r2 == 1) { r3 = r2; x = r3; } else x = 1; assert r2 == 0; int x = y = 0; However, in a JVM, after redundant read elimination

Example #3 35 r1 = x; y = r1; T1T2 r2 = y; if (r2 == 1) { r3 = r2; x = r3; } else x = 1; assert r2 == 0; int x = y = 0; However, in a JVM, after redundant read elimination r2 = y; If (r2 == 1) x = r2; else x = 1;

Example #3 36 r1 = x; y = r1; T1T2 r2 = y; if (r2 == 1) { r3 = r2; x = r3; } else x = 1; assert r2 == 0; int x = y = 0; However, in a JVM, after redundant read elimination r2 = y; x = 1; r2 = y; If (r2 == 1) x = r2; else x = 1;

Example #3 37 r1 = x; y = r1; T1T2 r2 = y; if (r2 == 1) { r3 = r2; x = r3; } else x = 1; assert r2 == 0; int x = y = 0; However, in a JVM, after redundant read elimination r2 = y; x = 1; r2 = y; If (r2 == 1) x = r2; else x = 1; Assertion failure possible!

Behaviors Allowed by Memory Models and JVMs 38 DRF0 Memory Model Java Memory Model Typical JVMs

Behaviors Allowed by Memory Models and JVMs 39 DRF0 Memory Model Java Memory Model Typical JVMs Unsatisfactory, impractical to enforce

Exposing Behaviors of Example #3 40 T1T2 int x = y = 0; Consider future value r1 = x; y = r1; r2 = y; if (r2 == 1) { r3 = y; x = r3; } else x = 1; assert r2 == 0;

Exposing Behaviors of Example #3 41 r1 = x; // r1 = 1 y = r1; // y = 1 T1T2 r2 = y; // r2 = 1 if (r2 == 1) { r3 = y; // r3 = 1 x = r3; // x = 1 } else x = 1; assert r2 == 0; int x = y = 0; Consider future value

Exposing Behaviors of Example #3 42 r1 = x; // r1 = 1 y = r1; // y = 1 T1T2 r2 = y; // r2 = 1 if (r2 == 1) { r3 = y; // r3 = 1 x = r3; // x = 1 } else x = 1; assert r2 == 0; int x = y = 0; Consider future value r1 = 1 justified! Assertion failure!

Exposing Behaviors of Example #3 43 r1 = x; y = r1; T1T2 r2 = y; if (r2 == 1) { r3 = y; x = r3; } else x = 1; assert r2 == 0; int x = y = 0; Requires returning future value or compiler optimization and reordering to trigger the assertion failure

Exposing Behaviors with Dynamic Analyses Typical approaches Simulate weak memory models behaviors [1,2,3] Explore multiple thread interleavings [4, 5] 44 1.Adversarial Memory, Flanagan & Freund, PLDI’09 2.Relaxer, Burnim et al, ISSTA’11 3.Portend+, Kasikci et al, TOPLAS’15 4.Replay Analysis, Narayanasamy et al, PLDI’07 5.RaceFuzzer, Sen, PLDI’08

Exposing Behaviors with Dynamic Analyses Typical approaches Simulate weak memory models behaviors [1,2,3] Explore multiple thread interleavings [4, 5] Coverage Limitation Return stale values only, not future values Cannot expose assertion failures in Examples #2, # Adversarial Memory, Flanagan & Freund, PLDI’09 2.Relaxer, Burnim et al, ISSTA’11 3.Portend+, Kasikci et al, TOPLAS’15 4.Replay Analysis, Narayanasamy et al, PLDI’07 5.RaceFuzzer, Sen, PLDI’08

Relationship among memory models and exposed behaviors 46 DRF0 Memory Model Java Memory Model Existing Dynamic Analyses Typical JVMs

Relationship among memory models and exposed behaviors 47 DRF0 Memory Model Our Goal Java Memory Model Existing Dynamic Analyses Typical JVMs

Relationship among memory models and exposed behaviors 48 DRF0 Memory Model Our Goal Java Memory Model Existing Dynamic Analyses Typical JVMs Example #1 data = new Foo(); if (flag) flag = true; data.bar(); Example #2 r = data; while (flag == 0) {} flag = 1; data = 1; Example #3 r1 = x; r2 = y; y = r1; if (r2 == 1) { r3 = y; x = r3; } else x = 1;

Relationship among memory models and exposed behaviors 49 DRF0 Memory Model Our Goal Java Memory Model Existing Dynamic Analyses Typical JVMs Example #1 data = new Foo(); if (flag) flag = true; data.bar(); Example #2 r = data; while (flag == 0) {} flag = 1; data = 1; Example #3 r1 = x; r2 = y; y = r1; if (r2 == 1) { r3 = y; x = r3; } else x = 1; Real-world evidence is valuable here!

Outline Memory Models and Behaviors of Data Races Design Prescient Memory (PM) PM-profiler PM Workflow Evaluation 50

Prescient Memory: Key Idea Speculatively “guess” a future value at a load Validate the speculative value at a later store 51

Prescient Memory: Key Idea Speculatively “guess” a future value at a load Validate the speculative value at a later store 52

Returning Future Values is Tricky 53 r1 = x; y = r1; T1T2 r2 = y; if (r2 == 0) x = 1; assert r1 == 0 || r2 == 0; int x = y = 0;

Returning Future Values is Tricky 54 r1 = x; y = r1; T1T2 assert r1 == 0 || r2 == 0; int x = y = 0; r2 = y; if (r2 == 0) x = 1;

Returning Future Values is Tricky 55 T1T2 int x = y = 0; r1 = x; // r1 = 1 y = r1; // y = 1 assert r1 == 0 || r2 == 0; r2 = y; // r2 = 1 if (r2 == 0) x = 1;

Returning Future Values is Tricky 56 T1T2 int x = y = 0; r1 = x; // r1 = 1 y = r1; // y = 1 assert r1 == 0 || r2 == 0; r2 = y; // r2 = 1 if (r2 == 0) x = 1; r1 = 1 not justified

Returning Future Values is Tricky 57 T1T2 int x = y = 0; r1 = x; // r1 = 1 y = r1; // y = 1 assert r1 == 0 || r2 == 0; r2 = y; // r2 = 1 if (r2 == 0) x = 1; r1 = 1 not justified Invalid execution!

Returning Future Values is Tricky 58 T1T2 int x = y = 0; r1 = x; y = r1; assert r1 == 0 || r2 == 0; r2 = y; if (r2 == 0) x = 1; Should never fail!

Returning Future Values is Tricky 59 r1 = x; y = r1; T1T2 r2 = y; if (r2 == 0) x = 1; assert r1 == 0 || r2 == 0; int x = y = 0; Validating speculative values is necessary to prevent nonsensical results

Prescient Memory: Key Idea Speculatively “guess” a future value at a load Validate the speculative value at a later store 60

Prescient Memory: Key Idea Speculatively “guess” a future value at a load Validate the speculative value at a later store 61 Valid future value Store writes the same value Store races with load

Prescient Memory: Key Idea Speculatively “guess” a future value at a load Maintain a per-variable speculative read history Records Validate the speculative value at a later store 62 Valid future value Store writes the same value Store races with load

PM Example 63 1: r = x; 2: y = 1; T1T2 int x = y = 0; 3: while (y == 0) {} 4: x = 1; assert r == 0; S[x] = ∅ Timestamp: K 1 Timestamp: K 2

PM Example 64 1: r = x; 2: y = 1; T1T2 int x = y = 0; 3: while (y == 0) {} 4: x = 1; 1 ⇠ predict(…) // guess value 1 S[x] = { } S[x] = ∅ assert r == 0; Timestamp: K 1 Timestamp: K 2

PM Example 65 1: r = x; 2: y = 1; T1T2 int x = y = 0; 3: while (y == 0) {} 4: x = 1; S[x] = ∅ validate S[x]: K 1 ⋢ K 2 && 1 == 1 1 is a valid future value! assert r == 0; Timestamp: K 1 Timestamp: K 2 1 ⇠ predict(…) // guess value 1 S[x] = { }

Challenges How to guess a future value? 66 predict(…) ?

Challenges How to guess a future value? Which load should return a future value? What value should be returned? 67

Challenges How to guess a future value? Which load should return a future value? What value should be returned? Solution Profile possible future values in a prior run 68

Profiling Future Values Helper Dynamic Analysis: PM-profiler Maintains a per-variable concrete read history At a load, records: 69

Profiling Future Values Helper Dynamic Analysis: PM-profiler At a store, detects: 70 Potential future value for a previous load Store races with the previous load Store writes a value distinct from visible values of the previous load

Prescient Memory Workflow 71 Data race detector PM-ProfilerPM Racy accesses Potential future values and loads First Execution Second Execution

Prescient Memory Workflow 72 Data race detector PM-ProfilerPM Racy accesses Potential future values and loads First Execution Second Execution Run-to-run nondeterminism affects validatable future values

Prescient Memory Workflow 73 Data race detector PM-ProfilerPM Racy accesses Potential future values and loads First Execution Second Execution Run-to-run nondeterminism affects validatable future values Solution: record and replay

Prescient Memory Workflow 74 Data race detector PM-ProfilerPM Fuzzy ReplayRecord Racy accesses Potential future values and loads

Prescient Memory Workflow 75 Data race detector PM-ProfilerPM Fuzzy ReplayRecord Racy accesses Potential future values and loads Returning a future value could diverge from the record execution Best-effort, fuzzy replay

Outline Memory Models and Behaviors of Data Races Design Prescient Memory (PM) PM-profiler PM Workflow Evaluation 76

Methodology and Implementation ◦Compare with Adversarial Memory (AM) [Flanagan & Freund, PLDI’09]: a dynamic analysis that only uses stale values 77

Methodology and Implementation ◦Compare with Adversarial Memory (AM) [Flanagan & Freund, PLDI’09]: a dynamic analysis that only uses stale values ◦Platform Jikes RVM DaCapo Benchmark 2006, 2009 and SPEC JBB 2000 & Core Intel Core i Record and Replay [Replay, Bond et al. PPPJ’15] 78

Methodology and Implementation ◦Compare with Adversarial Memory (AM) [Flanagan & Freund, PLDI’09]: a dynamic analysis that only uses stale values ◦Platform Jikes RVM DaCapo Benchmark 2006, 2009 and SPEC JBB 2000 & Core Intel Core i Record and Replay [Replay, Bond et al. PPPJ’15] ◦Implementation limitation Does not support reference-type fields 79

Exposed Erroneous Behaviors ProgramAMPM hsqldbNon-terminationData corruption hsqldbNonePerformance bug avroraData corruption lusearch (GNU Classpath)Performance bugNone sunflowNull ptr exception jbb2000Non-terminationData corruption jbb2000Data corruption jbb2005 (GNU Classpath)Data corruption jbb2005 (GNU Classpath)Data corruptionNone 80

Exposed Erroneous Behaviors 81 PM found 3 new erroneous behaviors! ProgramAMPM hsqldbNon-terminationData corruption hsqldbNonePerformance bug avroraData corruption lusearch (GNU Classpath)Performance bugNone sunflowNull ptr exception jbb2000Non-terminationData corruption jbb2000Data corruption jbb2005 (GNU Classpath)Data corruption jbb2005 (GNU Classpath)Data corruptionNone

Exposed Erroneous Behaviors 82 PM exposes most bugs that AM found. ProgramAMPM hsqldbNon-terminationData corruption hsqldbNonePerformance bug avroraData corruption lusearch (GNU Classpath)Performance bugNone sunflowNull ptr exception jbb2000Non-terminationData corruption jbb2000Data corruption jbb2005 (GNU Classpath)Data corruption jbb2005 (GNU Classpath)Data corruptionNone  

Exposed Erroneous Behaviors 83 Paper contains detailed analysis of each bug. ProgramAMPM hsqldbNon-terminationData corruption hsqldbNonePerformance bug avroraData corruption lusearch (GNU Classpath)Performance bugNone sunflowNull ptr exception jbb2000Non-terminationData corruption jbb2000Data corruption jbb2005 (GNU Classpath)Data corruption jbb2005 (GNU Classpath)Data corruptionNone

Conclusion First dynamic analysis to expose legal behaviors due to future values in large, real programs Successfully found new harmful behaviors due to future values in real programs Reaffirms that “benign” races are harmful Helps future revisions to language specifications by finding evidence of controversial behaviors in real programs 84

Backup slides 85

Data Races are Evil 86 Challenging to reason about correctness, unsafe software. Lacks semantic guarantees Atomicity violation, sequential consistency violation, nondeterminism. Leads to other concurrency errors Hard to avoid, detect, reproduce or eliminate Widespread

DRF0 Memory Model – Adve and Hill, ISCA, Racy execution No semantics Data-race-free execution Strong semantics Execution is sequentially consistent (SC)

Java Memory Model (JMM) Based on DRF0 Need to ensure memory and type safety 88 Racy execution Weak semantics Racy execution can still lead to surprising behaviors Unsatisfactory, impractical to enforce JVMs do not strictly conform to JMM

Relationship among memory models and exposed behaviors 89 DRF0 Memory Model Happens-Before Memory Model Our Goal Java Memory Model Existing Dynamic Analyses Typical JVMs

Relationship among memory models and exposed behaviors 90 DRF0 Memory Model Our Goal Java Memory Model Existing Dynamic Analyses Typical JVMs Out-of-thin-air results?

Example #5: Out-Of-Thin- Air Result 91 r1 = x; y = r1; T1 T2 r2 = y; x = r2 assert r1 != 42; int x = y = 0; Assertion can fail in DRF0!

Contrived Transformation 92 r1 = 42; y = r1; r3 = x; if (r3 != r1) y = r3; T1T2 r2 = y; x = r2 assert r1 != 42; int x = y = 0; r1 = x; y = r1; T1

Contrived Transformation 93 r1 = 42; y = r1; r3 = x; if (r3 != r1) y = r3; T1T2 r2 = y; x = r2 assert r1 != 42; int x = y = 0; r1 = x; y = r1; T1 Assertion can fail in DRF0!

Example #6: Out-of-thin-air results for a DRF program 94 r1 = x; if (r1 == 1) y = 1; T1 T2 r2 = y; if (r2 == 1) x = 1; assert r1 == 0 && r2 == 0; int x = y = 0; Cannot fail in DRF0 or JMM.

Example #7 95 r1 = z; if (r1 == 1) x = 1; T1 T2 r2 = x; if (r2 == 1) y = 1; else z = 1; assert r3 == 0; int x = y = z = 0; T3 r3 = y; if (r3 == 1) x = 1;

Example #7 96 r1 = z; if (r1 == 1) x = 1; T1 T2 r2 = x; if (r2 == 1) y = 1; else z = 1; assert r3 == 0; int x = y = z = 0; T3 r3 = y; if (r3 == 1) x = 1;

Example #7 97 r1 = z; // r1 = 0 if (r1 == 1) x = 1; T1 T2 r2 = x; // r2 = 1 if (r2 == 1) y = 1; else z = 1; assert r3 == 0; int x = y = z = 0; T3 r3 = y; // r3 = 1 if (r3 == 1) x = 1; // justifies r2 = 1

Model checkers Typical approaches Explore all behaviors allowed in the memory model Target routines for concurrent data structures Could expose behaviors due to future values Example: CDSChecker Coverage Limitation Do not scale to large, real programs 98

PM-profiler Example 99 1: r = x; 2: y = 1; T1T2 int x = y = 0; 3: while (y == 0) {} 4: x = 1; assert r == 0; RH[x] = ∅, RH[y] = ∅ Timestamp: K 1 Timestamp: K 2

PM-profiler Example 100 1: r = x; 2: y = 1; T1T2 int x = y = 0; 3: while (y == 0) {} 4: x = 1; assert r == 0; RH[x] = { } RH[x] = ∅, RH[y] = ∅ Timestamp: K 1 Timestamp: K 2

PM-profiler Example 101 1: r = x; 2: y = 1; T1T2 int x = y = 0; 3: while (y == 0) {} 4: x = 1; assert r == 0; RH[x] = { } RH[x] = ∅, RH[y] = ∅ check RH[y]: Timestamp: K 1 Timestamp: K 2 no previous load!

PM-profiler Example 102 1: r = x; 2: y = 1; T1T2 int x = y = 0; 3: while (y == 0) {} 4: x = 1; RH[x] = ∅, RH[y] = ∅ RH[y] = { } assert r == 0; Timestamp: K 1 Timestamp: K 2 RH[x] = { }

PM-profiler Example 103 1: r = x; 2: y = 1; T1T2 int x = y = 0; 3: while (y == 0) {} 4: x = 1; RH[x] = { } RH[x] = ∅, RH[y] = ∅ RH[y] = { } check RH[x]: K 1 ⋢ K 2 && 1 ∉ {0} 1 is a potential future value for load at assert r == 0; Timestamp: K 1 Timestamp: K 2