1. Tools for the development of parallel applications
Assaf Schuster
Technion – Israel Institute of Technology
4/28/2019
Intel MultiCore Conference

2. Intel MultiCore Conference
Message Passing?
People will not give up shared memory programming
Do not want to learn messaging APIs
Shared memory considered a “natural” extension of serial code
Distributed computing sounds scary; multithreading? “I’ve tried it, it worked ”
Shared memory considered “easier to implement” on upper layers of memory hierarchy
4/28/2019
Intel MultiCore Conference

3. Multithreading Issue #1: Debugging Tools
The hard problems: over- and under-synchronization
Very few tools
No full coverage
No precise tools
No scalable tools
No educational material
Some positive research techniques – very limited
4/28/2019
Intel MultiCore Conference

4. Intel MultiCore Conference
Therac 25
A medical radiation machine to treat cancer
6 patients got a radiation overdose
4 died
2 injured
4/28/2019
Intel MultiCore Conference

5. State of the Art - Industry
Intel Thread Checker?
IBM’s Contest:
Add yields on random locations
Expose deserted interleavings
Watch the output: any problem?
Look for the causes of the problem in the interleaving
Steps 2 & 3 human-based
 “travel debugging”
4/28/2019
Intel MultiCore Conference

6. State of the Art – Research
On the fly DR detection for C++ programs
Instrumentation-based
4/28/2019
[Pozniansky and Schuster, PPoPP 2003] (4-way IBM Netfinity server, 550MHz, Win-NT)

7. Playing With Detection Granularity
Coverage: a single interleaving
4/28/2019
[Pozniansky and Schuster, PPoPP 2003] (4-way IBM Netfinity server, 550MHz, Win-NT)

8. State of the Art - Research
Model Checking DRs (IBM tools)
Boosted with Dynamic Information (Lockset)
Provides a witness to the race
Detects races in many interleavings
……..
Scalability: 4 threads max
Scalability: Sequential Consistency only
[Sagiv, Shacham, Schuster, PPoPP 2005]
4/28/2019
Intel MultiCore Conference

9. Multithreading Issue #2: Memory Models
Observation: most current multithreaded “scalable” applications were/are created by programmers who do not know what MM is all about.
No educational material
“Here’s my new great book: introduction to parallel programming…”
Existing debugging tools ignore MM
4/28/2019
Intel MultiCore Conference

10. Memory Models Education
T1 T2
======= =======
Initially X=Y=0
Print(X) Print(Y)
Y = 1 X = 1
Is <<< 1 1 >>> a legitimate output?
4/28/2019
Intel MultiCore Conference

11. Intel MultiCore Conference
Java Memory Model
The first popular language attempting a “formal definition”. JLS 1995, 17 pages.
Theorem: JMM is Coherent [Gontmakher and Schuster, IPDPS 1997]
Did they know?
Consequence: ***All*** tested JVMs in breach JMM specification.
4/28/2019
Intel MultiCore Conference

12. Coherence makes reads prevent common compiler optimizations
p and q might point to same object
p.x = 0
p.x=1
a=p.x
b=q.x
assert(p==q  a  b  c)
Cannot put c=a
c=p.x
reads can make a
process see writes
by another process.
The read “kills” later
reuse of local values.
Coherence, prevents important compiler optimizations.
One of the common optimizations is a reuse of a local copy of a global variable.
When there is a read of another global variable which may be equal to the first, this might be a read of a modified value of the global variable, and thus the previously read value cannot be reused, otherwise Coherence is violated.
KONST> “<=“ to “”
4/28/2019
Intel MultiCore Conference

13. Intel MultiCore Conference
August 2004: A New JMM
Java Community Process, JSR-133
Bill Pugh
Sarita Adve
And others
Took three years (could have been longer)
Is now being adopted as a standard for C++
But is it any good??????
4/28/2019
Intel MultiCore Conference

14. Causality requirements: guarantees for correctly synchronized programs
The program is correctly synchronized if it is data-race free on a sequentially consistent platform
Java must guarantee sequentially consistent behavior for correctly synchronized program
Initially, x = y = 0
Thread | Thread 2
r1 = x; r2 = y;
if (r1 != 0) if (r2 != 0)
y = 1; x = 1;
r1 == r2 == 0 is the only legal behavior
4/28/2019
Intel MultiCore Conference

15. Causality Requirements: Safety Guarantees42 42
justified
Initially, x = y = 0
Thread | Thread 2
r1 = x; r2 = y;
y = r1; x = r2;
Incorrectly synchronized.
Still: r1 == r2 == 42
must not be allowed 42
Java must provide safety guarantees for incorrectly synchronized programs: values must not appear “out of thin air”
4/28/2019
Intel MultiCore Conference

16. Intel MultiCore Conference
Causality Test Case 6
Initially, A = B = 0
Thread | Thread 2
1: r1 = A; | 3: r2 = B;
if (r1 == 1) | if (r2 == 1)
2: B = 1; | 4: A = 1;
| if (r2 == 0)
| 5: A = 1;
r1 == r2 == 1 ALLOWED?!
4/28/2019
Intel MultiCore Conference

17. Intel MultiCore Conference
Conclusions
Parallelization algorithms, load balancing, most of the work in 30 years -- usually non issues
Debugging tools for multithreading
We do not have good technology, yet
And may never have, a hard problem
Memory models for shared memory
We do not know what we want, yet
I do not see programmers understand them
Solution
Better programming paradigms
Dara-race free languages
Valiant’s BSP (shared memory-like, easy to debug)
Next panel!
4/28/2019
Intel MultiCore Conference