1. A Novel Home Migration Protocol in Home-based DSM
W.J. Fang C.L. Wang W.Z. Zhu F.C.M. Lau
System Research Group
Department of Computer Science
The University of Hong Kong

2. Outline Background Problem Statement Our Approach
Performance Evaluation
Conclusion
4/19/2019

3. Distributed Java Virtual Machine (DJVM)
import java.util.*;
class worker extends Thread{
private long n;
public worker(long N){ n=N; }
public void run(){ long sum=0;
for(long i=0; i<n; i++) sum+=i;
System.out.println(“N=“+n+” Sum="+sum);} }
public class test { static final int N=100;
public static void main(String args[]){
worker [] w= new worker[N];
Random r = new Random();
for (int i=0; i<N; i++)
w[i] = new worker(r.nextLong());
for (int i=0; i<N; i++) w[i].start();
try{ for (int i=0; i<N; i++) w[i].join();}
catch (Exception e){}}
A distributed Java Virtual Machine (DJVM) consists of a group of extended JVMs running on a distributed environment to support true parallel execution of a multithreaded Java application.
A DJVM provides all the JVM services, that are compliant with the Java language specification.
Presents a Single System Image (SSI) of clusters to Java programs – an illusion that the program is running on a single machine, yet more powerful.
(Single System Image)
Global Object Space
DJVM
Thread
Exec
Class
JVM
JVM
JVM
JVM
Java thread
HKU JESSICA2 project:
4/19/2019

4. Global Object Space
Virtualizes a single object heap spanning on the whole cluster.
A DSM service in the distributed JVM.
Java Memory Model: Resembles lazy release consistency. T1 T2
Before lock: flush cache
Before unlock: write back
Access fault
4/19/2019

5. LRC - Homeless Multiple-Writer Protocol
False Sharing
Two processes simultaneously write on the different parts of the same shared object.
Ping-pong effect if using single-writer protocol.
Twin and Diff
Twin: the copy before writes.
Diff: the difference between the current copy and the twin.
Detect and batch the writes ever performed.
Homeless
The diff is saved and managed at each process.
4/19/2019

6. Drawbacks of Homeless Protocol
Communication overhead
Fetch diff from each previous writer.
Redundant operations
Fetch and apply diffs at each faulting node.
Memory consumption of twin and diff
Requires a global garbage collection.
Not scalable!
4/19/2019

7. LRC - Home-based Protocol
Each shared object has a home to which all writes (diffs) are propagated and from which all copies are derived.
Demonstrated to be more scalable than the homeless protocol.
Less communication, lower diff overhead, less memory consumption.
Access asymmetry
The access overhead at the home node is more lightweight than that at non-home nodes.
4/19/2019

8. Outline Background Problem Statement Our Approach
Performance Evaluation
Conclusion
4/19/2019

9. Home Assignment Problem
A home-based multiple-writer protocol imple-menting LRC P0
P1 (Home of X)
(Home)
Home Migration
Lock
Write(X)
Fault-in
Create twin
Create
diff
Diff propagation
Access symmetry
→ Home assignment problem
→ Find the optimal home
Unlock
Apply diff
4/19/2019

10. Related Work
JiaJia
Barrier manager detects single-writer pattern between two barriers.
JUMP
The process requiring the page may become the new home.
Blind to access pattern.
Jackal’s Lazy Flushing
Home is fixed.
A process may become the exclusive owner.
Some homeless DSMs (ADSM, Adaptive TreadMarks) can switch between single-writer protocol and multiple-writer protocol.
4/19/2019

11. Home Migration A must in home-based protocols.
Adapts to memory access pattern.
Targets at the single-writer pattern.
Home migration makes little difference in the multiple-writer situation as long as the home node is one of the writers.
Negative effect
Home migration notices
We need a timely and precise home migration protocol.
4/19/2019

12. Outline Background Problem Statement Our Approach
Performance Evaluation
Conclusion
4/19/2019

13. Sensitivity of Home Migration Protocol
P1 (Home)
remote read (fault-in)
remote write (diff)
The less sensitive the protocol is
! Migrate home
! Migrate home
! Migrate home
4/19/2019

14. Robustness of Home Migration Protocol
remote read
Home
Home
Home
remote write
HM notice P0 P1
A home migration protocol should be
robust enough to prevent unnecessary
home migrations. P2 P3
4/19/2019

15. Sensitivity vs. Robustness
At a first glance
Sensitivity , robustness , and vice versa.
Can we achieve both sensitivity and robustness?
Single-writer pattern
Lasting single-writer pattern
Transient single-writer pattern
A good home migration protocol should demonstrate
sensitivity to the lasting single-writer pattern
robustness against the transient single-writer pattern
home migration is prohibited.
4/19/2019

16. Our Previous Work A distributed JVM with GOS support
A home-based, object-based, multiple-writer protocol
Runtime detection of distributed-shared object
Adaptive optimizations
Object home migration (fixed threshold)
Synchronized method migration
Connectivity-based object pushing
Weijian Fang, Cho-Li Wang and Francis C.M. Lau, ``On the Design of Global Object Space for Efficient Multi-threading Java Computing on Clusters’’, Parallel Computing Journal, Vol.29, pp , November-December 2003.
4/19/2019

17. An Adaptive Threshold HM Protocol
An adaptive per-object home migration threshold.
decreases with increased likelihood that a particular object exhibits a lasting single-writer pattern.
A lower threshold  more likely a lasting single-writer patter  quicker home migration.
A higher threshold  less likely a lasting single-writer patter (probably a transient single-writer pattern)  more conservative home migration
Continuously adjusted at runtime according to the feedback of previous home migration decisions for each object.
4/19/2019

18. Runtime Feedback Consecutive remote writes
Issued from the same remote node and not interleaved with the writes from either the home node or other remote nodes.
To detect single-writer pattern.
Exclusive home writes
There is no remote write between an exclusive home write and an earlier home write.
Positive feedback of previous home migrations.
Redirected object requests
Home migration notices.
Negative feedback of previous home migrations.
Home access coefficient
The overhead ratio of one eliminated pair of object fault-in and diff propagation to one home migration notice.
4/19/2019

19. Formalization
Ci : the number of consecutive remote writes since the (i - 1)th home migration.
Ti : the value of the adaptive home migration threshold since the (i - 1)th home migration.
Tinit : the initial threshold, which is equal to 1.
Ri : the number of redirected object requests since the (i - 1)th home migration.
Ei : the number of exclusive home writes since the (i - 1)th home migration.
: the home access coefficient.
 : the feedback coefficient. It is set to 1.
4/19/2019

20. Formalization (contd.)
Home migration is taken if
Ci = Ti
where
Ti = max{(Ti-1 + (Ri - Ei)), Tinit}
T0 = Tinit = 1
4/19/2019

21. Outline Background Problem Statement Our Approach
Performance Evaluation
Conclusion
4/19/2019

22. Effects of Home Migration
4/19/2019

23. Adaptive Threshold vs. Fixed Threshold
4/19/2019

24. Sensitivity and Robustness Analysis
4/19/2019

25. Contributions
This is the first home migration protocol taking home migration notice overhead into account.
Our protocol is both sensitive to the lasting single-writer pattern and robust against the transient single-writer pattern.
We formulate the protocol.
The protocol uses a per-object adaptive migration threshold, which decreases with increased likelihood that the object presents the lasting single-writer pattern.
The threshold is continuously adjusted according to the feedback of previous home migration decisions at runtime.
We implement and evaluate this protocol in a distributed JVM.
4/19/2019

26. Questions?
4/19/2019