A Novel Home Migration Protocol in Home-based DSM W.J. Fang C.L. Wang W.Z. Zhu F.C.M. Lau System Research Group Department of Computer Science The University of Hong Kong

Outline Background Problem Statement Our Approach Performance Evaluation Conclusion 4/19/2019 wjfang@csis.hku.hk

Distributed Java Virtual Machine (DJVM) import java.util.*; class worker extends Thread{ private long n; public worker(long N){ n=N; } public void run(){ long sum=0; for(long i=0; i<n; i++) sum+=i; System.out.println(“N=“+n+” Sum="+sum);} } public class test { static final int N=100; public static void main(String args[]){ worker [] w= new worker[N]; Random r = new Random(); for (int i=0; i<N; i++) w[i] = new worker(r.nextLong()); for (int i=0; i<N; i++) w[i].start(); try{ for (int i=0; i<N; i++) w[i].join();} catch (Exception e){}} A distributed Java Virtual Machine (DJVM) consists of a group of extended JVMs running on a distributed environment to support true parallel execution of a multithreaded Java application. A DJVM provides all the JVM services, that are compliant with the Java language specification. Presents a Single System Image (SSI) of clusters to Java programs – an illusion that the program is running on a single machine, yet more powerful. (Single System Image) Global Object Space DJVM Thread Exec Class JVM JVM JVM JVM Java thread HKU JESSICA2 project: http://www.srg.cs.hku.hk/ 4/19/2019 wjfang@csis.hku.hk

Global Object Space Virtualizes a single object heap spanning on the whole cluster. A DSM service in the distributed JVM. Java Memory Model: Resembles lazy release consistency. T1 T2 Before lock: flush cache Before unlock: write back Access fault 4/19/2019 wjfang@csis.hku.hk

LRC - Homeless Multiple-Writer Protocol False Sharing Two processes simultaneously write on the different parts of the same shared object. Ping-pong effect if using single-writer protocol. Twin and Diff Twin: the copy before writes. Diff: the difference between the current copy and the twin. Detect and batch the writes ever performed. Homeless The diff is saved and managed at each process. 4/19/2019 wjfang@csis.hku.hk

Drawbacks of Homeless Protocol Communication overhead Fetch diff from each previous writer. Redundant operations Fetch and apply diffs at each faulting node. Memory consumption of twin and diff Requires a global garbage collection. Not scalable! 4/19/2019 wjfang@csis.hku.hk

LRC - Home-based Protocol Each shared object has a home to which all writes (diffs) are propagated and from which all copies are derived. Demonstrated to be more scalable than the homeless protocol. Less communication, lower diff overhead, less memory consumption. Access asymmetry The access overhead at the home node is more lightweight than that at non-home nodes. 4/19/2019 wjfang@csis.hku.hk

Outline Background Problem Statement Our Approach Performance Evaluation Conclusion 4/19/2019 wjfang@csis.hku.hk

Home Assignment Problem A home-based multiple-writer protocol imple-menting LRC P0 P1 (Home of X) (Home) Home Migration Lock Write(X) Fault-in Create twin Create diff Diff propagation Access symmetry → Home assignment problem → Find the optimal home Unlock Apply diff 4/19/2019 wjfang@csis.hku.hk

Related Work JiaJia Barrier manager detects single-writer pattern between two barriers. JUMP The process requiring the page may become the new home. Blind to access pattern. Jackal’s Lazy Flushing Home is fixed. A process may become the exclusive owner. Some homeless DSMs (ADSM, Adaptive TreadMarks) can switch between single-writer protocol and multiple-writer protocol. 4/19/2019 wjfang@csis.hku.hk

Home Migration A must in home-based protocols. Adapts to memory access pattern. Targets at the single-writer pattern. Home migration makes little difference in the multiple-writer situation as long as the home node is one of the writers. Negative effect Home migration notices We need a timely and precise home migration protocol. 4/19/2019 wjfang@csis.hku.hk

Outline Background Problem Statement Our Approach Performance Evaluation Conclusion 4/19/2019 wjfang@csis.hku.hk

Sensitivity of Home Migration Protocol P1 (Home) remote read (fault-in) remote write (diff) The less sensitive the protocol is ! Migrate home ! Migrate home ! Migrate home 4/19/2019 wjfang@csis.hku.hk

Robustness of Home Migration Protocol remote read Home Home Home remote write HM notice P0 P1 A home migration protocol should be robust enough to prevent unnecessary home migrations. P2 P3 4/19/2019 wjfang@csis.hku.hk

Sensitivity vs. Robustness At a first glance Sensitivity , robustness , and vice versa. Can we achieve both sensitivity and robustness? Single-writer pattern Lasting single-writer pattern Transient single-writer pattern A good home migration protocol should demonstrate sensitivity to the lasting single-writer pattern robustness against the transient single-writer pattern home migration is prohibited. 4/19/2019 wjfang@csis.hku.hk

Our Previous Work A distributed JVM with GOS support A home-based, object-based, multiple-writer protocol Runtime detection of distributed-shared object Adaptive optimizations Object home migration (fixed threshold) Synchronized method migration Connectivity-based object pushing Weijian Fang, Cho-Li Wang and Francis C.M. Lau, ``On the Design of Global Object Space for Efficient Multi-threading Java Computing on Clusters’’, Parallel Computing Journal, Vol.29, pp. 1563-1587, November-December 2003. 4/19/2019 wjfang@csis.hku.hk

An Adaptive Threshold HM Protocol An adaptive per-object home migration threshold. decreases with increased likelihood that a particular object exhibits a lasting single-writer pattern. A lower threshold  more likely a lasting single-writer patter  quicker home migration. A higher threshold  less likely a lasting single-writer patter (probably a transient single-writer pattern)  more conservative home migration Continuously adjusted at runtime according to the feedback of previous home migration decisions for each object. 4/19/2019 wjfang@csis.hku.hk

Runtime Feedback Consecutive remote writes Issued from the same remote node and not interleaved with the writes from either the home node or other remote nodes. To detect single-writer pattern. Exclusive home writes There is no remote write between an exclusive home write and an earlier home write. Positive feedback of previous home migrations. Redirected object requests Home migration notices. Negative feedback of previous home migrations. Home access coefficient The overhead ratio of one eliminated pair of object fault-in and diff propagation to one home migration notice. 4/19/2019 wjfang@csis.hku.hk

Formalization Ci : the number of consecutive remote writes since the (i - 1)th home migration. Ti : the value of the adaptive home migration threshold since the (i - 1)th home migration. Tinit : the initial threshold, which is equal to 1. Ri : the number of redirected object requests since the (i - 1)th home migration. Ei : the number of exclusive home writes since the (i - 1)th home migration. : the home access coefficient.  : the feedback coefficient. It is set to 1. 4/19/2019 wjfang@csis.hku.hk

Formalization (contd.) Home migration is taken if Ci = Ti where Ti = max{(Ti-1 + (Ri - Ei)), Tinit} T0 = Tinit = 1 4/19/2019 wjfang@csis.hku.hk

Outline Background Problem Statement Our Approach Performance Evaluation Conclusion 4/19/2019 wjfang@csis.hku.hk

Effects of Home Migration 4/19/2019 wjfang@csis.hku.hk

Adaptive Threshold vs. Fixed Threshold 4/19/2019 wjfang@csis.hku.hk

Sensitivity and Robustness Analysis 4/19/2019 wjfang@csis.hku.hk

Contributions This is the first home migration protocol taking home migration notice overhead into account. Our protocol is both sensitive to the lasting single-writer pattern and robust against the transient single-writer pattern. We formulate the protocol. The protocol uses a per-object adaptive migration threshold, which decreases with increased likelihood that the object presents the lasting single-writer pattern. The threshold is continuously adjusted according to the feedback of previous home migration decisions at runtime. We implement and evaluate this protocol in a distributed JVM. 4/19/2019 wjfang@csis.hku.hk

Questions? 4/19/2019 wjfang@csis.hku.hk