SQUARE Scalable Quorum-based Atomic Memory with Local Reconfiguration Vincent Gramoli, Emmanuelle Anceaume, Antonino Virgillito

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Context and Motivations Distributed systems become  Large-scale  Dynamic  Unpredictable Challenges in Distributed Shared Memory  Atomic Consistency  Load Support

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Distributed Shared Memory (DSM) Atomic Consistency  Object Composition: we focus on a single object  Read operations return the last value written Replicated Object  Replica is a node maintaining the value of the object  Memory is the set of replicas Read/Write Operations [ABD95]  Any client can read and modify (write) the object  To do so, it contacts quorums (a.k.a. mutually intersecting sets) of replicas

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Existing DSM solutions Lack of Independence Non-terminating operation may block others undefinitely Lack of Scalability Lack of Adaptiveness Memory Underloaded Memory (unused resource) Overloaded Memory (bursts of load) Memory Clients keep track of all memory replicas (replacing a replica is complex)

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito System Model Object replicated on failure-prone nodes  The replicas r 1, …, r k share a 2-dim coordinate space r1r1 r2r2 r3r3 r4r4 r5r5 r6r6 r7r7 r8r8 … … r k-1 rkrk

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito System Model Unreliable communication through neighborhood  Each replica r i can communicate only with its nearest neighbors riri

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito System Model Topology takeover mechanism (CAN [RFH+01] )  Upon node failure/departure the space sharing is modified accordingly If a node r i fails, a takeover node r j replaces it riri rjrj

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito System Model Topology takeover mechanism (CAN [RFH+01] )  Upon node failure/departure the space sharing is modified accordingly If a node r i fails, a takeover node r j replaces it rjrj

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Introducing our Dynamic Quorums Dual-type Dynamic Quorums  Vertical Quorum: All replicas responsible of an abscissa x  Horizontal Quorum: All replicas responsible of an ordinate y Intersection for Atomicity requirement  Values are propagated (consulted) at a vertical (horizontal) quorum  Thus, all consultations obtain the lastly propagated value x y For any horizontal quorum H and any vertical quorum V: H  V ≠ Ø

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito SQUARE features Atomicity and Independence  Atomic operations are independent from each other Local Knowledge  Reactive Quorum Access: wrapping around the torus Fast Adaptive Read Operations  Single phase operations: accessing a single horizontal quorum is sufficient Memory Adaptiveness  If overloaded (global approximation), then expand  If underloaded (local observation), then shrink

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Operation Execution Basic Read Operation: 1)Get up-to-date value, 2) Propagate this value on a vertical Quorum. Basic Write Operation: 1)Get up-to-date value, 2) Propagate the value to write (and a higher version number) twice on the same vertical quorum Fast Adaptive Read Op: 1)Get up-to-date value once on a single horizontal quorum.

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Adjustment of the overlay size SQUARE thwarts if the requested replica is overloaded: Other replicas on its diagonal are contacted in turn until a non- overloaded one is found SQUARE expands if all contacted replicas are overloaded: A node outside the memory is added, and the object value is replicated at this node. SQUARE shrinks if a replica gets underloaded: The replica simply leaves the memory after neighbors notification.

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Simulation Results Self-Adaptiveness

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Simulation Results Load-Balancing

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Conclusion Atomic Consistency is guaranteed  Using dynamic quorum intersection,  Each failed/leaving participant is replaced to ensure quorum availability. Adaptiveness makes the algorithm tunable  Minimizing operation latency as much as possible,  Maximizing capability to support bursts of load. Perspective on operation speed up  Kleinberg’s model to route in polylog(q) hops

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Some References [CGG+05] Reconfigurable distributed storage for dynamic networks. G. Chockler, S. Gilbert, V. Gramoli, P. M. Musial, and A. A. Shvartsman. In Proc. of 9th Int’l Conf. on Principles of Distributed Systems (OPODIS’05), [AGGV05] P2P Architecture for Self-*Atomic Memory E. Anceaume, M. Gradinariu, V. Gramoli, A. Virgillito In Proc of the 8th Intl Symposium on Parallel Architectures, Algorithms,and Networks (I- SPAN’05) 214–219, [RFH+01] A Scalable Content Adressable Network S. Ratnasamy, P. Francis, M. Handley, R. Karp, S. Shenker In Proc. of the ACM SIGCOMM, 161–172, [ABD95] Sharing Memory Robustly in Message Passing Systems H. Attiya, A. Bar-Noy, D. Dolev In Journal of the ACM, 42(1):124–142, 1995.

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Simulation Results Operation Latency Request rate Read latency Write latency Max. memory size Max. hor quorum size Max. vert. quorum size 1/ / / / /

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Simulation Results Fault-tolerance

ACM SAC’07 March 14th V. Gramoli, E. Anceaume, A. Virgillito Simulation Results Scalability