Tolerating Latency in Replicated State Machines through Client Speculation April 22, 2009 Benjamin Wester1, James Cowling2, Edmund B. Nightingale3, Peter.

Slides:



Advertisements
Similar presentations
Remus: High Availability via Asynchronous Virtual Machine Replication
Advertisements

CS 542: Topics in Distributed Systems Diganta Goswami.
CSE 486/586, Spring 2013 CSE 486/586 Distributed Systems Byzantine Fault Tolerance Steve Ko Computer Sciences and Engineering University at Buffalo.
The SMART Way to Migrate Replicated Stateful Services Jacob R. Lorch, Atul Adya, Bill Bolosky, Ronnie Chaiken, John Douceur, Jon Howell Microsoft Research.
1 Attested Append-Only Memory: Making Adversaries Stick to their Word Byung-Gon Chun (ICSI) October 15, 2007 Joint work with Petros Maniatis (Intel Research,
Yee Jiun Song Cornell University. CS5410 Fall 2008.
CS 582 / CMPE 481 Distributed Systems Fault Tolerance.
EEC 693/793 Special Topics in Electrical Engineering Secure and Dependable Computing Lecture 16 Wenbing Zhao Department of Electrical and Computer Engineering.
EEC 693/793 Special Topics in Electrical Engineering Secure and Dependable Computing Lecture 15 Wenbing Zhao Department of Electrical and Computer Engineering.
2/23/2009CS50901 Implementing Fault-Tolerant Services Using the State Machine Approach: A Tutorial Fred B. Schneider Presenter: Aly Farahat.
Attested Append-only Memory: Making Adversaries Stick to their Word Distributed Storage Systems CS presented by: Hussam Abu-Libdeh.
EEC 693/793 Special Topics in Electrical Engineering Secure and Dependable Computing Lecture 16 Wenbing Zhao Department of Electrical and Computer Engineering.
EEC 693/793 Special Topics in Electrical Engineering Secure and Dependable Computing Lecture 15 Wenbing Zhao Department of Electrical and Computer Engineering.
CS 603 Communication and Distributed Systems April 15, 2002.
BASE: Using Abstraction to Improve Fault Tolerance Rodrigo Rodrigues, Miguel Castro, and Barbara Liskov MIT Laboratory for Computer Science and Microsoft.
Clock-RSM: Low-Latency Inter-Datacenter State Machine Replication Using Loosely Synchronized Physical Clocks Jiaqing Du, Daniele Sciascia, Sameh Elnikety.
EEC 688 Secure and Dependable Computing Lecture 16 Wenbing Zhao Department of Electrical and Computer Engineering Cleveland State University
Byzantine fault tolerance
Slingshot: Deploying Stateful Services in Wireless Hotspots Ya-Yunn Su Jason Flinn University of Michigan.
Byzantine Fault Tolerance CS 425: Distributed Systems Fall Material drived from slides by I. Gupta and N.Vaidya.
Operating System Support for Application-Specific Speculation Benjamin Wester Peter Chen and Jason Flinn University of Michigan.
HQ Replication: Efficient Quorum Agreement for Reliable Distributed Systems James Cowling 1, Daniel Myers 1, Barbara Liskov 1 Rodrigo Rodrigues 2, Liuba.
Practical Byzantine Fault Tolerance
Byzantine fault-tolerance COMP 413 Fall Overview Models –Synchronous vs. asynchronous systems –Byzantine failure model Secure storage with self-certifying.
From Viewstamped Replication to BFT Barbara Liskov MIT CSAIL November 2007.
1 ZYZZYVA: SPECULATIVE BYZANTINE FAULT TOLERANCE R.Kotla, L. Alvisi, M. Dahlin, A. Clement and E. Wong U. T. Austin Best Paper Award at SOSP 2007.
SPECULATIVE EXECUTION IN A DISTRIBUTED FILE SYSTEM E. B. Nightingale P. M. Chen J. Flint University of Michigan.
Byzantine fault tolerance
Practical Byzantine Fault Tolerance and Proactive Recovery
Speculative Execution in a Distributed File System Ed Nightingale Peter Chen Jason Flinn University of Michigan.
Byzantine Fault Tolerance CS 425: Distributed Systems Fall 2012 Lecture 26 November 29, 2012 Presented By: Imranul Hoque 1.
EEC 688/788 Secure and Dependable Computing Lecture 15 Wenbing Zhao Department of Electrical and Computer Engineering Cleveland State University
Ben Vandiver, Hari Balakrishnan, Barbara Liskov, and Sam Madden CSAIL, MIT Tolerating Byzantine Faults in Database Systems using Commit Barrier Scheduling.
Byzantine Fault Tolerance
Systems Research Barbara Liskov October Replication Goal: provide reliability and availability by storing information at several nodes.
Replication predicates for dependent-failure algorithms Flavio Junqueira and Keith Marzullo University of California, San Diego Euro-Par Conference, Lisbon,
Fault Tolerance
Speculative Execution in a Distributed File System Ed Nightingale Peter Chen Jason Flinn University of Michigan Best Paper at SOSP 2005 Modified for CS739.
BChain: High-Throughput BFT Protocols
The consensus problem in distributed systems
Alternative system models
Secure Causal Atomic Broadcast, Revisited
CPS 512 midterm exam #1, 10/7/2016 Your name please: ___________________ NetID:___________ /60 /40 /10.
Principles of Computer Security
View Change Protocols and Reconfiguration
Byzantine Fault Tolerance
EECS 498 Introduction to Distributed Systems Fall 2017
Providing Secure Storage on the Internet
Outline Announcements Fault Tolerance.
Distributed Systems, Consensus and Replicated State Machines
Principles of Computer Security
Jacob Gardner & Chuan Guo
Replication Improves reliability Improves availability
EEC 688/788 Secure and Dependable Computing
EEC 688/788 Secure and Dependable Computing
Fault-tolerance techniques RSM, Paxos
From Viewstamped Replication to BFT
IS 651: Distributed Systems Fault Tolerance
EEC 688/788 Secure and Dependable Computing
Lecture 21: Replication Control
EECS 498 Introduction to Distributed Systems Fall 2017
Speculative execution and storage
EEC 688/788 Secure and Dependable Computing
Building Dependable Distributed Systems, Copyright Wenbing Zhao
View Change Protocols and Reconfiguration
The SMART Way to Migrate Replicated Stateful Services
Lecture 21: Replication Control
Sisi Duan Assistant Professor Information Systems
Sisi Duan Assistant Professor Information Systems
Presentation transcript:

Tolerating Latency in Replicated State Machines through Client Speculation April 22, 2009 Benjamin Wester1, James Cowling2, Edmund B. Nightingale3, Peter M. Chen1, Jason Flinn1, Barbara Liskov2 University of Michigan1, MIT CSAIL2, Microsoft Research3

Simple Service Configuration 1 x=1 x=0 ++x NSDI'09 Benjamin Wester University of Michigan CSE

Replicated State Machines (RSM) 2 x=1 x=2 ++x 2 x=1 ++x 2 Agree on request All non-faulty replies are identical 2 NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE RSMs have high latency x=2 2 2 2 Need many replies Agreement Geographic Distribution NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE Hide the Latency Use speculative execution inside RSM Speculate before consensus is reached Without faults, any reply predicts consensus value Let client continue after receiving one reply NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE Overview Introduction Improving RSMs with speculation Application to PBFT Performance Conclusion NSDI'09 Benjamin Wester University of Michigan CSE

Speculative Execution in RSM Take Checkpoint Predict: 1 Speculate! Blocked Commit Rollback x=1 x=2 x=1 Continue processing while waiting NSDI'09 Benjamin Wester University of Michigan CSE

Critical path: first reply 1 Completion latency less relevant First reply latency sets critical path Speed Accuracy Other desirable properties Throughput Stability under contention Smaller number of replicas NSDI'09 Benjamin Wester University of Michigan CSE

Requests while speculative while !check_lottery(): submit_tps() buy_corvette() Predict win? = yes yes buy win? What do we do with this? Hold request Bad performance Distributed commit/rollback State tracking complex NSDI'09 Benjamin Wester University of Michigan CSE

Resolve speculations on the replicas while !check_lottery(): submit_tps() buy_corvette() Predict win? = yes win? = yes buy = keys yes keys yes buy if win?=yes: buy buy win? Explicitly encode dependencies as predicates No special request handling needed Replicas need to log past replies Local decision at replicas matches client NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE Overview Introduction Improving RSMs with speculation Application to PBFT Performance Conclusion NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE Practical BFT -CS [Castro and Liskov 1999] client primary f=1 NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE Additional Details Tentative execution PBFT/PBFT-CS complete in 4 phases Read-only optimization Accurate answer from backup replica Failure threshold Bound worst-case failure Correctness NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE Overview Introduction Improving RSMs with speculation Application to PBFT Performance Conclusion NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE Benchmarks Shared counter Simple checkpoint No computation NFS: Apache httpd build Complex checkpoint Significant computation NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE Topology Primary-local Primary-remote Uniform 2.5 or 15 ms Primary NSDI'09 Benjamin Wester University of Michigan CSE

Base case: no replication Primary-local Primary-remote Uniform 2.5 or 15 ms NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE Shared Counter Primary-local topology NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE Shared Counter Primary-local topology [Kotla et al. 07] NSDI'09 Benjamin Wester University of Michigan CSE

Shared Counter Uniform & Primary-remote topology NSDI'09 Benjamin Wester University of Michigan CSE

Shared Counter Uniform & Primary-remote topology NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE NFS: Apache build Primary-local topology NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE NFS: Apache build Uniform topology NSDI'09 Benjamin Wester University of Michigan CSE

NFS: Apache build Primary-remote topology NSDI'09 Benjamin Wester University of Michigan CSE

Benjamin Wester University of Michigan CSE NFS: With Failure Primary-local topology (1% fail) NSDI'09 Benjamin Wester University of Michigan CSE

Throughput (Shared Counter) LAN topology NSDI'09 Benjamin Wester University of Michigan CSE

Conclusion Makes WAN deployments more practical Integrate client speculation within RSMs Predicated requests: performance without complexity Clients less sensitive to latency between replicas 5x speedup over non-speculative protocol Makes WAN deployments more practical NSDI'09 Benjamin Wester University of Michigan CSE