Sisi Duan Assistant Professor Information Systems

Slides:



Advertisements
Similar presentations
Impossibility of Distributed Consensus with One Faulty Process
Advertisements

Distributed Systems Overview Ali Ghodsi
Distributed Computing 8. Impossibility of consensus Shmuel Zaks ©
Consensus Hao Li.
1 Asynchronous Broadcast Protocols in Distributed System Oct. 10, 2002 JaeHyrk Park ICU.
 Idit Keidar, Principles of Reliable Distributed Systems, Technion EE, Spring Principles of Reliable Distributed Systems Lecture 7: Failure Detectors.
1 Fault-Tolerant Consensus. 2 Failures in Distributed Systems Link failure: A link fails and remains inactive; the network may get partitioned Crash:
Idit Keidar, Principles of Reliable Distributed Systems, Technion EE, Spring Principles of Reliable Distributed Systems Lecture 5: Synchronous Uniform.
Distributed systems Module 2 -Distributed algorithms Teaching unit 1 – Basic techniques Ernesto Damiani University of Bozen Lesson 4 – Consensus and reliable.
CPSC 668Set 11: Asynchronous Consensus1 CPSC 668 Distributed Algorithms and Systems Fall 2009 Prof. Jennifer Welch.
 Idit Keidar, Principles of Reliable Distributed Systems, Technion EE, Spring Principles of Reliable Distributed Systems Lecture 12: Impossibility.
Distributed Algorithms: Agreement Protocols. Problems of Agreement l A set of processes need to agree on a value (decision), after one or more processes.
Composition Model and its code. bound:=bound+1.
Securing Every Bit: Authenticated Broadcast in Wireless Networks Dan Alistarh, Seth Gilbert, Rachid Guerraoui, Zarko Milosevic, and Calvin Newport.
Failure Resilience in the Peer-to-Peer-System OceanStore Speaker: Corinna Richter.
Low-Overhead Byzantine Fault-Tolerant Storage James Hendricks, Gregory R. Ganger Carnegie Mellon University Michael K. Reiter University of North Carolina.
Distributed Algorithms – 2g1513 Lecture 9 – by Ali Ghodsi Fault-Tolerance in Distributed Systems.
Consensus and Its Impossibility in Asynchronous Systems.
Practical Byzantine Fault Tolerance
Byzantine fault-tolerance COMP 413 Fall Overview Models –Synchronous vs. asynchronous systems –Byzantine failure model Secure storage with self-certifying.
CS4231 Parallel and Distributed Algorithms AY 2006/2007 Semester 2 Lecture 8 Instructor: Haifeng YU.
1 ZYZZYVA: SPECULATIVE BYZANTINE FAULT TOLERANCE R.Kotla, L. Alvisi, M. Dahlin, A. Clement and E. Wong U. T. Austin Best Paper Award at SOSP 2007.
Chap 15. Agreement. Problem Processes need to agree on a single bit No link failures A process can fail by crashing (no malicious behavior) Messages take.
UNIVERSITY of WISCONSIN-MADISON Computer Sciences Department
Systems Research Barbara Liskov October Replication Goal: provide reliability and availability by storing information at several nodes.
Alternating Bit Protocol S R ABP is a link layer protocol. Works on FIFO channels only. Guarantees reliable message delivery with a 1-bit sequence number.
Fault tolerance and related issues in distributed computing Shmuel Zaks GSSI - Feb
1 Fault-Tolerant Consensus. 2 Communication Model Complete graph Synchronous, network.
Distributed Agreement. Agreement Problems High-level goal: Processes in a distributed system reach agreement on a value Numerous problems can be cast.
Randomized Algorithms for Distributed Agreement Problems Peter Robinson.
Unreliable Failure Detectors for Reliable Distributed Systems Tushar Deepak Chandra Sam Toueg Presentation for EECS454 Lawrence Leinweber.
Intrusion Tolerant Consensus in Wireless Ad hoc Networks Henrique Moniz, Nuno Neves, Miguel Correia LASIGE Dep. Informática da Faculdade de Ciências Universidade.
BChain: High-Throughput BFT Protocols
The consensus problem in distributed systems
When Is Agreement Possible
Distributed Systems – Paxos
Secure Causal Atomic Broadcast, Revisited
CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS
Principles of Computer Security
Byzantine Fault Tolerance
Outline Distributed Mutual Exclusion Distributed Deadlock Detection
Providing Secure Storage on the Internet
Implementing Consistency -- Paxos
Alternating Bit Protocol
Distributed Consensus
Agreement Protocols CS60002: Distributed Systems
Distributed Systems, Consensus and Replicated State Machines
Distributed Consensus
Principles of Computer Security
Fault-tolerant Consensus in Directed Networks Lewis Tseng Boston College Oct. 13, 2017 (joint work with Nitin H. Vaidya)
Presented By: Md Amjad Hossain
Active replication for fault tolerance
PERSPECTIVES ON THE CAP THEOREM
EEC 688/788 Secure and Dependable Computing
From Viewstamped Replication to BFT
Consensus, FLP, and Paxos
EEC 688/788 Secure and Dependable Computing
CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS
FLP Impossibility of Consensus
CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS
EEC 688/788 Secure and Dependable Computing
EEC 688/788 Secure and Dependable Computing
EEC 688/788 Secure and Dependable Computing
EEC 688/788 Secure and Dependable Computing
Cryptography Lecture 24.
Implementing Consistency -- Paxos
Sisi Duan Assistant Professor Information Systems
Blockchains Lecture 6.
Announcement Sign up google sheet for in class lectures
Sisi Duan Assistant Professor Information Systems
Presentation transcript:

Sisi Duan Assistant Professor Information Systems sduan@umbc.edu IS 698/800-01: Advanced Distributed Systems Asynchronous Byzantine Fault Tolerance Sisi Duan Assistant Professor Information Systems sduan@umbc.edu

Overview Asynchrony vs. partial synchrony Randomized BFT HoneyBadgerBFT BEAT Asynchronous Binary Agreement

Asynchrony vs. Partial Synchrony In an asynchronous system, there are no bounds on message transmission delays or process execution rates Yields protocols that are most robust to timing and denial-of-service attacks Deterministic total ordering is impossible [Fischer et al., JACM 1985], and so protocols can be only probabilistically live In a partially synchronous system, there exist such bounds but they are unknown to participants in the system Where most of the action has been in the last 20 years These almost always rely on a view-change subprotocol to cope with failures

General BFT SMR vs. (Just) BFT Storage In both general BFT SMR and BFT Storage, state is represented as a key-value store “BFT Storage” supports only Read and Write operations Sometimes called “atomic registers” in the distributed computing literature “General BFT SMR” supports arbitrary operations (transactions) on the key-value store

Another Look at FLP Consensus: getting a number of processors to agree a value In asynchronous system A faulty node cannot be distinguished from a slow node Correctness of a distributed system Safety No two correct nodes will agree on inconsistent values Liveness Correct nodes eventually agree

Put FLP to consensus Cannot guarantee termination p0 p1 p2 p3

Overview of Randomized BFT Key to termination Random shared coin Same for all the replicas If enough votes are collected Agree on the value Otherwise, agree on the random coin

HoneyBadgerBFT [Miller et al., CCS 2016] Implements total order using Asynchronous Common Subset (ACS) [Ben-Or et al., PODC 1994; Cachin et al., CRYPTO 2001] Implements ACS, in turn, using Reliable broadcast (RBC) and asynchronous binary Byzantine agreement (ABA)

Asynchronous Common Subset (ACS) The goal Every node proposes some transactions Agree on the superset of all the proposed transactions

Asynchronous Common Subset (ACS) RBC: Reliable broadcast Every node proposes some transactions Randomly from the transaction pool ABA Agreement on the proposed transactions by each node N parallel ABAs

From HoneyBadger to BEAT A family of protocols for asynchronous networks Built as a series of adaptations to HoneyBadgerBFT [Miller et al., CCS 2016] Protocol Applicability Features BEAT0 General BFT SMR Efficient threshold encryption & coin flipping Efficient erasure-coding support BEAT1 Reduced latency via different subprotocol combination BEAT2 Reduced latency via client-side threshold encryption BEAT3 BFT Storage Storage/bandwidth savings through new protocol combo BEAT4 Read bandwidth savings through new fingerprinted cross checksums

Reliable Broadcast [Bracha, Information and Computation 1987] A designated replica (the broadcaster) broadcasts a message to all replicas, with the following guarantees: If any correct replica delivers a broadcast message B, then all correct replicas deliver B. If the broadcaster is correct, then every correct replica delivers the broadcaster’s message. Bracha’s protocol tolerates f faulty replicas out of n = 3f+1.

Reliable Broadcast [Bracha, Information and Computation 1987] “Initiate”

Reliable Broadcast [Bracha, Information and Computation 1987] “Initiate” “Echo”

Reliable Broadcast [Bracha, Information and Computation 1987] “Initiate” “Echo”

Reliable Broadcast [Bracha, Information and Computation 1987] Communication complexity is O(n2|B|) B p0 p1 p2 p3 “Initiate” “Echo” “Ready”

How to make it cheaper? Multiple methods One method: Reduce the network bandwidth Erasure coding

AVID [Cachin and Tessaro, SRDS 2005] Leverages erasure coding and cross checksums to reduce communication complexity to O(n|B|), assuming hashes are free d1 B d2 Cross checksum d3 d4 Erasure coded fragments Hashes

AVID [Cachin and Tessaro, SRDS 2005] Broadcaster sends fragment di and cross-checksum to replica i, and replica i echoes these to all replicas For example, at replica 2 … d1 From broadcaster: d2 B d2 Echo from replica 3: d3 d3 Echo from replica 4: d4 d4

AVID-FP [Hendricks et al., PODC 2007] Additionally leverages homomorphic fingerprints to reduce communication complexity to O(|B|), assuming fingerprints are free d1 B d2 d3 d4 Erasure coded fragments Fingerprints

AVID-FP [Hendricks et al., PODC 2007] Broadcaster sends fragment di and fingerprinted cross-checksum to replica i, and replica i echoes only the fingerprinted cross checksum For example, at replica 2 … Replica 2 echoes only the fingerprinted cross-checksum From broadcaster: d2

BEAT All erasure-coded systems for arbitrary failures use conventional maximum distance separable (MDS) codes (e.g., Reed-Solomon) Modern erasure-coding schemes can further reduce the read bandwidth, however BEAT4 incorporates a new adaptation of AVID-FP to pyramid codes [Huang et al., ACM TOS 2013] Improves read bandwidth by ~50%, in exchange for ~10% additional storage

Asynchronous Binary Agreement Secret sharing Based on threshold cryptography A secret is shared among n parties such that the cooperation of at least t + 1 parties is needed to recover s

Asynchronous Binary Agreement Binary Byzantine Agreement Propose(b): start the protocol decide(b): terminate it, for a bit b Correctness Validity: If all honest servers propose v, then some honest server eventually decides v. Agreement: If some honest server decides v and a distinct honest server decides v′, then v=v′. Termination: Every honest server eventually decides

Consider crash failures only for now Cachin, Christian, Rachid Guerraoui, and Luís Rodrigues. Introduction to reliable and secure distributed programming. Springer Science & Business Media, 2011. Phase 1: propose v If receiving v from the majority, ok Otherwise, proposal = NULL Phase 2, propose proposal If receiving matching v from the majority, decision = v Otherwise, if it has a value w, proposal = w Otherwise, proposal = c (common coin) Continue phase 1 if no decision is made

Consider crash failures only for now The difficulty of majority voting without common coin Phase 1: propose v If receiving v from the majority, ok Otherwise, proposal = NULL Phase 2, propose proposal If receiving matching v from the majority, decision = v Otherwise, if it has a value w, proposal = w (??) Otherwise, proposal = c (common coin) Continue phase 1 if no decision is made In the first phase, p gets 1 and 0 (from q). Decide NULL. Q receives 1 from p, decide NULL. R receives 0 from q, select 0. In the second phase, p receives null from itself and q, select 1. q receives NULL from itself and p, select 0.

Asynchronous Binary Agreement BFT Version Asynchronous Binary Agreement Random coin

Asynchronous Binary Agreement Lemma 1: If all honest servers start some round r with vote v0, then all honest servers will also terminate round r with vote v0 Lemma 2: If two distinct honest servers start some round r with different votes, then with probability at least 1/2, all honest servers will terminate round r with the same vote. Lemma 3: Binary agreement will terminate. Simple reason: BA reaches agreement with probability at least 1/2 in every round, the expected number of rounds is 2, and the expected number of messages sent is O(n3)

BEAT Implementation Implemented seven protocols Five BEAT instances (BEAT0-4) HoneyBadgerBFT (HB), HB + Bracha (for comparison) Jerasure 2.0 for erasure coding GF-complete for fingerprinted cross-checksum and GF operations P-256 curve (128-bit security) for labeled threshold encryption and threshold PRF

Evaluation Amazon EC2 Up to 92 nodes from ten different regions in different continents Evaluate the protocols under different network sizes (number of replicas) and contention levels (batch sizes) Vary the batch size where nodes propose 1 to 20,000 transactions at a time “LAN tests”: nodes are selected from the same Amazon EC2 region “WAN tests”: nodes are uniformly selected from different regions

Latency in LAN Settings

Throughput in LAN Setting (f = 1)

Throughput in WAN Setting (f = 1)

Scalability: BEAT3 vs. HoneyBadgerBFT Solid lines: BEAT3 Dashed lines: HB

Lessons BEAT explores various points in the design space of asynchronous total-ordering protocols As in the partially synchronous case, there appears to be no one-size- fits-all protocol Asynchronous protocols like HB and BEAT that avoid view-change subprotocols are arguably simpler than partially synchronous ones

References G.Bracha, An asynchronous[(n−1)/3]-resilient consensus protocol,Proc. 3rdACM Symposium on Principles of Distributed Computing (PODC), 1984, pp. 154–162. S. Toueg, Randomized Byzantine agreements, Proc. 3rd ACM Symposium on Princi- ples of Distributed Computing (PODC), 1984, pp. 163–178. Lecture notes: https://disco.ethz.ch/courses/ss04/distcomp/lecture/chapter9.pdf Miller, Andrew, et al. "The honey badger of BFT protocols." Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2016. Duan, Sisi, Michael K. Reiter, and Haibin Zhang. "BEAT: Asynchronous BFT made practical." Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2018.