Replication Management using the State-Machine Approach Fred B. Schneider Summary and Discussion : Hee Jung Kim and Ying Zhang October 27, 2005.

Slides:



Advertisements
Similar presentations
1 Process groups and message ordering If processes belong to groups, certain algorithms can be used that depend on group properties membership create (
Advertisements

CS 542: Topics in Distributed Systems Diganta Goswami.
Byzantine Generals. Outline r Byzantine generals problem.
Agreement: Byzantine Generals UNIVERSITY of WISCONSIN-MADISON Computer Sciences Department CS 739 Distributed Systems Andrea C. Arpaci-Dusseau Paper: “The.
The Byzantine Generals Problem Boon Thau Loo CS294-4.
Consistency and Replication (3). Topics Consistency protocols.
Time and Clock Primary standard = rotation of earth De facto primary standard = atomic clock (1 atomic second = 9,192,631,770 orbital transitions of Cesium.
Byzantine Generals Problem: Solution using signed messages.
Distributed Systems Fall 2010 Replication Fall 20105DV0203 Outline Group communication Fault-tolerant services –Passive and active replication Highly.
CS 582 / CMPE 481 Distributed Systems Fault Tolerance.
CS 582 / CMPE 481 Distributed Systems
EEC 688/788 Secure and Dependable Computing Lecture 12 Wenbing Zhao Department of Electrical and Computer Engineering Cleveland State University
2/23/2009CS50901 Implementing Fault-Tolerant Services Using the State Machine Approach: A Tutorial Fred B. Schneider Presenter: Aly Farahat.
1 Principles of Reliable Distributed Systems Lecture 5: Failure Models, Fault-Tolerant Broadcasts and State-Machine Replication Spring 2005 Dr. Idit Keidar.
© Chinese University, CSE Dept. Distributed Systems / Distributed Systems Topic 9: Time, Coordination and Replication Dr. Michael R. Lyu Computer.
CPSC 668Set 15: Broadcast1 CPSC 668 Distributed Algorithms and Systems Fall 2006 Prof. Jennifer Welch.
Distributed Systems Fall 2009 Replication Fall 20095DV0203 Outline Group communication Fault-tolerant services –Passive and active replication Highly.
Distributed Systems 2006 Group Membership * *With material adapted from Ken Birman.
Lecture 12 Synchronization. EECE 411: Design of Distributed Software Applications Summary so far … A distributed system is: a collection of independent.
State Machines CS 614 Thursday, Feb 21, 2002 Bill McCloskey.
Time, Clocks, and the Ordering of Events in a Distributed System Leslie Lamport (1978) Presented by: Yoav Kantor.
Fault Tolerance via the State Machine Replication Approach Favian Contreras.
Distributed Algorithms – 2g1513 Lecture 9 – by Ali Ghodsi Fault-Tolerance in Distributed Systems.
Copyright © George Coulouris, Jean Dollimore, Tim Kindberg This material is made available for private study and for direct.
CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS Fall 2011 Prof. Jennifer Welch CSCE 668 Set 15: Broadcast 1.
Reliable Communication in the Presence of Failures Based on the paper by: Kenneth Birman and Thomas A. Joseph Cesar Talledo COEN 317 Fall 05.
CSE 486/586, Spring 2013 CSE 486/586 Distributed Systems Replication with View Synchronous Group Communication Steve Ko Computer Sciences and Engineering.
DISTRIBUTED ALGORITHMS By Nancy.A.Lynch Chapter 18 LOGICAL TIME By Sudha Elavarti.
Replicated State Machines ITV Model-based Analysis and Design of Embedded Software Techniques and methods for Critical Software Anders P. Ravn Aalborg.
Practical Byzantine Fault Tolerance
Byzantine fault-tolerance COMP 413 Fall Overview Models –Synchronous vs. asynchronous systems –Byzantine failure model Secure storage with self-certifying.
DISTRIBUTED SYSTEMS II REPLICATION Prof Philippas Tsigas Distributed Computing and Systems Research Group.
IM NTU Distributed Information Systems 2004 Replication Management -- 1 Replication Management Yih-Kuen Tsay Dept. of Information Management National Taiwan.
Agenda Fail Stop Processors –Problem Definition –Implementation with reliable stable storage –Implementation without reliable stable storage Failure Detection.
Communication & Synchronization Why do processes communicate in DS? –To exchange messages –To synchronize processes Why do processes synchronize in DS?
Replication (1). Topics r Why Replication? r System Model r Consistency Models – How do we reason about the consistency of the “global state”? m Data-centric.
Copyright © George Coulouris, Jean Dollimore, Tim Kindberg This material is made available for private study and for direct.
Time, Clocks, and the Ordering of Events in a Distributed System Leslie Lamport Massachusetts Computer Associates,Inc. Presented by Xiaofeng Xiao.
EEC 688/788 Secure and Dependable Computing Lecture 10 Wenbing Zhao Department of Electrical and Computer Engineering Cleveland State University
Fault Tolerant Services
CSE 60641: Operating Systems Implementing Fault-Tolerant Services Using the State Machine Approach: a tutorial Fred B. Schneider, ACM Computing Surveys.
Consensus and leader election Landon Cox February 6, 2015.
D u k e S y s t e m s Asynchronous Replicated State Machines (Causal Multicast and All That) Jeff Chase Duke University.
Chap 15. Agreement. Problem Processes need to agree on a single bit No link failures A process can fail by crashing (no malicious behavior) Messages take.
UNIVERSITY of WISCONSIN-MADISON Computer Sciences Department
Building Dependable Distributed Systems, Copyright Wenbing Zhao
Replication and Group Communication. Management of Replicated Data FE Requests and replies C Replica C Service Clients Front ends managers RM FE RM Instructor’s.
Hwajung Lee.  Improves reliability  Improves availability ( What good is a reliable system if it is not available?)  Replication must be transparent.
Revisiting failure detectors Some of you asked questions about implementing consensus using S - how does it differ from reaching consensus using P. Here.
Replication Improves reliability Improves availability ( What good is a reliable system if it is not available?) Replication must be transparent and create.
Hwajung Lee. Primary standard = rotation of earth De facto primary standard = atomic clock (1 atomic second = 9,192,631,770 orbital transitions of Cesium.
Fail-Stop Processors UNIVERSITY of WISCONSIN-MADISON Computer Sciences Department CS 739 Distributed Systems Andrea C. Arpaci-Dusseau One paper: Byzantine.
1 AGREEMENT PROTOCOLS. 2 Introduction Processes/Sites in distributed systems often compete as well as cooperate to achieve a common goal. Mutual Trust/agreement.
Chapter 8 Fault Tolerance. Outline Introductions –Concepts –Failure models –Redundancy Process resilience –Groups and failure masking –Distributed agreement.
Replication Chapter Katherine Dawicki. Motivations Performance enhancement Increased availability Fault Tolerance.
EEC 688/788 Secure and Dependable Computing Lecture 10 Wenbing Zhao Department of Electrical and Computer Engineering Cleveland State University
Replication Hari Shreedharan. Papers! ● Implementing Fault-Tolerant Services using the State Machine Approach (Dec, 1990) Fred B. Schneider Cornell University.
Coordination and Agreement
Distributed Systems – Paxos
COT 5611 Operating Systems Design Principles Spring 2012
CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS
Replication Improves reliability Improves availability
Active replication for fault tolerance
EEC 688/788 Secure and Dependable Computing
UNIVERSITY of WISCONSIN-MADISON Computer Sciences Department
Fault-Tolerant State Machine Replication
Fault-Tolerant State Machine Replication
EEC 688/788 Secure and Dependable Computing
EEC 688/788 Secure and Dependable Computing
COT 5611 Operating Systems Design Principles Spring 2014
Presentation transcript:

Replication Management using the State-Machine Approach Fred B. Schneider Summary and Discussion : Hee Jung Kim and Ying Zhang October 27, 2005

4 Introduction 4 State Machines 4 Fault Tolerance 4 Fault-tolerant State Machines 4 Tolerating Faulty Output Devices 4 Tolerating Faulty Clients 4 Using Time to Make Request 4 Reconfiguration

4 Why Replication ? 4 Two kinds of replication are.. 4 State machine Approach is.. 4 What can be discussed in each sections Introduction

4 A general method for implementing a fault- tolerant service by replicating servers and coordinating client interactions with server replicas. State-Machine Approach

4 State machine consist of - State Variables - Commands. 4 Command might be implemented by - Sharing data amongst procedures, - Queuing requests - Using interrupt handlers. State Machines

4 Requests from clients processed in causal order. – O1: Requests issued by a single client processed by sm in the order they are issued – O2: r1 could have caused r2 => r1 processed by sm before r2 Assumption !

4 “ Outputs of a state machine are completely determined by the sequence of requests it processes, independent of time or any other activity of a system” Semantic Characterization

monitor: process do true -> val := sensor; ; delay D od end monitor Is this a state machine ? pc: state-machine var q:real; adjust: command(sensor-val: real) q := F(q, sensor-val); send q to actuator end adjust end pc YES !! NO !!

4 Byzantine failures: “arbitrary and malicious” 4 Failstop failures: “other components [can] detect that a failure has occurred” Fault Tolerance

“A system consisting of a set of distinct components is t fault-tolerant if it satisfies its specification provided that no more than t of those components become faulty during some interval of interest.” T Fault-Tolerance

4 Replicate State Machines and run on separate processors. 4 Each replica – Starts in the same initial state – Executes same requests in the same order 4 Assuming independent failure – Combine outputs of the replicas of this ensemble. Fault-tolerant SM

4 Replica Coordination All replicas receive and process the same sequence of requests. – Agreement : Each Non-Fault replica receives every request. – Order : Each Non-Fault replica processes the requests in the same relative order. Fault-tolerant SM

4 Any protocol that allows a designated processor called the transmitter so that – IC1: All non-faulty processors agree on the same value. – IC2: If the transmitter is non-faulty, then all non-faulty processors use its value as the one on which they agree. Agreement

Order requirement can be satisfied by – Assigning unique ids to requests. – Processing the requests according to a total ordering on the unique ids. Order and Stability

Order Implementation “A replica next processes the stable request with smallest unique ids.” 4 Using Logical Clocks. 4 Synchronized Real-Time Clocks. 4 Using Replica-Generated Identifiers.

Using Logical Clocks 4 A logical clock is a mapping T from events to the integers. 4 LCl: Tp is incremented after each event at P. 4 LC2: Upon receipt of a message -with timestamp ts, process p resets Tp,: Tp := max(Tp, ts) + 1.

Using Logical Clocks 4 Assumption to property of communication channels. – FIFO channels between processors – Failure Detection Assumption (for fail- stop processors) : A processor p detects that a fail-stop processor q has failed only after p has received the last message sent to p by q.

Logical Clocks Stability Test 4 Every client periodically makes some- possibly null-request to the state machine. 4 Request stable at smi if a request with larger timestamp has been received from every client running on a non-faulty processor.

Synchronized Real-time Clocks  Tp(e) : the real-time clock at processor p when event e occurs. Unique id : Tp(e) appended by fixed bit string that uniquely identifies p. - O1 satisfied if only one request in between successive clock ticks - O2 satisfied if degree on synchronization is better than the minimum message delivery time.

Synchronized Real-time Clocks (cont’d) 4 Real-time Clock Stability Test I r is stable at smi executed at p if the local clock at p reads ts and uid(r) < ts– td 4 Real Clock Stability Test II r is stable at smi if a request with larger uid has been received from every client.

Using Replica-Generated Ids. 4 Unique ids assigned by the replicas Two phase protocol – Replicas propose candidate unique ids – One candidate is selected 4 Elaboration of the protocol – Seen : smi has seen r once it has received r and proposed a candidate unique id for it. – Accepted: smi has accepted r once it knows the final choice of uid(r).

Using Replica-Generated Ids. 4 Constraints on the proposed ids(cuid(smi,r)) – UID1: cuid(smi,r) < = uid(r) – UID2: if r’ SEEN at smi after r has been accepted then uid(r) < cuid(smi,r’) 4 Replica-Generated Id Stability Test: r that has been accepted by smi is stable provided there is no request r’ that has i) Been seen by smi ii) Not been accepted by smi iii) cuid(smi,r’) < = uid(r)

Using Replica-Generated Ids. 4 Replica-generated Unique Identifiers : smi maintains – SEENi : largest cuid( sm i,r) so far assigned by sm i – ACCEPT i : largest uid(r) so far assigned by sm i on receipt of r – cuid(smi,r) = max( ) + 1+ i – Disseminates cuid( sm i,r) to other replicas, awaits receipt of a candidate uid from every non-faulty replica. – uid(r) = maxj(cuid( sm i,r))

4 Outputs used outside system : Use replicated voters and output devices. 4 Outputs used inside system : the client need not gather a majority of responses to its request to the state machine. It can use the single response produced locally. Tolerating Faulty Output Devices

4 Replicate the client - However, requires changes to state machines that handle requests from that client. 4 Defensive programming - Sometimes, a client cannot be made fault-tolerant by using replication. - Careful design of state machine can limit the effects of requests from faulty clients. Tolerating Faulty Clients

4 Assume that - All clients and state machine replicas have clocks synchronized to within r, and - Election starts at time strt and known to all clients and state machine replicas. 4 Transmitting a default vote - If client has not made a request by time strt + r, then a request with that client’s default vote has been made. Using Time to Make Request

 “ An ensemble of state machine replicas can tolerate more than t faults if it is possible to remove state machine replicas running on faulty processors from the ensemble and add replicas running on repaired processors. ” Reconfiguration

4 Combining Condition: P(t) - F(t) > X for all 0 <=t where X : -. P(t)/2 (Byzantine failure) -. 0 (fail-stop failure) P(t) = total number of processors at time t F(t) = faulty number of processors at time t Reconfiguration

Unbounded total number of fault possible if.. Fl: Byzantine failures, removed faulty replica from the ensemble before the Combining Condition is violated by subsequent processor failures. F2: Replicas running on repaired processors are added to the ensemble before the Combining Condition is violated by subsequent processor failures.

Configuration The configuration of the system is defined as: C: The clients S: The state-machine replicas O: The output devices To change system configuration.. - the value of C,S,O must be available - whenever C,S,O added, state must be updated

Managing Configuration A non -faulty configurator satisfies.. C1: Only a faulty element is removed from the configuration. C2: Only a non-faulty element is added to the configuration.

Integration with Failstop Processors and Logical Clocks If e is a client or output device, then sm i sends the state variables to before sending any output with ids > r join. If e is a state-machine replica, sm new, then sm i : 1. sends state variables and copies of any pending requests to sm new, 2. sends sm new subsequent request r received from c such that uid(r) < uid(r c ), where r c is the first request that sm new received directly from c after being restarted.

Integration with Failstop Processors and Realtime Clocks If e is a client or output device, then sm i sends the state variables to before sending any output with ids > r join. If e is a state-machine replica, sm new, then sm i : 1. sends state variables and copies of any pending requests to sm new, 2. sends to sm new every request received during the next interval of duration. Simplified !!

Stability Revised When requests made by a client can be received from two sources-the client and via a relay. The stability test must be changed.. Stability Test During Restart : r received directly from c by a restarting sm new is stable only after the last request from c relayed by another processor has been received by sm new

4State Machines approach is.. 4Coping with failures (Byzantine, Failstop).. -. Fault-tolerant State Machines -. Tolerating Faulty Output Devices -. Tolerating Faulty Clients 4Optimization : -. Using time to request 4Dynamic reconfiguration -. Managing the configuration -. Integrating a repaired object Summary

Thank you !!! Any question ???