Replication Aspects in Distributed Systems

Replication Aspects in Distributed Systems
By: Shantanu Sharma Advisor: Prof. Shlomi Dolev Department of Computer Science, Ben-Gurion University of the Negev, Israel.

Outline Introduction and An Overview of the Tasks Investigated
Self-Stabilizing End-to-End Communication Algorithm Mapping Schema Problems in MapReduce Meta-MapReduce Interval Join Computing Marginals of a Data Cube Privacy-Preserving Computation using MapReduce Future Directions

Introduction and An Overview of the Tasks Investigated
Replication Creates and maintains several copies of an identical data Provides availability, consistency, and reliability Tasks To design a self-stabilizing end-to-end algorithm In MapReduce the model design overlapping intervals and computing marginals of a data cube privacy-preserving MapReduce computations

Self-Stabilizing End-to-End Communication Algorithm Mapping Schema Problems in MapReduce Meta-MapReduce Interval Join Computing Marginals of a Data Cube Privacy-Preserving Computation using MapReduce Conclusion and Future Directions

Shlomi Dolev1, Ariel Hanemann1, and Elad M. Schiller2
14th International Symposium on Stabilization, Safety, and Security of Distributed Systems (SSS 2012) Self-Stabilizing End-to-End Communication in Bounded Capacity, Omitting, Duplicating and non-FIFO Dynamic Networks Joint work with Shlomi Dolev1, Ariel Hanemann1, and Elad M. Schiller2 1 Department of Computer Science, Ben-Gurion University of the Negev, Israel 2 Department of Computer Science and Engineering, Chalmers University of Technology, Sweden

Intromission of Errors in Message Transmission
Dynamic networks have a greater chance of having errors, like, omission, duplication, reordering, than conventional static networks Network is aware of a known capacity constant, that is an upper bound on number of packets in the network Omission of packets Reorder Duplication by the sender Duplication by the channel capacity of the channel capacity of the “channel” R R R R S S S S S R R S capacity packets

The Problem Design an algorithm for bounded capacity, non-FIFO, and dynamic networks that Guarantees exactly one copy of message delivery Tolerates omission, duplication of messages by the “channel” and the sender Delivers messages in the same order as they were sent Applicable to any starting configuration

The Algorithm S R 1,0,d1 1,1,d2 1,2,d3 2,2,d3 1 1,3,d4 2,3,d4 capacity
Packet_set 1,0,d1 1st message 2edmessage 3rd message plth message Messages with errors 1st message 2edmessage 3rd message plth message Correct messages 1,1,d2 ACK_set (2,2,d3) (1,2,d3) 1,2,d3 2,2,d3 Alternating Index AltIndexs= 1 LastDeliveredIndexr = 0 LastDeliveredIndexr = 1 1 (1,3,d4) (2,3,d4) 1,3,d4 2,3,d4 S R (1, capacity, dc) (1, capacity, dc) (1, n-1, dn-1) (1, 3, d4) (1, 2, d3) (1, 1, d2) (1, 0, d1) (1, capacity+1, dc+1) (1, 1, d2) Alternating Index Label Data capacity, 1 1 , 1 0, 1 1,capacity,dc (0, 2, d3) (1, 1, d2) (1, 0, d1) (2, 3, d4) (2, 2, d3) (1, 0, d1) capacity 1,capacity+1,dc+1 Capacity 1,n-1,dn-1 After the delivery of the correct messages, the receiver acknowledges the sender. Now, the receiver selects n distinct labels which have identical alternating index except LastDeliveredIndex. Transmission of acknowledgements from the receiver to the sender. Transmission of packets from the sender to the receiver.

The Algorithm at Sender
ml 1st message The sender fetches pl messages of ml length 2ed message 3rd message plth message

The Algorithm at Sender
n > ml lbl ml 1 2 3 ml n-1 n AltIndex ai ai ai ai ai ai 1st message Consider all the bit positions of each pl message 2ed message 3rd message Error Correcting Encoding plth message 1st bit 2ed bit 3rd bit mlth bit 1st Packet 3rd Packet The sender appends redundant bits to each pl message that result in message length to be n (n>ml) nth Pacet Error Correcting Bits 2ed Packet

The Algorithm at Receiver
The receiver collects n distinctly labeled packets that have alternating index different from the last delivered index capacity distinct labels may be from the channel n – capacity distinctly labeled packets must be from the sender 1 2 3 ml n-1 n ai ai ai ai ai ai

Assignment of Different-Sized Inputs in MapReduce
28th International Symposium on Distributed Computing (DISC 2014) and 2nd Algorithms and Systems for MapReduce and Beyond (BeyondMR 2015) Assignment of Different-Sized Inputs in MapReduce Joint work with Foto N. Afrati1, Shlomi Dolev2, Ephraim Korach2, and Jeffrey D. Ullman3 1 National Technical University of Athens, Greece 2 Ben-Gurion University of the Negev, Israel 3 Stanford University, USA

MapReduce MapReduce is a programming model used for parallel processing over large-scale data

MapReduce An Example – Word Count I like apple Mapper 1 is fruit I
Reducer for I (I, 2) (like, 2) (apple, 2) (is, 1) (fruit, 1) (banana, 1) 1 like 2 apple Reducer for like I like apple. Apple is fruit. Mapper 1 1 is 1 fruit Reducer for apple Reducer for is 1 I Reducer for fruit Mapper 2 I like banana. 1 like 1 banana Reducer for banana

Inputs and Outputs in Our Context
1 I Reducer for I (I, 2) (like, 2) (apple, 2) (is, 1) (fruit, 1) (banana, 1) 1 like 2 apple Reducer for like Inputs I like apple. Apple is fruit. Mapper 1 1 is 1 fruit Reducer for apple Reducer for is 1 I Reducer for fruit Mapper 2 I like banana. 1 like 1 banana Reducer for banana

We consider two special matching problems
Reducer Capacity Values, provided by each mapper, have some sizes (input size) Reduce capacity: an upper bound on the sum of the sizes of the values that are assigned to the reducer Example: reducer capacity to be the size of the main memory of the processors on which reducers run We consider two special matching problems

Mapping Schema Mapping schema is an assignment of the set of inputs to some given reducers, such that Respect the reducer capacity A reducer is assigned only inputs whose sum is less than or equal to the reducer capacity Assignment of inputs M1 (1GB) M2 (2GB) M3 (2GB) Reducer (4GB)

State-of-the-Art Unit input size
F. Afrati, A.D. Sarma, S. Salihoglu, and J.D. Ullman, “Upper and Lower Bounds on the Cost of a Map- Reduce Computation,” PVLDB, 2013. Unit input size

Problem Statement Notation ki: key Communication cost between the map and the reduce phases is a significant factor How we can reduce the communication cost? A lesser number of reducers, and hence, a smaller communication cost How to minimize the total number of reducers? Not an easy task All-with-All mapping schema problem X-with-Y mapping schema problem Reducer for k1 (1, 2) input1 Mapper for 1st input input1 Mapper for 1st input input1 k1 input1 k1 input1 k2 input2 input2 Reducer for k2 (1, 3) Mapper for 2nd input Reducer for k1 (1, 2, 3) Mapper for 2nd input input2 k1 input2 k1 input2 k3 input3 input3 Mapper for 3rd input Mapper for 3rd input input3 k2 Reducer for k3 (2, 3) input3 k1 input3 k3

All-to-All Mapping Schema Problem
A set of inputs is given Each pair of inputs corresponds to one output Inputs w1 = w2 = w3 = 0.20q, w4 = w5 = 0.19q, w6 = w7 = 0.18q One way Another way Group inputs such that size of a group is no more than q/2 .22q is misused w1, w2 w3, w4 w5, w6 w7 w1, w2 w3, w4 w3, w4 w5, w6 w1, w2, w3, w4, w7 w1, w2 w5, w6 w3, w4 w7 w1, w2, w5, w6, w7 w1, w2 w7 w5, w6 w7 w3, w4, w5, w6, w7 6 reducers and non-optimum communication cost 3 reducers and optimum communication cost

A2A Mapping Schema Problem
Polynomial time solution for one and two reducers NP-hard for z > 2 reducers Approximation Algorithms for A2A Mapping Schema Problem First-Fit Decreasing (FFD) or Best-Fit Decreasing (BFD) bin-packing algorithm Pseudo-polynomial bin-packing algorithm* 2-step Algorithms The selection of a prime number p *D. R. Karger and J. Scott. Efficient algorithms for fixed-precision instances of bin packing and euclidean tsp. In APPROX-RANDOM, pages 104–117, 2008.

Approximation Algorithms for A2A Mapping Schema Problem
Sx S4 Sx-1 All the input sizes are different Use a bin-packing algorithm to create x bins (S1, S2, …, Sx) of size at most q/2 Use x(x-1)/2 reducers to assign each bin with each other w1 w2 w3 wm S1 S2 Sx

X2Y Mapping Schema Problem
Two disjoint sets X and Y are given Each pairs of element xi, yj (where xi  X, yj  Y, i, j) of the sets X and Y corresponds to one output

w1=w2=0.25q, w3=w4=0.24q, w5=w6=0.23q, w7=w8=0.22q, w9=w10=0.21q, w11=w12=0.20q Inputs of set 𝑋 Inputs of set 𝑌 𝑤 1 ′ = 𝑤 2 ′ =0.25𝑞, 𝑤 3 ′ = 𝑤 4 ′ =0.24𝑞 One way Another way Group inputs such that size of a group is no more than q/2 12 reducers Make groups by taking three inputs from X 16 reducers w1, w2 w3, w4 w5, w6 w7, w8 w1, w2, w3 w9, w10 𝑤 1 ′ w11, w12 w1, w2, w3 𝑤 3 ′ w4, w5, w6 𝑤 1 ′ w4, w5, w6 𝑤 3 ′ 𝑤 1 ′ , 𝑤 2 ′ 𝑤 3 ′ , 𝑤 4 ′ w1, w2 𝑤 1 ′ , 𝑤 2 ′ w1, w2 𝑤 3 ′ , 𝑤 4 ′ w7, w8, w9 𝑤 1 ′ w7, w8, w9 𝑤 3 ′ w3, w4 𝑤 1 ′ , 𝑤 2 ′ w3, w4 𝑤 3 ′ , 𝑤 4 ′ w10, w11, w12 𝑤 1 ′ w10, w11, w12 𝑤 3 ′ w5, w6 𝑤 1 ′ , 𝑤 2 ′ w5, w6 𝑤 3 ′ , 𝑤 4 ′ w1, w2, w3 𝑤 2 ′ w1, w2, w3 𝑤 4 ′ w7, w8 𝑤 1 ′ , 𝑤 2 ′ w7, w8 𝑤 3 ′ , 𝑤 4 ′ w4, w5, w6 𝑤 2 ′ w4, w5, w6 𝑤 4 ′ w9, w10 𝑤 1 ′ , 𝑤 2 ′ w9, w10 𝑤 3 ′ , 𝑤 4 ′ w7, w8, w9 𝑤 2 ′ w7, w8, w9 𝑤 4 ′ w11, w12 𝑤 1 ′ , 𝑤 2 ′ w11, w12 𝑤 3 ′ , 𝑤 4 ′ w10, w11, w12 𝑤 2 ′ w10, w11, w12 𝑤 4 ′

Polynomial time solution for one reducer NP-hard for z > 1 reducers Bin-packing-based Approximation Algorithm for X2Y Mapping Schema Problem

17th International Symposium on Stabilization, Safety, and Security of Distributed Systems (SSS 2015) Meta-MapReduce A Technique for Reducing Communication in MapReduce Computations Joint work with Foto N. Afrati1, Shlomi Dolev2, and Jeffrey D. Ullman3 1 National Technical University of Athens, Greece 2 Ben-Gurion University of the Negev, Israel 3 Stanford University, USA

Communication Cost: Join of two relations
Organization A Map Phase Reduce Phase Organization B Final outputs Outputs

But then how to get answers??
Problem Statement Do we need to send the whole database to the cloud before performing join operations? NO But then how to get answers?? Work on metadata

Meta-MapReduce A new algorithmic approach for MapReduce algorithms that decreases the communication cost significantly Work on metadata, which varies according to problems and very small in size as compared to the original database Decreases the communication cost

Meta-MapReduce Master process Original input data
Step 1: MapReduce job assignment Master process Step 3: Read and Map tasks’ execution Step 4: Read and Reduce tasks’ execution Mapper for 1st split Step 2: Meta-data transmission split1 Split1 Split2 Splitm Input meta-data Reducer for k1 Output 1 Mapper for 2nd split Chunk1 Meta- data Reducer for k2 Output 2 split2 Original input data Reducer for kr Mapper for mth split splitm Step 4: Call Function: Data request and data transmission

Applications Geographically distributed MapReduce computations
k-nearest-neighbors problem Shortest part problem in a social graph Multiway join Amazon EMR

Bounds for Overlapping Interval Join on MapReduce
2nd Algorithms and Systems for MapReduce and Beyond (BeyondMR 2015) Bounds for Overlapping Interval Join on MapReduce Joint work with Foto N. Afrati1, Shlomi Dolev2, and Jeffrey D. Ullman3 1 National Technical University of Athens, Greece 2 Ben-Gurion University of the Negev, Israel 3 Stanford University, USA

An Interval and Overlapping Intervals
A pair [starting time , ending time] Example: My talk Overlapping intervals Two intervals, say interval i and interval j are called overlapping intervals if the intersection of both the interval is nonempty T e i = 12pm T s i = 11am 10am 10:35am Overlapping intervals Talk Coffee break 10:30am 11am

State-of-the-Art B. Chawda, H. Gupta, S. Negi, T.A. Faruquie, L.V. Subramaniam, and M.K. Mohania, “Processing Interval Joins On Map-Reduce,” EDBT, 2014. MapReduce-based 2-way and multiway interval join algorithms of overlapping intervals Not regarding the reducer capacity No analysis of a lower bound on replication of individual intervals No analysis of the replication rate of the algorithms offered therein

Our Contribution An algorithm for variable-length intervals that can start at any time Before this, we consider two simple cases of Unit-length and equally-spaced intervals and provide algorithm Variable-length and equally-spaced intervals and provide algorithm

Unit-Length and Equally Spaced Intervals
Relations X and Y of n intervals All intervals do not have beginning beyond k and before 0 Hence, spacing between starting points of two successive intervals = k n < 1 X Y n = 9 and k = 2.25, so spacing = 0.25

Variable-Length and Equally Spaced Intervals
Two types of intervals Big and small intervals Different length intervals

Computing Marginals using MapReduce
Joint work with Foto N. Afrati1, Jeffrey D. Ullman2, and Jonathan Ullman3 1 National Technical University of Athens, Greece 2 Stanford University, USA 3 Northeastern University, USA

Data Cube A user may easily solve a query such as the total number of users from New York and London accessing Web sites at 8am-10am. City New York London Tel Aviv 5M 2M 1M 10M 3M 2.5M 4M We can add more dimensions to this 3-dimensional data cube such as Day, Month, Year, and Site_name Time Users

Marginals The marginals include SUM over one dimension, two dimensions, and so on. Example Consider a 5-D cube [D1,D2,D3,D4,D5] Query sum the data values in all those tuples that have value D1=X, D3=Y, and D4=Z This is called 2nd-order marginals [X,*,Y,Z,*]

The Problem: Compute all the 2nd-order Marginals
Consider there are d = 10 inputs in each dimension Total inputs = 105 Total marginals = = (5 choose 3)*1000 One marginal contains 100 inputs So we need at least q = 100 and total reducers = What will happen when q > 100, say q =1000, i.e., q =103 How many reducers are required while minimizing the communication cost? A trivial solution Make 10 teams of 100 reducers

Covering Number How to compute all the second-order marginals while minimizing the communication cost Find minimum number of handles i.e., a set of handles = C(n,m,k) = C(5,3,2) = 4 i.e., the minimum number of sets of size m out of n elements such that every set of size k out of the same n elements is contained in one of the sets of size m C(n,m,k) = replication rate, if q = dm, d are inputs in each dimension

2nd-order Marginals by 3rd-order Handles
C(n,m,k) = C(5,3,2) Only 4 handles and 4 teams of 100 reducers in each Create n-2 handles such that each of these handles has the last two dimensions and one of the first n-2 dimensions Recursively create handles for the first n-2 dimensions C(n,m,k) <= (n-1)2/4 D1,D4,D5 D2,D4,D5 D3,D4,D5 D1,D2,D3 Step 1 Step 2

2nd-order Marginals by 3rd-order Handles and the General Case
Algorithms for covering all the 2nd–order marginals With larger size handles A special case for C(3i,3,2), i>0, that meets the lower bound on the replication, i.e., C(3i,3,2) <= n2/6 An algorithm for C(n,m,k) that works for all n and for all m > k Extending the ideas of isoperimetric inequalities for the hypercube* *This part is done by Prof. Jonathan Ullman.

Privacy-Preserving Computations using MapReduce
Joint work with Shlomi Dolev and Yin Li Ben-Gurion University of the Negev, Israel

Privacy in MapReduce Why is it required? Where is it required?
Users send data on the clouds Curious mappers and reducers can Store useful data Know the given job Where is it required? Banking, financial, retail, and healthcare

Make information secure data
Privacy in MapReduce What others do? Work on encrypted data Authentication & Compress + Encrypt data Encrypt data-at-rest Secure storage of data in HDFS Provide authentication before using Hadoop cluster They are making ‘computational secure’ data. But, for how long is it secured?? Make information secure data

Privacy in MapReduce Accumulating Automata*
Make shares of data (or input split) Send these shares to mappers Mappers do not know the computation and data Mappers have a defined accumulating automata Example: Search a pattern “LO” in the string “LOXLO” *S. Dolev, N. Giboa, X. Li, “Accumulating Automata and Cascaded Equations Automata for Communicationless Information Theoretically Secure Multi-Party Computation: Extended Abstract,” pages 21—29, 2015.

Privacy in MapReduce Example: Search a pattern “LO” in the string “LOXLO” L = {v3,v4} O = {v5,v10} X ={v1,v1} L = {v4,v5} O = {v15,v20} Mapper 1 𝑁 1 𝑀 𝑘+1 =𝑣0 𝑁 2 𝑀 𝑘+1 = 𝑁 1 𝑀 𝑘 . 𝑣1 𝑁 3 𝑚 𝑘+1 = 𝑁 3 𝑀 𝑘 +𝑁 2 𝑀 𝑘 . 𝑣2 N1 N2 N3 v140 L = {v5,v6} O = {v10,v19} X = {v2,v2} L = {v6,v7} O = {v20,v29} Mapper 2 N1 N2 N3 v698 Reducer LO, 2 L = {v7,v9} O ={v15,v28} X = {v3,v3} Mapper 3 N1 N2 N3 v1964 L = {v9,v12} O={v20,v37} X = {v4,v4} Mapper 4 N1 N2 N3 v4226

Other Operations Fetch tuples Equijoin Range query
Count the occurrence, Fetch address by dividing the relation, and then fetch tuples Equijoin Use two layers of clouds, where the first layer performs fetch operation and the second layer performs equijoin operation Range query By using 2’s complement Count the occurrence of number that lies in the range and then fetch those tuples

Outline Introduction and Overview of the Tasks Investigated

Conclusion Self-stabilizing end-to-end communication algorithm
Different-sized inputs and the reducer capacity Working on metadata (Meta-MapReduce) Bounds for the problem of the interval join Unit-length equally spaced and variable-length equally spaced Computing many marginals of a data cube at a reducer in one round Privacy-preserving computations on MapReduce Accumulating-Automata Count, search, fetch, equijoin, and range quires

Future Directions How to handle streaming of data in our scope?
How to perform dynamic programming? how to store temporary data where to store temporary data for how many iterations to retain temporary data what types of operations can be performed on temporary data. Rigorous securing and privacy in MapReduce Role of self-stabilizing in the future cellular network (real-time and life-critical system and cloud-RAN)

References Research papers Survey papers
Self-stabilizing end-to-end communication in (bounded capacity, omitting, duplicating and non-fifo) dynamic networks Assignment of different-sized inputs in MapReduce Meta-MapReduce: A technique for reducing communication in MapReduce computations Bounds for overlapping interval join on MapReduce Computing marginals using MapReduce Private Secret Shared MapReduce Survey papers Security and privacy aspects in MapReduce on clouds: A survey. A survey on geographically distributed data processing using MapReduce A survey on 5G: The next generation of mobile communication

I Am Heartily Thankful To
Prof. Shlomi Dolev (my PhD advisor), Prof. Jeffrey Ullman, and Prof. Foto Afrati helped a lot during the journey of doctoral degree Prof. Ehud Gudes Prof. Ephraim Korach Prof. Elad M. Schiller Prof. Awadhesh Kumar Singh (my Master’ advisor) Prof. Jonathan Ullman Nisha Panwar Ariel Hanemann Philip Derbeko, Yin Li, and Ximing Li

Presentation is available at http://www.cs.bgu.ac.il/~sharmas
Shantanu Sharma Department of Computer Science, Ben-Gurion University of the Negev, Israel

Replication Aspects in Distributed Systems

Similar presentations

Presentation on theme: "Replication Aspects in Distributed Systems"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Replication Aspects in Distributed Systems

Similar presentations

Presentation on theme: "Replication Aspects in Distributed Systems"— Presentation transcript:

Similar presentations

About project

Feedback