Providing Resiliency to Load Variations in Distributed Stream Processing Ying Xing, Jeong-Hyon Hwang, Ugur Cetintemel, Stan Zdonik Brown University

Jeong-Hyon Hwang 2 Stream Processing Monitoring Apps Financial Data Streams Surveillance Network Monitoring Click Stream Analysis Traffic Monitoring Sensor Network

Jeong-Hyon Hwang 3 Distributed Stream Processing

Jeong-Hyon Hwang 4 Roadmap Problem Statement Linear Load Model Feasible Set The Algorithm Extensions Lower Bound of Input Rates Non-linear Load Model Network Bandwidth / Communication Overhead Experimental Results Related Work Conclusions

Jeong-Hyon Hwang 5 Problem Statement Goal Find an operator distribution with the largest feasible set size r1r1 r2r2 r1r1 r1r1 r2r2 r1r1 r2r2 Input Rate Space Operator Distribution feasible infeasible Feasible Set

Jeong-Hyon Hwang 6 Linear Load Model r j - input rate of input j (tuples/sec) c k - processing cost of operator o k (CPU cycles/tuple) l(o k ) - the processing load of operator o k (CPU cycles/sec) s k - selectivity of operator o k ( [# output tuples] / [# of input tuples] ) o1o1 o1o1 o3o3 o3o3 o2o2 o2o2 o4o4 o4o4

Jeong-Hyon Hwang 7 Example Feasible Sets o1o1 o1o1 o3o3 o3o3 o2o2 o2o2 o4o4 o4o4 r1r1 r2r2 0 o1o1 o1o1 o4o4 o4o4 o2o2 o2o2 o3o3 o3o3 r1r1 r2r2 0 o1o1 o1o1 o3o3 o3o3 o2o2 o2o2 o4o4 o4o4 r1r1 r2r2 0

Jeong-Hyon Hwang 8 “Ideal” Feasible Set Theorem 1. Feasible Set is maximized when load coefficients of each input are perfectly balanced over all nodes (relative to their capacities) o1o1 o1o1 o3o3 o3o3 o2o2 o2o2 o4o4 o4o4 r1r1 r2r2 0 r1r1 r2r2 0

Jeong-Hyon Hwang 9 Resilient Operator Distribution Algorithm 1. Compute the Ideal Feasible Set 2. Sort Operators based on Load Coefficients 3. For each operator, determine the destination server r2r2 0 r1r1 Ideal Feasible Set

Jeong-Hyon Hwang 10 Result: R.O.D. vs Load Balancing 10 nodes 5 input streams

Jeong-Hyon Hwang 11 Result: Latency of a Network Monitoring Query

Jeong-Hyon Hwang 12 Extension: Network Bandwidth & Comm. Overhead Network Bandwidth Comm. Overhead

Jeong-Hyon Hwang 13 Extension: Nonlinear Load Model Add an artificial variable … r1r1 … o1o1 o1o1 ouou ouou o u+1 omom omom … r1r1 o1o1 o1o1 ouou ouou r2r2 … omom omom r2r2

Jeong-Hyon Hwang 14 Extension: Lower Bound of Input Rates Use the lower bound instead of the origin 0 r1r1 r2r2 0 r1r1 r2r2

Jeong-Hyon Hwang 15 Related Work Traditional Distributed Systems - Load balancing and load sharing [Shivaratri92] [Diekmann97] - Parallel query processing [DeWitt92] - Graph partitioning [Walshaw97] [Schloegel00] Stream Processing Systems - Load management Flux [Shah03] – data partitioning based parallel continuous query processing Medusa [Balazinska04] – federated distributed stream processing

Jeong-Hyon Hwang 16 Conclusion Distributed Stream Processing Resilient Operator Distribution - Maximize feasible set size Performance - Much better than conventional load distribution algorithms

Backup Slides

Computation Complexity Computation time is determined by n – number of nodes m –number of operators d –number of system input streams k – number of samples in load time series Static operator distribution Dynamic operator distribution

Jeong-Hyon Hwang 19 Heuristics Heuristic #1 Choose the case where feasibility boundaries are close on each axis Heuristic #2 Choose the case where all the feasibility boundaries are far from the orgin. r1r1 r2r2 0 r1r1 r2r2 0 r1r1 r2r2 0 r1r1 r2r2 0

Resilient vs. Optimal 2 nodes 4 input streams

Varying Bandwidth Constraints Resilient vs. Connected-Load-Balancing

Varying Data Communication CPU Overhead Resilient vs. Connected-Load-Balancing