1 Load Shedding Algorithm Evaluation Step –When to shed load? Load Shedding Road Map (LSRM) –Where to shed load? –How much load to shed?

2 Load Evaluation Load Coefficients (L) –the number of processor cycles required to push a single tuple through the network to the outputs c1s1c1s1 c2s2c2s2 cnsncnsn … IO L = n operators c i = cost s i = selectivity

3 Load Evaluation Load Coefficient L 1 = 10 + (0.5 * 10) + (0.5 * 0.8 * 5) + (0.5 * 10) = 22 L 2 = 10 + (0.8 * 5) = 14 1 c 1 = 10 s 1 = c 2 = 10 s 2 = c n = 5 s n = 1.0 I O1O1 4 c 2 = 10 s 2 = 0.9 O2O2 L 1 = 22 L 2 = 14 L 3 = 5 L 4 = 10 L(I) = 22

4 Stream Load (S) –load created by the current stream rates Load Evaluation S = m input streams L i = load coefficient r i = input rate

5 Load Evaluation Stream Load S = 22 * 10 = c 1 = 10 s 1 = c 2 = 10 s 2 = c n = 5 s n = 1.0 I O1O1 4 c 2 = 10 s 2 = 0.9 O2O2 L 1 = 22 L 2 = 14 L 3 = 5 L 4 = 10 L(I) = 22 r = 10

6 Queue Load (Q) –load due to any queues that may have built up since the last load evaluation step MELT_RATE = how fast to shrink the queues (queue length reduction per unit time) Load Evaluation Q = MELT_RATE * L i * q i L i = load coefficient q i = queue length

7 Load Evaluation Queue Load MELT_RATE = 0.1 Q = 0.1 * 5 * 100 = 50 1 c 1 = 10 s 1 = c 2 = 10 s 2 = c n = 5 s n = 1.0 I O1O1 4 c 2 = 10 s 2 = 0.9 O2O2 L 1 = 22 L 2 = 14 L 3 = 5 L 4 = 10 L(I) = 22 r = 10 q = 100

8 Load Evaluation Total Load Total Load (T) = S + Q T = = c 1 = 10 s 1 = c 2 = 10 s 2 = c n = 5 s n = 1.0 I O1O1 4 c 2 = 10 s 2 = 0.9 O2O2 L 1 = 22 L 2 = 14 L 3 = 5 L 4 = 10 L(I) = 22 r = 10 q = 100

9 The system is overloaded when Load Evaluation T > H * C headroom factor processing capacity

10 Load Shedding Algorithm Evaluation Step –When to drop? Load Shedding Road Map (LSRM) –How much to drop? –Where to drop?

11 Load Shedding Road Map (LSRM) <Cycle Savings Coefficients (CSC) Drop Insertion Plan (DIP) Percent Delivery Cursors (PDC)> set of drops that will be inserted how many cycles will be saved where the system will be running when the DIP is adopted … max savings … (0,0,0,…,0) CSC DIP PDC ENTRY n……ENTRY 1 cursor more load sheddingless load shedding

12 LSRM Construction set Drop Locations compute & sort Loss/Gain ratios how much to drop? take the least ratio insert Drop create LSRM entry how much to drop? take the least ratio insert Filter create LSRM entry determine predicate Drop-Based LSFilter-Based LS

13 Drop Locations Single Query set Drop Locations compute & sort Loss/Gain ratios Drop-Based LSFilter-Based LS 1 c 1 = 10 s 1 = c 2 = 10 s 2 = c n = 5 s n = 1.0 I O L 1 = 17 L 2 = 14 L 3 = 5 A BCD

14 Drop Locations Single Query set Drop Locations compute & sort Loss/Gain ratios Drop-Based LSFilter-Based LS 1 c 1 = 10 s 1 = c 2 = 10 s 2 = c n = 5 s n = 1.0 I O L 1 = 17 L 2 = 14 L 3 = 5 A

15 Drop Locations Shared Query 1 c 1 = 10 s 1 = c 2 = 10 s 2 = c n = 5 s n = 1.0 I O1O1 4 c 2 = 10 s 2 = 0.9 O2O2 L 1 = 22 L 2 = 14 L 3 = 5 L 4 = 10 A B C DE F set Drop Locations compute & sort Loss/Gain ratios Drop-Based LSFilter-Based LS

16 Drop Locations Shared Query 1 c 1 = 10 s 1 = c 2 = 10 s 2 = c n = 5 s n = 1.0 I O1O1 4 c 2 = 10 s 2 = 0.9 O2O2 L 1 = 22 L 2 = 14 L 3 = 5 L 4 = 10 A B C set Drop Locations compute & sort Loss/Gain ratios Drop-Based LSFilter-Based LS

17 Loss/Gain Ratio Loss Loss – utility loss as tuples are dropped – determined using loss-tolerance QoS graph set Drop Locations compute & sort Loss/Gain ratios Drop-Based LSFilter-Based LS % tuples utility Loss for first piece of graph = (1 – 0.7) / 50 = 0.006

18 Loss/Gain Ratio Gain Gain – processor cycles gained R = input rate into drop operator L = load coefficient x = drop percentage D = cost of drop operator STEP_SIZE = increments for x to find G(x) Gain G(x) = set Drop Locations compute & sort Loss/Gain ratios Drop-Based LSFilter-Based LS

19 Drop-Based Load Shedding how much to drop? Take the least Loss/Gain ratio Determine the drop percentage p how much to drop? take the least ratio insert Drop create LSRM entry Drop-Based LS

20 Drop-Based Load Shedding where to drop? how much to drop? take the least ratio insert Drop create LSRM entry Drop-Based LS 1 c 1 = 10 s 1 = c 2 = 10 s 2 = c n = 5 s n = 1.0 I O L 1 = 17 L 2 = 14 L 3 = 5 A drop If there are other drops in the network, modify their drop percentages.

21 Drop-Based Load Shedding make LSRM entry All drop operators with the modified percentages form the DIP Compute CSC Advance QoS cursors and store in PDC LSRM Entry <Cycle Savings Coefficients (CSC) Drop Insertion Plan (DIP) Percent Delivery Cursors (PDC)> how much to drop? take the least ratio insert Drop create LSRM entry Drop-Based LS

22 Filter-Based Load Shedding how much to drop? predicate for filter Start dropping from the interval with the lowest utility. Keep a sorted list of intervals according to their utility and relative frequency. Find out how much to drop and what intervals are needed to. Determine the predicate for filter. how much to drop? take the least ratio insert Filter create LSRM entry determine predicate Filter-Based LS

23 Filter-Based Load Shedding place the filter how much to drop? take the least ratio insert Filter create LSRM entry determine predicate Filter-Based LS 1 c 1 = 10 s 1 = c 2 = 10 s 2 = c n = 5 s n = 1.0 I O L 1 = 17 L 2 = 14 L 3 = 5 A filter If there are other filters in the network, modify their selectivities.