1 A Deterministic Algorithm for Summarizing Asynchronous Streams over a Sliding Window Costas Busch Rensselaer Polytechnic Institute Srikanta Tirthapura Iowa State University

2 Introduction Algorithm Analysis Outline of Talk

3 Time Data stream: For simplicity assume unit valued elements

4 Current time Most recent time window of duration Compute the sum of elements with time stamps in time window Goal: Data stream:

5 Example I: All packets on a network link, maintain the number of different ip sources in the last one hour Example II: Large database, continuously maintain averages and frequency moments

6 Synchronous stream t i : In ascending order Asynchronous stream t i : No order guaranteed Data stream:

7 Why Asynchronous Data Streams? Synchronous streamAsynchronous stream Synchronous Asynchronous Merge w/o control Network delay & multi-path routing

8 Processing Requirements: One pass processing Small workspace: poly-logarithmic in the size of data Fast processing time per element Approximate answers are ok

9 Our results: A deterministic data aggregation algorithm Time: Space: Relative Error:

10 Previous Work: [Datar, Gionis, Indyk, Motwani. SIAM Journal on Computing, 2002] Deterministic, Synchronous [Tirthapura, Xu, Busch, PODC, 2006] Randomized, Asynchronous Merging buckets Random sampling

11 Introduction Algorithm Analysis Outline of Talk

12 Current time Time Data stream: For simplicity assume unit valued elements

13 Current time Most recent time window of duration Data stream: Compute the sum of elements with time stamps in time window Goal:

14 Divide time into periods of duration

15 The sliding window may span at most two time periods sliding window

16 sliding window Sum can be written as two sub-sums In two time periods

17 sliding window Data structure that maintains an estimate of In left time period

18 Without loss of Generality, Consider data structure in time period

19 Data structure consists of various levels is an upper bound of the sum in a period

20 Counts up to elements Time period Bucket at Level Consider level

21 Increase counter value Stream:

22 Increase counter value Stream:

23 Stream: Increase counter value

24 Increase counter value Stream:

25 Stream: Counter threshold of reached Split bucket

26 Stream: New buckets have threshold also

27 Stream: Increase appropriate bucket

28 Stream: Increase appropriate bucket

29 Stream: Increase appropriate bucket

30 Stream: Split bucket

31 Stream:

32 Stream: Increase appropriate bucket

33 Stream: Split bucket

34 Stream:

35 Splitting Tree

36 Leaf buckets of duration 1 are not split any further Max depth =

37 The initial bucket may be split into many buckets Leaf buckets

38 Due to space limitations we only keep the last buckets Leaf buckets

39 Suppose we want to find the sum of elements in time period

40 of splitting threshold Consider various levels

41 First level with a leaf bucket that intersects timeline

42 Estimate of S: Consider buckets on right of timeline

43 First level with a leaf bucket On right timeline OR

44 Introduction Algorithm Analysis Outline of Talk

45 Suppose that we use level in order to compute the estimate

46 Stream: A data element is counted in the appropriate bucket Consider splitting threshold level

47 Stream: We can assume that the element is placed in the respective bucket

48 Stream: We can assume that when bucket splits the element is placed in an arbitrary child bucket

49 Stream: If: GOOD! Element counted in correct bucket

50 Stream: If: BAD! Element counted in wrong bucket

51 Consider Leaf Buckets If GOOD!

52 Consider Leaf Buckets If BAD! Element counted in wrong bucket

53 :elements of left part counted on right Consider Leaf Buckets :elements of right part counted on left

54 elements of left part counted on right Must have been initially inserted in one of these buckets

55 Since tree depth

56 Since tree depth Similarly, we can prove Therefore:

57 It can be proven Since

58 It can be proven Since Combined with We obtain relative error :