1. Indexing Time Series Based on Slides by C. Faloutsos (CMU) and D. Gunopulos (UCR)

2. Outline Find Similar/interesting objects in a database Similarity Retrieval Multimedia

3. Time Series Databases A time series is a sequence of real numbers, representing the measurements of a real variable at equal time intervals Stock price movements Volume of sales over time Daily temperature readings ECG data all NYSE stocks A time series database is a large collection of time series

4. Time Series Problems (from a databases perspective) The Similarity Problem X = x 1, x 2, …, x n and Y = y 1, y 2, …, y n Define and compute Sim(X, Y) E.g. do stocks X and Y have similar movements? Retrieve efficiently similar time series (Similarity Queries)

5. Types of queries whole match vs sub-pattern match range query vs nearest neighbors all-pairs query

6. Examples Find companies with similar stock prices over a time interval Find products with similar sell cycles Cluster users with similar credit card utilization Find similar subsequences in DNA sequences Find scenes in video streams

7. day $price 1365 day $price 1365 day $price 1365 distance function: by expert (eg, Euclidean distance)

8. Problems Define the similarity (or distance) function Find an efficient algorithm to retrieve similar time series form a database (Faster than sequential scan) The Similarity function depends on the Application

9. Euclidean Similarity Measure View each sequence as a point in n-dimensional Euclidean space (n = length of each sequence) Define (dis-)similarity between sequences X and Y as p=1 Manhattan distance p=2 Euclidean distance

10. Easy to compute: O(n) Allows scalable solutions to other problems, such as indexing clustering etc... Advantages

11. Does not allow for different baselines Stock X fluctuates at $100, stock Y at $30 Does not allow for different scales Stock X fluctuates between $95 and $105, stock Y between $20 and $40 Disadvantages

12. [Goldin and Kanellakis, 1995] Normalize the mean and variance for each sequence Let µ(X) and  (X) be the mean and variance of sequence X Replace sequence X by sequence X’, where x’ i = (x i - µ (X) )/  (X) Normalization

13. Similarity definition still too rigid Does not allow for noise or short-term fluctuations Does not allow for phase shifts in time Does not allow for acceleration-deceleration along the time dimension etc …. But..

14. Example

15. A general similarity framework involving a transformation rules language [Jagadish, Mendelzon, Milo] Each rule has an associated cost

16. Examples of Transformation Rules Collapse adjacent segments into one segment new slope = weighted average of previous slopes new length = sum of previous lengths [l 1, s 1 ] [l 2, s 2 ] [l 1 +l 2, (l 1 s 1 +l 2 s 2 )/(l 1 +l 2 )]

17. Combinations of Moving Averages, Scales, and Shifts [Rafiei and Mendelzon, 1998] Moving averages are a well-known technique for smoothening time sequences Example of a 3-day moving average x’ i = (x i–1 + x i + x i+1 )/3

18. Disadvantages of Transformation Rules Subsequent computations (such as the indexing problem) become more complicated We will see later why!

19. Dynamic Time Warping [Berndt, Clifford, 1994] Allows acceleration-deceleration of signals along the time dimension Basic idea Consider X = x 1, x 2, …, x n, and Y = y 1, y 2, …, y n We are allowed to extend each sequence by repeating elements Euclidean distance now calculated between the extended sequences X’ and Y’

20. X Y warping path j = i – w j = i + w Dynamic Time Warping [Berndt, Clifford, 1994]

21. Restrictions on Warping Paths Monotonicity Path should not go down or to the left Continuity No elements may be skipped in a sequence Warping Window | i – j | <= w

22. Formulation Let D(i, j) refer to the dynamic time warping distance between the subsequences x 1, x 2, …, x i y 1, y 2, …, y j D(i, j) = | x i – y j | + min { D(i – 1, j), D(i – 1, j – 1), D(i, j – 1) }

23. Solution by Dynamic Programming Basic implementation = O(n 2 ) where n is the length of the sequences will have to solve the problem for each (i, j) pair If warping window is specified, then O(nw) Only solve for the (i, j) pairs where | i – j | <= w

24. Longest Common Subsequence Measures (Allowing for Gaps in Sequences) Gap skipped

25. Basic LCS Idea X = 3, 2, 5, 7, 4, 8, 10, 7 Y = 2, 5, 4, 7, 3, 10, 8, 6 LCS = 2, 5, 7, 10 Sim(X,Y) = |LCS| or Sim(X,Y) = |LCS| /n Edit Distance is another possibility

26. [Ge & Smyth, 2000] Previous methods primarily “distance based”, this method “model based” Basic ideas Given sequence Q, construct a model M Q (i.e. a probability distribution on waveforms) Given a new pattern Q’, measure similarity by computing p(Q’|M Q ) Probabilistic Generative Modeling Method

27. The model M Q a discrete-time finite-state Markov model each segment in data corresponds to a state data in each state typically generated by a regression curve a state to state transition matrix is provided

28. On entering state i, a duration t is drawn from a state- duration distribution p(t) the process remains in state i for time t after this, the process transits to another state according to the state transition matrix

29. Solid lines: the two states of the model Dashed lines: the actual noisy observations Example: output of Markov Model

30. Landmarks [Perng et. al., 2000] Similarity definition much closer to human perception (unlike Euclidean distance) A point on the curve is a n-th order landmark if the n-th derivative is 0 Thus, local max and mins are first order landmarks Landmark distances are tuples (e.g. in time and amplitude) that satisfy the triangle inequality Several transformations are defined, such as shifting, amplitude scaling, time warping, etc

31. Similarity Models Euclidean and Lp based Edit Distance and LCS based Probabilistic (using Markov Models) Landmarks Next, how to index a database for similarity retrieval…

32. Main idea Seq. scanning works - how to do faster?

33. Idea: ‘GEMINI’ (GEneric Multimedia INdexIng) Extract a few numerical features, for a ‘quick and dirty’ test

34. day 1365 day 1365 S1 Sn F(S1) F(Sn) ‘GEMINI’ - Pictorially eg, avg eg,. std

35. GEMINI Solution: Quick-and-dirty' filter: extract n features (numbers, eg., avg., etc.) map into a point in n-d feature space organize points with off-the-shelf spatial access method (‘SAM’) discard false alarms

36. GEMINI Important: Q: how to guarantee no false dismissals? A1: preserve distances (but: difficult/impossible) A2: Lower-bounding lemma: if the mapping ‘makes things look closer’, then there are no false dismissals

37. GEMINI Important: Q: how to extract features? A: “if I have only one number to describe my object, what should this be?”

38. Time sequences Q: what features?

39. Time sequences Q: what features? A: Fourier coefficients (we’ll see them in detail soon) B: Moments (average, variance, etc) …. (more next!) D feature (F(x), F(y)) <= D(x, y)

40. Indexing Use SAM and index the objects in the feature space (ex. R-tree) Fast retrieval and easy to implement

41. Sub-pattern matching Problem: find sub-sequences that match the given query pattern

42. day $price 1 300 day $price 1365 day $price 1 400 30

43. Sub-pattern matching Q: how to proceed? Hint: try to turn it into a ‘whole-matching’ problem (how?)

44. Sub-pattern matching Assume that queries have minimum duration w; (eg., w=7 days) divide data sequences into windows of width w (overlapping, or not?)

45. Sub-pattern matching Assume that queries have minimum duration w; (eg., w=7 days) divide data sequences into windows of width w (overlapping, or not?) A: sliding, overlapping windows. Thus: trails Pictorially:

46. Sub-pattern matching

47. sequences -> trails -> MBRs in feature space

48. Sub-pattern matching Q: do we store all points? why not?

49. Sub-pattern matching Q: how to do range queries of duration w?

50. Next: more on feature extraction and indexing