Zahid Irfan & Dr. Asim Karim (Advisor) (zahidi, CS-509-Masters of Science (CS) Project Lahore University of Management Sciences, Lahore, Pakistan 8 May 2004 Approximate Query Processing (AQP) in Data Streams

Acknowledgement This work is primarily based on the research paper “One-pass wavelets decompositions of data streams” by Gilbert, Muthukrishnan, Strauss and Kotidis, IEEE Trans. Knowledge and Data Engineering May/June, Work by Muthukrishnan, Piotr Indyk and of course Johnson-Lindenstrauss.

Introduction Streams and Streaming Models Wavelet Transform & Embedded Vectors Pseudo-Random Number Generator Implementation Details Test Results Conclusions and Future Work AQP in Data Streams

Lets solve a puzzle. Guess the missing number in a random sequence of numbers [1…N] without repetition. Introduction Space Requirements O (1). Time Complexity O (n). What about two numbers, three numbers …. and so on…

Data Stream “A sequence of digitally encoded signals used to represent information in transmission”. Input stream is the sequence a [i], arrives sequentially item by item. Data Streams

Applications Networks Data Monitoring. Applied to Traffic Flow Analysis World Wide Web. Website hits, statistics etc. Online Transactions Processing System Large Databases Query Processing Data Streams Applications

Time Series Comprises value of the same quantity over different time intervals. Typical examples Daily closing values of Stock Exchange Traffic at an IP-Link at time intervals. Stream Models

Cash Register Model Positive updates arrive over period of time. Typical examples well … Cash Register Cricket Scores Internet web-site hits or other statistics. Stream Models

Turnstile Model Fully dynamic model Updates are both negative & positive e.g. Passengers in an airport Relative Hardness Turnstile > Cash Register > Time Series “Depends and varies from application to application”. Stream Models

Wavelets A mathematical hierarchical tool for decomposition of signals/ functions. Types of Wavelets Haar Wavelets Daubechies Wavelets Many more… Wavelet Transform

Haar Wavelet Example Resolution Averages Detail Coefficients D = [2, 2, 0, 2, 3, 5, 4, 4] [2, 1, 4, 4][0, -1, -1, 0] [1.5, 4][0.5, 0] [2.75][-1.25] Haar Wavelet Decomposition [2.75, -1.25, 0.5, 0, 0, -1, -1, 0]

Wavelet in Space Haar Wavelets can be represented as the following. Example vector A of N=4, 4 coefficients. W1= 1/N*[ ], W2 = 1/N*[ ], W3=1/N*[ ], W4=1/N*[ ] 1 st Coefficient =.Average Coefficient 2 nd Coefficient =.Detail Coefficient 3 rd Coefficient =.Detail Coefficient 4 th Coefficient =.Detail Coefficient

Embedding Vectors Any n-point metric space can be embedded into an O(log 2 n) dimensional Euclidean space and L 1 metric with 1+є distortion f(v) = embedding for vector v =,, … >

Johnson-Lindenstrauss (JL) Lemma Simply stated ~ * Where j=1…k, k<<N r j is random vector= {1, -1 with equal probability} Implications Represent a vector in R N space in k-dimensional space. Benefits : Approximate Queries… ?? Johnson-Lindenstruass Lemma

~ * Approximate queries can be used by choosing special b. Query i th value choose b=[ …0] Range Query (i,j) value choose b=[ …0], where b[x]=1 for i<=x<=j. What's the catch?? … r j is also size of N. So where to store the random vectors?? AQP & JL-Lemma

Solution to large space over head is generate the random vectors on the fly!! Such as : for (i=0;i<k;i++) {srand (i); for (j=0;j<N;j++) { rand (); } } This solution works but there is a more elegant solution to this problem. Reed-Muller Codes Extractor. Pseudo-Random Generator

Reed-Muller Generator The Matrix values represent RM codes. RM (x,y)= Replace 0  1 & 1  -1 we get wavelet basis vectors.

Benefits of Reed-Muller Pseudo Random generator Generated on the fly. Every value is independently computed without anything to do with the previous values. Most nearly imitates Wavelet basis vectors. Hence the sketch contains most of the energy of the signal. Reed-Muller PR Generator

Things learnt so far There is a way to embed the N data into k<<N vectors JL-Lemma : ~ Reed-Muller Codes excellent imitators of both wavelet basis vectors as well as random vectors. Query Processing is possible thanks to JL- Lemma. Lessons so far !!

Implementation Details Implementation Trivia Implemented in Visual C Design follows Classes and Objects paradigm Test Results and graphs from MS Excel

Data Flow Diagram

Dataset Generator Synthetic Data Set was generated using Random Distributions. Normal Distribution Calling Telephone Number ~ (1000 lines) Receiving Telephone Number Exponential Distribution Call Time 0~512 minutes

Data Streamer The data streaming class offers methods, which help in useful imitation of a real-time data stream by continuously presenting the program with data. Type DataStreamer::getData();

Pseudo Random Generator This class calculates the Reed-Muller based Pseudo-random Numbers. type PseudoRandomGenerator::getRandom (int X,int Y); Uses the formula

Data Decomposition The data is decomposed into a sketch by calculating the dot product of data stream with O (log N) random vectors. The sketch is stored into Main Memory to be utilized by the query processing engine. Sketch [j]+=Data [i]*Random (i, j); Here i=(1,N) and j=(1,k);

Query Processing Engine The Query Processing Engine uses the sketch and a new vector b. Uses the same old JL-Lemma ~ * Setting various values of b result in theoretically any sort of query.

Point Query Processing Point Query Point Query can be processed by asking for any single value in the whole data stream. Point Query Algorithm Prepare b[i]={0 for i !=j, 1 for i=j} and generate QuerySketch[j] +=B[i] * Random (i,j); Result = (DataSketch * Query Sketch)/ N

Range Query Processing Range Query Range Queries specify the low and high between which the query is to be processed. Even multiple ranges can be specified Query Algorithm Prepare b[i]={0 for i !=j, 1 for i=j} and generate QuerySketch[j] +=B[i] * Random (i,j); Result = (DataSketch * Query Sketch)/ N

AQP Test Time Complexity Analysis Query Processing Accuracy with Data Size Query Processing Accuracy with Sketch Size

Time Complexity The following Time complexities were found to be linear in size of data. Sketching Time Query Processing Time

Time Complexity (Sketching)

Time Complexity (Query)

Accuracy versus Data Size Data Size versus Accuracy of Query PSNR (dB) versus Data Size Data Size is increased by Power of 2 Sketch size assumed to be log N

PSNR (dB) versus Data Size

Accuracy versus Sketch Size Accuracy of Query against the Sketch Size. PSNR (dB) versus Sketch Size Data Size is assumed to be constant = Sketch Size is varied

PSNR (dB) versus Sketch Size

Conclusions Space Complexity Reduction Prohibitively large data stream in sub-linear space. Time Complexity Reduction one-pass data stream algorithm. Scalability to multi-dimensions

Applications and Future Work Data Mining Streams Multimedia & Databases Trying it with Video coding might be fun or disaster Graph Theory Problems MST, Matching etc. need to be solved in the streaming model. Computational Geometry Earth observation data streams or weather data streams Solve any problem that can be modeled as a data stream

References S. Acharaya, P.B. Gibbons, V. Poosala and S. Ramaswamy, “Join Synopsis for Approximate Query Answering”, ACM In Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, J. M. Hellerstein, P. J. Haas and H. J. Wang, “Online Aggregation”, In the Proceedings of 1997 ACM SIGMOD International Conference on Management of Data, Y. E. Iaonnidis and V. Poosala, “Histograms-Based Approximation to Set- Valued Query Answers”, In the proceedings of 25th International Conference on Very Large Databases, K. Chakrabarti, M. Garofalakis, R. Rastogi and K. Shim, “Approximate Query Processing Using Wavelets”, The Proceedings of the 26th Conference on Very Large Databases, Eygpt, F. Olken, “Random Sampling in Databases”, PhD Thesis, University of California at Berkeley, A.C. Gilbert, Y. Kotidis, S. Muthukrishnan and M. J. Strass, “One-pass wavelet Decomposition of Data Streams”, IEEE Transactions of Knowledge and Data Engineering, Vol. 15, No.3, May/June A. Ta-Shma, D. Zuckerman, and S. Safra, “Extractors from Reed-Muller Codes” In Proceedings of 42nd Annual IEEE Symposium on Foundations of Computer Science, 2001.

Questions & Answers Thanks to the following for their sincere help in this project Dr. Asim Karim, Dr. Sarmad Abbasi, Dr. Asim Loan, Dr. Sohaib A. Khan and all my friends specially Laeeq Aslam and Aimal Tariq Rextin.