1. Answering Top-k Queries Using Views By: Gautam Das (Univ. of Texas), Dimitrios Gunopulos (Univ. of California Riverside), Nick Koudas (Univ. of Toronto), Dimitris Tsirogiannis (Univ. of Toronto) Presented By: Kushal Shah Lipsa Patel

2. Views Definition: Views Declaring Views Advantages of using Views

3. Views A view may be thought of as a table, that is derived from one or more underlying base table. Two kinds: 1.Virtual: Not stored in the database; just a query for constructing the relation. 2.Materialized: Actually constructed and stored.

4. Declaring Views Materialized: CREATE [MATERIALIZED] VIEW AS ; Virtual: Default

5. Advantages of using Views If we have several tables in a DB and we want to view only specific columns from specific tables we can go for views. Suffice the needs of security: Sometimes allowing specific users to see only specific columns based on the permission that we can configure on the views.

6. Answering Top-k Queries Using Views By: Gautam Das (Univ. of Texas), Dimitrios Gunopulos (Univ. of California Riverside), Nick Koudas (Univ. of Toronto), Dimitris Tsirogiannis (Univ. of Toronto) Presented By: Kushal Shah Lipsa Patel

7. Top-k Query Top-k Query Processing – Definition Top-k Example Algorithms for Top-k Query Processing

8. Top-k Query Processing Top-k query processing = Finding k objects that have the highest overall Score

9. Top-k Example R Users preferences regarding the ordering of the tuples of a relation can be expressed as a scoring functions on the attributes of a relation, eg f q = 3x1 + 2x2 + 5x3 The top-k problem is to find the k tuples with the highest score according to a given scoring function. t id X1X1 X2X2 X3X3 182159 2531983 3299915 480458 5283239 t id Score 2612 1543 4370 3360 5343

10. Algorithms for Top-k Query Processing How?  Which algorithms? – Related Work How we complement existing approaches? TA [Fagin] PREFER [Hristidis] Stores the multiple copies of a relation and each copy is ordered according to a different scoring function. In order to answer a top-k query the algorithm utilizes a single copy with a scoring function which is closest to the scoring function of the query.

11. (a, 0.9) (b, 0.8) (c, 0.72) (d, 0.6)........ Sorted L 1 (d, 0.9) (a, 0.85) (b, 0.7) (c, 0.2)........ N a b c d........ Object ID 0.9 0.8 0.72 0.6........ Attribute 1 0.85 0.2 0.9........ Attribute 2 0.7 M Sorted L 2 Example – Simple Database model

12. IDA1A1 A2A2 Min(A 1,A 2 ) Step 1: - parallel sorted access to each list (a, 0.9) (b, 0.8) (c, 0.72) (d, 0.6)........ L1L1 L2L2 (d, 0.9) (a, 0.85) (b, 0.7) (c, 0.2)........ a d 0.9 0.85 0.6 For each object seen: - get all grades by random access - determine Min(A1,A2) - amongst 2 highest seen ? keep in buffer Example – Threshold Algorithm

13. IDA1A1 A2A2 Min(A 1,A 2 ) a: 0.9 b: 0.8 c: 0.72 d: 0.6........ L1L1 L2L2 d: 0.9 a: 0.85 b: 0.7 c: 0.2........ Step 2: - Determine threshold value based on objects currently seen under sorted access. T = min(L1, L2) a d 0.9 0.85 0.6 T = min(0.9, 0.9) = 0.9 - 2 objects with overall grade ≥ threshold value ? stop else go to next entry position in sorted list and repeat step 1 Example – Threshold Algorithm

14. IDA1A1 A2A2 Min(A 1,A 2 ) Step 1 (Again): - parallel sorted access to each list (a, 0.9) (b, 0.8) (c, 0.72) (d, 0.6)........ L1L1 L2L2 (d, 0.9) (a, 0.85) (b, 0.7) (c, 0.2)........ a d 0.9 0.85 0.6 For each object seen: - get all grades by random access - determine Min(A1,A2) - amongst 2 highest seen ? keep in buffer b0.80.7 Example – Threshold Algorithm

15. IDA1A1 A2A2 Min(A 1,A 2 ) a: 0.9 b: 0.8 c: 0.72 d: 0.6........ L1L1 L2L2 d: 0.9 a: 0.85 b: 0.7 c: 0.2........ Step 2 (Again): - Determine threshold value based on objects currently seen. T = min(L1, L2) a b 0.9 0.7 0.85 0.8 0.7 T = min(0.8, 0.85) = 0.8 - 2 objects with overall grade ≥ threshold value ? stop else go to next entry position in sorted list and repeat step 1 Example – Threshold Algorithm

16. c IDA1A1 A2A2 Min(A 1,A 2 ) a: 0.9 b: 0.8 c: 0.72 d: 0.6........ L1L1 L2L2 d: 0.9 a: 0.85 b: 0.7 c: 0.2........ Situation at stopping condition a b 0.9 0.7 0.85 0.8 0.7 T = min(0.72, 0.7) = 0.7 Example – Threshold Algorithm 0.720.2

17. Related Work for Top-k Query Processing TA: Sequential as well as Random Access PREFER

18. Approach for Top-k Query Processing Top-k Query Answering using Views Views are Materialized (incurring space overhead) Advantages of using views: increased performance because views are small in size Space-Performance tradeoff

19. Example Views Rt id X1X1 X2X2 X3X3 182159 2531983 3299915 480458 5283239 Three attribute relation R V1V1 t id Score 3553 4385 5216 2201 1169 Top-5 query using function f1 = 2x1 + 5x2 V2V2 t id Score 2351 1237 5177 3159 488 Top-5 query using function f2 = x2 + 2x3 Top-k ranking queries in SQL-like syntax: SELECT TOP[k] FROM R ORDER BY Score(q) Score(q) - function that assigns numeric score to any tuple t Ranking Views: Views only aim to rank A ranking view is the materialized result of a previously asked top-k query. Can we answer new top-k queries efficiently using ranking views? Let’s see

20. Formal Definitions Ranking Queries Ranking Views

21. Ranking Queries Ranking Queries: Top-k ranking queries in SQL-like syntax: Select Top[k] from R where Range(q) Order By Score(q) A ranking query may be expressed as a triple Q = (Score(q), k, Range(q)), where Score(q)= Function that assigns numeric score to any tuple t Range(q) = defines selection condition for the tuples of R Semantics: Retrieve the k tuples with the top scores satisfying the selection condition.

22. Ranking Views Materialized Ranking View V : for a previously executed query Q 1 = (Score Q 1, k 1, Range Q 1 ), the corresponding materialized ranking view is a set of k(tid, score Q (tid)) pairs, ordered by decreasing values of score Q (tid).

23. Problems we are going to solve Top-k Query Answer using Views View Selection

24. Top-k Query Answer using Views Given: Set U of views Query Q Obtain an answer to Q combining all the information conveyed by the views in U Solution: Algorithm named LPTA

25. Problems we are going to solve Top-k Query Answer using Views View Selection

26. Problem: Given a collection of views V={V1…Vr} – base views and a query Q, determine the most efficient subset U of V to execute Q on. Input to LPTA: subset U Obtaining an answer to ranking query: Running TA on base views. Find the subset U that when utilized by LPTA 1. Provide answer to query 2. Provide answer faster than running TA on the base views V

27. Outline LPTA Algorithm View Selection Problem

28. LPTA: Linear Programming Adaptation of the Threshold Algorithm 1. Scoring function of Query: Q - f Q = 3x1 + 10x2 2. Scoring function of Views: V1 – f v1 = 2x1 + 5x2 Subset of Views UV2 – f v2 = x1 + 2x2 LPTA for Top-k Query Answer using Views Top-1 Query View is a set of pairs of (tuple identifier, score). The LPTA algorithm requires sorted access on each view in non-increasing order of that score.

29. LPTA Example tidx1x2x3 182159 2531983 32912 4802290 528887 6125582 7169942 8184267 942123 10232188 R tidScore 7527 6299 4270 8246 2201 V1 Top-5 Query f1 = 2x1 + 5x2 tidScore 6219 4202 10197 Top-3 Query f2 = x2 + 2x3 V2 Answer Top-2 Query using LPTA

30. LPTA Setting The algorithm initializes the top-k buffer to empty. Top-2 Buffer tidScore 7527 6299 4270 8246 2201tidScore 6219 4202 10197 V1V2 7169942 For each tid read, random access on R to retrieve tuple and compute score acc to query function f3 = 3x1 + 10x2 + 5x3 6125582 (7,1248) (6,996) Top-2 Buffer Check for stopping Condition

31. Check for Stopping Condition The unseen tuples in the view have satisfy the following inequalities: The domain of each attribute of R [1,100] 0<=X1,X2,X3<=100---------------------------(1) 2x1 + 5x2 <= sd1-------------------------------(2) x2 + 2x3 <= sd2---------------------------------(3) sd1 = 527 and sd2 = 219 Unseen max = Solution to the linear program where we maximize the function f3 = 3x1 + 10x2 + 5x3 subject to these inequalities. The solution of the linear program gives the maximum score of any unseen tuple. Unseen max = the maximum possible score (with respect to the ranking query’s scoring function) of any tuple not yet visited in the views. The algorithm terminates when the top-k buffer is full and Unseen max <= topk min

32. Calculating Unseen max Unseen max = Solution to the linear program where we maximize the function f3 = 3x1 + 10x2 + 5x3 subject to these inequalities. A linear programming problem may be defined as the problem of maximizing or minimizing a linear function subject to linear constraints. The constraints may be equalities or inequalities. Here is a simple example. Find numbers x1 and x2 that maximize the sum x1 + x2 subject to the constraints x1 ≥ 0, x2 ≥ 0, and x1 + 2x2 ≤ 4 4x1 + 2x2 ≤ 12 −x1 + x2 ≤ 1 Objective Function

33. Maximize the function Convex region This system of inequalities defines a convex region. Occasionally, the maximum occurs along an entire edge or face of the constraint set, but then the maximum occurs at a corner point as well.

34. LPTA - Example V1V2 Top-1 query V1 V2 Q stopping condition X1X1 X2X2 R(X 1, X 2 ) Normalized Domain[0,1] Views and top-k query represented by vectors denoting the direction of increasing score Sweeping line perpendicular to V1 from infinity to origin Score of a tuple with respect to the query: project that tuple to the vector of the query Score of a tuple with respect to a view: project that tuple to the the vector of the view Max posssible score of any tuple not yet visited in the views with respect to the scoring func of query UNSEEN MAX

35. LPTA - Example (cont’) V1V2 Top-1 V1 V2 Q stopping condition X1X1 X2X2 R(X 1, X 2 ) The algorithm will stop early if the scoring function of the views is “similar” to the scoring function of the query.

36. LPTA Algorithm LPTA Algorithm – Pseudo Code There is Sequential as well as Random Access. Sequential access on views Random Access on base table to find the tuple

37. Comparison of LPTA with TA LPTA becomes TA when the set of views U = set of base views Execution cost: Both have Sequential as well as Random Access These I/O Operations play a significant role – overshadow the costs of CPU operations such as updated top-k buffer, testing for stopping condition & so on.

38. Determining Factor for performance LPTA versus TA Highly correlated: every sequential access incurs a random access. As a result the determining factor for the performance is (distance from the beginning of the view each algorithm has to traverse (read sequentially) before coming into a halt with the correct answer) X (the number of views participating in the process). d=number of lock-step r = no of views Running Cost: O(dr)

39. Outline LPTA Algorithm View Selection Problem

40. Given a collection of views V = {V 1,…,V r } and a Query Q, determine the most efficient subset U C V to execute Q on. Conceptual discussion of View Selection  Two attribute relation (in two dimension)  Multi attribute relation (for any dimension) Domain of each attribute is normalized to [0,1] M-attribute relation is refer as m-dimension

41. View Selection – Two Dimension(same side) Min top-k tuple Q V1 V2 Square OPRT Two views V1 and V2 and Query Q are represented by vectors. Both the view vectors are to the same side (clockwise) of the query vector A B B1B1 B2B2 M AB 1 Q passes through M & intersect unit square ABR – Top-k tuples ABPOT – Remaining tuples Sorted access to V1 – sweeping line 1 to V1 from infinity to origin Stopping condition for V1: sweepline crosses AB1 b’coz convex polygon AB1POT – unseen tuples and score(unseen) <= Score(M) Number of sorted access V1 = NumTuples(AB1R)V2 = NumTuples(AB2R)

42. View Selection – Two Dimension(same side) Conclusion V2 is slower compared to V1 If several views in two dimension are available & all their vectors are to one side of query vector, then it is optimal for LPTA to use the vector that is closet to the query vector.

43. Estimating the Number of Tuples Estimating and Comparing the Number of Tuples by simply comparing the areas of respective triangles. Such approach: Need to have an uniform distribution within the triangles, which is often quite unrealistic. In our approach for view selection, utilize the conceptual conclusions + borrow knowledge of actual data distribution.

44. View Selection – Two Dimension(either side of query) A B Min top-k tuple Q V1 V2 A1 B1 M Can use only V1 or only V2 for execution If uses only v1 to answer the query the stopping condition will be reached once the sweepline perpendicular to v1 crosses position A1B/ For V2 - AB1

45. View Selection – Two Dimension(either side of query) A B Min top-k tuple Q V1 V2 A1 B1 M Running LPTA on both V1 and V2, rather than just running on only one of V1 or V2? Two views are better than one A1 1 B1 1 A2 1 B2 1 The intersection point of the sweep lines perpendicular to v1 and v2 is on the line AB The stopping condition is reached when the sweeplines resp crosses A1 1 B1 1 and A2 1 B2 1 such that 1) intersection pt of A1 1 B1 1 and A2 1 B2 1 is on line AB 2) NumTuples(A1 1 B1 1 R) = NumTuples(A21B21PR) since algo sweeps each view in lock-step

46. LPTA on both Views versus One For two views the position of each sweepline is before the respective stopping positions if only one view has been used. Total number of sorted accesses for two views: NumTuples (A1 1 B1 1 R) + NumTuples (A2 1 B2 1 R) = 2 NumTuples (A1 1 B1 1 R) If Min ( NumTuples (A1BR), NumTuples (AB1PR), 2 NumTuples (A1 1 B1 1 R) ) = NumTuples (A1BR) - Use V1 If Min ( NumTuples (A1BR), NumTuples (AB1PR), 2 NumTuples (A1 1 B1 1 R) ) = NumTuples (AB1PR) - Use V2 Else use both V1, V2

47. Theorem for Two Dimensional Case Theorem 1: Set of Views = {V 1,…,V r } Query = Q Two Dimensional dataset V a = Closest to query in Anticlockwise V c = Closest to query in Clockwise So they are on either side of the query Optimal execution of LPTA requires the use of either V a or V c i.e., the use of subset from {V a, V c }

48. View Selection – Higher Dimension Extension of Theorem 1 Theorem 2: Set of Views = {V 1,…,V r } Query = Q m-dimensional dataset Optimal execution of LPTA requires the use of subset of views U C V such that |U| <= m

49. Outline LPTA Algorithm View Selection Problem  Cost Estimation Framework

50. Cost Estimation Framework – Running LPTA Cost Estimation Framework: The cost of running LPTA when a specific set of views is used to answer a query. Cost = total number of sequential accesses in a view Uses 2 views to answer a query Cost = 6 sequential accesses Min top-k tuple Can we find that cost without actually running LPTA? A B Q V1 V2

51. Cost Estimation Framework – without Running LPTA EstimateCost(Q, U): Returns an estimate of the cost of running LPTA on exactly this set of views: U Used within SelectViews(Q,V) to search the subset U that minimizes EstimateCost(Q,U) EstimateCost(Q,U) takes into account – Multi-attribute views –Non-uniform data distribution

52. Simulating LPTA on Histograms rather than on views U Equi-depth histograms: The number of tuples in each bucket is the same Base Table R : n tuples (10) H i Equi-depth histogram b buckets – 2buckets : represent the distribution of points along the X i attribute Each bucket will represent n/b data points 10/2 = 5 data points

53. Simulating LPTA on Histograms rather than on views U In our estimation procedure: H Q – represents the distribution of score of all tuples of the database according to the scoring function Q Cannot calculate the score of all tuples, so approximate H Q

54. Simulation of LPTA on Histograms Simulate LPTA in a bucket by bucket lock step to estimate the cost. HQHQ H V1 H V2 topk min H Q : approximates the score distribution of the query Q b buckets histograms for the score distribution of views n/b tuples per bucket Cost We cannot afford to run LPTA on views U Pre-estimate topk min b’coz we do not have access to actual tuples or their tids. The value of topk min is estimated from H Q by determining the bucket that contains the kth highest tuple. Since topk min is very likely inside this bucket we use linear interpolation with in the bucket to estimate the topk min Cheap procedure because we have one iteration of the LPTA algorithm for every n/b tuples using the values from the bucket boundaries. Approx the value of func

55. Calculating the Estimated cost Number of buckets visited along each view’s = d(3) Number of views = r 1 (2) Number of tuples per bucket n/b (10) Compute the smallest number of tuples n 1 need to be scanned from the last bucket before stopping Estimated number of sorted access ((d-1)n/b +n 1 ) r 1 ((2)(10) + 2) 2 = 44 Therefore running time is O((d-1) + logn 1 ) lock-step iteration

56. Outline LPTA Algorithm View Selection Problem  Cost Estimation Framework  View Selection Algorithms

57. EstimateCost(Q, U) EstimateCost(Q, U) – Pseudo- code SelectViews(Q, V) : Select the subset of views U which minimizes the EstimateCost Exhaustive (E) Approach: Estimate the cost of all possible subsets of V and select the subset of views with the smallest cost. Feasible for database with few attributes Greedy Approach: Keep expanding the set of views to use until the estimated cost stops reducing. SelectViews(Q,V) – Pseudo codeSelectViews(Q,V)

58. Requires the solution of a single linear program. Fix the score s Uniform Data distribution & very cheap Maximize the scoring function of the query Max(fq) using the inequalities that scoring function of each view <= s /fv <= s Q Selected Views whose hyperplanes intersect at the point which maximize the scoring function SelectViewsSpherical (SVS) T

59. Select Views By Angle (SVA) Select Views By Angle (SVA): Sort the views by increasing angle with respect to Query vector. Q Selected Views: view closer to query will result in minimum running time for the algo V1 V2V3V4

60. Outline LPTA Algorithm View Selection Problem  Cost Estimation Framework  View Selection Algorithms Experimental Evaluation

61. Two types of dataset: Real and synthetic (uniform and zipf data with varying skew distribution) The real dataset contains 30K tuples from a website specialized on automobiles. Experiments Conducted: –Performance comparison of LPTA, PREFER and TA –Performance of LPTA using each of the view selection algorithms –Scalability of the LPTA algorithm

62. Performance comparison of LPTA, PREFER and TA Uniform dataset, 3dReal dataset, 2d

63. Conclusions Using views for top-k query answering LPTA: linear programming adaptation of TA View selection problem, cost estimation framework, view selection algorithms Experimental evaluation

64. References Answering Top-k Queries Using Views: Gautam Das, Dimitrios Gunopulos, Nick Koudas Optimal Aggregation Algorithms for Middleware : Ronald Fagin, Amnon Lotem & Moni Naor aitrc.kaist.ac.kr/~vldb06/slides/R13-1.ppt