1. IO-Top-k: Index-access Optimized Top-k Query Processing Debapriyo Majumdar Max-Planck-Institut für Informatik Saarbrücken, Germany Joint work with Holger Bast, Ralf Schenkel, Martin Theobald, Gerhard Weikum VLDB 2006, Seoul, Korea

2. Setup priceresolutionzoom camera 1 €300 camera 5 8MP camera 3 7x camera 3 €330 camera 1 7MP camera 1 5x camera 5 €490 camera 4 6MP camera 2 4x camera 4 €580 camera 2 4MP camera 5 4x ………… ………… ………… Pre-computed index-lists over multiple attributes lists are accessible by both sorted and random accesses combine scores by some monotonic aggregation function: . res + .  zoom - . price Goal: find the top-k items with highest total scores single numeric score for every item for each attribute

3. Top-k algorithms: example lists sorted by score item 25 0.6 item 17 0.6 item 83 0.9 item 78 0.5 item 38 0.6 item 17 0.7 item 83 0.4 item 14 0.6 item 61 0.3 item 17 0.3 item 5 0.6 item 81 0.2 item 21 0.2 item 83 0.5 item 65 0.1 item 91 0.1 item 21 0.3 item 10 0.1 item 44 0.1 List 1 List 2 List 3 Fagin’s NRA Algorithm:

4. Top-k algorithms: example lists sorted by score item 25 0.6 item 17 0.6 item 83 0.9 item 78 0.5 item 38 0.6 item 17 0.7 item 83 0.4 item 14 0.6 item 61 0.3 item 17 0.3 item 5 0.6 item 81 0.2 item 21 0.2 item 83 0.5 item 65 0.1 item 91 0.1 item 21 0.3 item 10 0.1 item 44 0.1 Fagin’s NRA Algorithm: round 1 item 83 [0.9, 2.1] item 17 [0.6, 2.1] item 25 [0.6, 2.1] Candidates min top-2 score: 0.6 maximum score for unseen items: 2.1 min-top-2 < best-score of candidates List 1 List 2 List 3 read one item from every list current score best-score

5. Top-k algorithms: example lists sorted by score Fagin’s NRA Algorithm: round 2 item 17 [1.3, 1.8] item 83 [0.9, 2.0] item 25 [0.6, 1.9] item 38 [0.6, 1.8] item 78 [0.5, 1.8] Candidates min top-2 score: 0.9 maximum score for unseen items: 1.8 item 25 0.6 item 17 0.6 item 83 0.9 item 78 0.5 item 38 0.6 item 17 0.7 item 83 0.4 item 14 0.6 item 61 0.3 item 17 0.3 item 5 0.6 item 81 0.2 item 21 0.2 item 83 0.5 item 65 0.1 item 91 0.1 item 21 0.3 item 10 0.1 item 44 0.1 List 1 List 2 List 3 read one item from every list min-top-2 < best-score of candidates

6. Top-k algorithms: example lists sorted by score item 83 [1.3, 1.9] item 17 [1.3, 1.9] item 25 [0.6, 1.5] item 78 [0.5, 1.4] Candidates min top-2 score: 1.3 maximum score for unseen items: 1.3 item 25 0.6 item 17 0.6 item 83 0.9 item 78 0.5 item 38 0.6 item 17 0.7 item 83 0.4 item 14 0.6 item 61 0.3 item 17 0.3 item 5 0.6 item 81 0.2 item 21 0.2 item 83 0.5 item 65 0.1 item 91 0.1 item 21 0.3 item 10 0.1 item 44 0.1 Fagin’s NRA Algorithm: round 3 List 1 List 2 List 3 read one item from every list min-top-2 < best-score of candidates no more new items can get into top-2 but, extra candidates left in queue

7. Top-k algorithms: example item 25 0.6 item 17 0.6 item 83 0.9 item 78 0.5 item 38 0.6 item 17 0.7 item 83 0.4 item 14 0.6 item 61 0.3 item 17 0.3 item 5 0.6 item 81 0.2 item 21 0.2 item 83 0.5 item 65 0.1 item 91 0.1 item 21 0.3 item 10 0.1 item 44 0.1 lists sorted by score item 17 1.6 item 83 [1.3, 1.9] item 25 [0.6, 1.4] Candidates min top-2 score: 1.3 maximum score for unseen items: 1.1 Fagin’s NRA Algorithm: round 4 List 1 List 2 List 3 read one item from every list min-top-2 < best-score of candidates no more new items can get into top-2 but, extra candidates left in queue

8. Top-k algorithms: example item 25 0.6 item 17 0.6 item 83 0.9 item 78 0.5 item 38 0.6 item 17 0.7 item 83 0.4 item 14 0.6 item 61 0.3 item 17 0.3 item 5 0.6 item 81 0.2 item 21 0.2 item 83 0.5 item 65 0.1 item 91 0.1 item 21 0.3 item 10 0.1 item 44 0.1 lists sorted by score item 83 1.8 item 17 1.6 Candidates min top-2 score: 1.6 maximum score for unseen items: 0.8 Done! Fagin’s NRA Algorithm: round 5 List 1 List 2 List 3 read one item from every list no extra candidate in queue

9. Top-k algorithms NRA performs only sorted accesses (SA) (No Random Access) Random access (RA) –lookup actual (final) score of an item –costlier than SA (100 – 100,000 times), c R /c S := (cost of RA)/(cost of SA) –often very useful CA (Combined Algorithm), (Fagin et al., 2001) –one RA after every c R /c S SAs –total cost of SA ~ total cost of RA Measure of effectiveness (access cost): #SA + c R /c S x #RA Full-merge: compute scores for all items followed by partial sort –simple and efficient –important baseline for any top-k algorithm Problems with NRA, CA –high bookkeeping overhead: cannot beat full-merge in runtime –for “high” values of k, gain in even access cost not significant

10. Top-k algorithms Greedy heuristics for sorted access scheduling, based on crude estimate of scores (Guntzer, Balke, Kiessling, ITCC 2001) RankSQL: ordering of binary rank joins at query planning time (Ilyas et al., SIGMOD ’04 and Li et al., SIGMOD ’05) Scheduling RAs on “expensive predicates”, where SAs may not even be possible on all attributes (our setting is different) –MPro (Chang and Hwang, SIGMOD 2002) –Upper, Pick (Bruno, Gravano and Marian, ICDE ’02, ACM TODS ’04) Probabilistic pruning of candidates, RA scheduling (Theobald, Schenkel and Weikum, VLDB ’04, VLDB ’05) Main related previous works: NRA, CA

11. Our algorithm: IO-Top-k lists sorted by score item 25 0.6 item 17 0.6 item 83 0.9 item 78 0.5 item 38 0.6 item 17 0.7 item 83 0.4 item 14 0.6 item 61 0.3 item 17 0.3 item 5 0.6 item 81 0.2 item 21 0.2 item 83 0.5 item 65 0.1 item 91 0.1 item 21 0.3 item 10 0.1 item 44 0.1 Round 1: same as NRA item 83 [0.9, 2.1] item 17 [0.6, 2.1] item 25 [0.6, 2.1] Candidates min top-2 score: 0.6 maximum score for unseen items: 2.1 List 1 List 2 List 3 min-top-2 < best-score of candidates not necessarily round robin

12. Our algorithm: IO-Top-k lists sorted by score item 25 0.6 item 17 0.6 item 83 0.9 item 78 0.5 item 38 0.6 item 17 0.7 item 83 0.4 item 14 0.6 item 61 0.3 item 17 0.3 item 5 0.6 item 81 0.2 item 21 0.2 item 83 0.5 item 65 0.1 item 91 0.1 item 21 0.3 item 10 0.1 item 44 0.1 Round 2 item 17 [1.3, 1.8] item 83 [0.9, 2.0] item 25 [0.6, 1.9] item 78 [0.5, 1.4] Candidates min top-2 score: 0.9 maximum score for unseen items: 1.4 List 1 List 2 List 3 min-top-2 < best-score of candidates not necessarily round robin

13. Our algorithm: IO-Top-k lists sorted by score item 25 0.6 item 17 0.6 item 83 0.9 item 78 0.5 item 38 0.6 item 17 0.7 item 83 0.4 item 14 0.6 item 61 0.3 item 17 0.3 item 5 0.6 item 81 0.2 item 21 0.2 item 83 0.5 item 65 0.1 item 91 0.1 item 21 0.3 item 10 0.1 item 44 0.1 Round 3 item 17 1.6 item 83 [1.3, 1.9] item 25 [0.6, 1.4] Candidates min top-2 score: 1.3 maximum score for unseen items: 1.1 List 1 List 2 List 3 min-top-2 < best-score of candidates not necessarily round robin potential candidate for top-2

14. Our algorithm: IO-Top-k lists sorted by score item 25 0.6 item 17 0.6 item 83 0.9 item 78 0.5 item 38 0.6 item 17 0.7 item 83 0.4 item 14 0.6 item 61 0.3 item 17 0.3 item 5 0.6 item 81 0.2 item 21 0.2 item 83 0.5 item 65 0.1 item 91 0.1 item 21 0.3 item 10 0.1 item 44 0.1 Round 4: random access for item 83 item 83 1.8 item 17 1.6 Candidates min top-2 score: 1.6 maximum score for unseen items: 1.1 Done! fewer sorted accesses carefully scheduled random access List 1 List 2 List 3 random access for item 83 no extra candidate in queue not necessarily round robin

15. Outline Our contributions –Inverted block-index data structure –Sorted access scheduling –Random access scheduling –Lower bound Experiments Conclusion

16. Inverted block-index Lists are first sorted by score

17. Inverted block-index Lists are first sorted by score sort each block by item-id 333 222 111 Top-k algorithm with block-index 111 22 33 111 222 333 full-merge blocks are sorted by item ids, efficiently merged by full-merge! and so on… full merge pruned split into blocks Choose block size balancing disk seek time and data transfer rate Low overhead: prune once every round

18. Sorted access scheduling List 1 List 2 List 3 Inverted Block-Index General Paradigm

19. Sorted access scheduling List 1 List 2 List 3 b 11 b 21 b 31 b 12 b 22 b 32 b 13 b 23 b 33 b 14 b 24 b 34 General Paradigm We assign benefits to every block of each list Optimization problem –Goal: choose a total of 3 blocks from any of the lists such that the total benefit is maximized –We can show: this problem is NP-Hard, the well known Knapsack problem reduces to it –But, the number of blocks to choose and number of lists to choose from are small: we can solve it by enumerating all possibilities –We choose the schedule with maximum benefit, and continue to next round Inverted Block-Index

20. Sorted access scheduling List 1 List 2 List 3 b 11 b 21 b 31 b 12 b 22 b 32 b 13 b 23 b 33 b 14 b 24 b 34 General Paradigm We assign benefits to every block of each list Optimization problem –Goal: choose a total of 3 blocks from any of the lists such that the total benefit is maximized –We can show: this problem is NP-Hard, the well known Knapsack problem reduces to it –But, the number of blocks to choose and number of lists to choose from are small: we can solve it by enumerating all possibilities –We choose the schedule with maximum benefit, and continue to next round Inverted Block-Index

21. Sorted access scheduling List 1 List 2 List 3 b 11 b 21 b 31 b 12 b 22 b 32 b 13 b 23 b 33 b 14 b 24 b 34 General Paradigm We assign benefits to every block of each list Optimization problem –Goal: choose a total of 3 blocks from any of the lists such that the total benefit is maximized –We can show: this problem is NP-Hard, the well known Knapsack problem reduces to it –But, the number of blocks to choose and number of lists to choose from are small: we can solve it by enumerating all possibilities –We choose the schedule with maximum benefit, and continue to next round Inverted Block-Index scans to different depths in lists

22. Sorted access scheduling List 1 List 2 List 3 Knapsack for Score Reduction (KSR) Pre-compute score reduction  ij of every block of each list : (max-score of the block – min-score of the block) Inverted Block-Index List 1 List 2 List 3    

23. Sorted access scheduling Knapsack for Score Reduction (KSR) Pre-compute score reduction  ij of every block of each list : (max-score of the block – min-score of the block) Candidate item d is already seen in list 3. If we scan list 3 further, score s d and best-score b d of d do not change: no benefit In list 2, d is not yet seen. If we scan one block (block 22 ) from list 2 –with high probability d will not be not found in that block: best-score b d of d decreases by  22 Benefit of block B in list i  d  B (1 - Pr[d found in B]) ~  d  B sum taken over all candidates d not yet seen in list i Inverted Block-Index List 1 List 2 List 3     item d [s d,b d ] scanned till some depth

24. Random access scheduling List 1 List 2 List 3 Redundant random accesses of CA CA: one RA after every c R /c S SAs Many RAs turn out to be redundant Our strategy: two-phase algorithm First sorted access rounds only, then switch to random access: no redundant random access Switch from SA to RA, when –max-score for unseen ≤ min-top-k score –estimated RA-cost ≤ total SA-cost so far –cost of SA ~ cost of RA List 1 List 2 List 3 CA: RA for item d But d is found anyway in subsequent SA round need to estimate cost of RA

25. Random access scheduling current min-top-3 score candidate items sorted by best score: CA style random access List 1 List 2 List 3 lists scanned till some depths by sorted access Estimating number of random accesses best-scores current scores Each random access can prune some candidates, so better estimate of #RAs necessary A crude upper estimate: #of items in queue pruned

26. Random access scheduling current min-top-3 score candidate items sorted by best score: CA style Estimating number of random accesses item d [s d,b d ] bdbd If there are at least three items before d with final score > b d, d will be pruned before random access random accesses d is pruned

27. Random access scheduling current min-top-3 score candidate items sorted by best score: CA style Estimating number of random accesses item d [s d,b d ] bdbd If there are less than three items before d with final score > b d, a random access for d must be made random accesses next: RA for d

28. Random access scheduling current min-top-k score candidate items sorted by best score Estimating number of random accesses item d [s d,b d ] bdbd Let d be the j-th item d j by best-score ordering For all i < j, define random variables F i,j F i,j = 1 if final-score(d i ) > the best-score(d), 0 otherwise We compute Pr[F i,j = 1] using histogram of the score distributions of the lists Observation: Pr[RA is made for d] = Pr[F 1,j +  + F j-1,j < k] Expected #of random accesses  j Pr[F 1,j +  + F j-1,j < k] the sum is taken over all candidate items For General k: There will be random access for d if and only if #of items before d with final score > b d is less than k j-1 items

29. Experiments: estimate of RA queue size EST DONE TREC Terabyte data, TREC 2005 adhoc task queries After all sorted accesses #items in queue, #RA estimated and #RA actually done Total RA for 50 queries

30. Lower bound: what is the best possible? List 1 List 2 List 3 Try every possible SA-schedule Count essential number of RAs that must be done

31. Lower bound: what is the best possible? List 1 List 2 List 3 Try every possible SA-schedule Count essential number of RAs that must be done #SA C R /C S x #RA = Total cost Schedule 16 x 10000 + 1000 x 75 = 135,000 block size 10,000

32. Lower bound: what is the best possible? List 1 List 2 List 3 Try every possible SA-schedule Count essential number of RAs that must be done #SA C R /C S x #RA = Total cost Schedule 16 x 10000 + 1000 x 75 = 135,000 Schedule 29 x 10000 + 1000 x 12 = 102,000 block size 10,000

33. Lower bound: what is the best possible? List 1 List 2 List 3 Try every possible SA-schedule Count essential number of RAs that must be done #SA C R /C S x #RA = Total cost Schedule 16 x 10000 + 1000 x 75 = 135,000 Schedule 29 x 10000 + 1000 x 12 = 102,000 Schedule 312 x 10000 + 1000 x 3 = 123,000 …………………… …………………… Lower bound……102,000 carefully engineered dynamic programming to try out all schedules block size 10,000

34. Experiments: TREC 1050100200500 0 4,000,000 k average cost (#SA + 1000 x #RA) full merge NRA CA IO-Top-k (OUR) lower bound 1050100200500 0 250 k average running time (milliseconds) full merge NRA IO-Top-k (OUR) 100 TREC Terabyte benchmark collection over 25 million documents, 426 GB raw data 50 queries from TREC 2005 adhoc task CA

35. Experiments: HTTP logs FIFA World Cup HTTP logs World cup 1998 1.3 billion HTTP requests schema Log( interval, user-id, bytes ) aggregated for each user within one-day intervals typical query: find k users with most usage during June 1-10 full merge NRA CA IO-Top-k (OUR) lower bound

36. Experiments: IMDB IMDB movie data more than 375,000 movies, 1,200,000 persons attributes: Title, Genre, Actors, Description 20 human generated queries

37. Conclusion We presented An inverted block-index data structure –efficient: optimizes disk access –performs fast merge in blocks, minimizes overhead Integrated sorted access and random access scheduling –SA scheduling: maximizes benefit of scanning blocks –RA scheduling: effectively estimate RA-cost at every round –postpone RA till the end of all SA: save redundant RAs Lower Bound –shows that our algorithm is close to the best possible

38. Thank you!

41. Appendix

42. Sorted access scheduling List 1 List 2 List 3 Knapsack for Benefit Aggregation (KBA) Pre-compute expected score e ij of an item seen in block j of list i : (average score of the block) Pre-compute score reduction  ij of every block of each list : (max-score of the block – min-score of the block) Inverted Block-Index List 1 List 2 List 3 e 11 e 21 e 31 e 12 e 22 e 32 e 13 e 23 e 33 e 14 e 24 e 34    

43. Sorted access scheduling Knapsack for Benefit Aggregation (KBA) Pre-compute expected score e ij of an item seen in block j of list i : (average score of the block) Pre-compute score reduction  ij of every block of each list : (max-score of the block – min-score of the block) Candidate item d is already seen in list 3. If we scan list 3 further, score s d and best-score b d of d do not change In list 2, d is not yet seen. If we scan one block from list 2 –either d is found in that block: score s d of d increases, expected increase = e 22 –or d is not found in that block: best-score b d of d decreases by  22 Benefit of block B in list i  d e B Pr[d found in B] +  B (1 - Pr[d found in B]) The sum is taken over all candidates d not yet seen in list i Inverted Block-Index List 1 List 2 List 3 e 11 e 21 e 31 e 12 e 22 e 32 e 13 e 23 e 33 e 14 e 24 e 34     item d [s d,b d ]

44. Random access scheduling: details current min-top-k score candidate items sorted by best score Estimating number of random accesses item d [s d,b d ] bdbd Let d be the j-th items by best-score ordering For all i < j, Define random variables F i,j which takes value 1 if final score of the i-th item is greater than the best-score of d, 0 otherwise Compute Pr[F i,j = 1] using the expected score gain of the i-th item from lists where it is not yet seen Also define a random variable R j which takes value 1 if a random access is made for d, 0 otherwise Observation: Pr[R j = 1] = Pr[F 1,j +  + F j-1,j < k] Let X j := F 1,j +  + F j-1,j Assume F i,j s are independent, then X j follows Poisson distribution with mean  i Pr[F i,j = 1] We can compute Pr[X j < k] using the incomplete gamma function Expected number of random accesses is  j E(R j ) =  j Pr[R j = 1] =  j Pr[X j < k] the sum is taken over all candidate items There will be random access for d if and only if #of items before d with final score > b d is less than k j-1 items

45. Other Experiments For different values of cost of RA compared to cost of SA C R/S ratio: 100, 1000 and 10000 varying query size title fields: average size 3 description fields: average size 8 TREC Terabyte collection indexed with BM25 scores query size: 3query size: 8 20,000,000 0 cost (cost of RA)/(cost of SA) cost 0 3,000,000

46. End of appendix