1. Rank-Sensitive Data Structures Iwona Bialynicka-Birula and Roberto Grossi (Università di Pisa) 12 th Symposium on String Processing and Information Retrieval (SPIRE 2005)

2. Overview Motivation Our results Solution outline

3. Motivation Output-sensitive data structures can still be too costly Most often additional criteria exist Examples Web pages – PageRank or similar Geometrical objects – Z-order Various databases – physical location News items – time stamp Biological databases – biological relevance...

4. Rank-sensitive data structures Output-sensitive data structures Data structure holds n items A query reports all (say l) matching items Query time is O(t(n)+l) Rank-sensitive data structures Additional, independent, a priori ranking function Additional parameter k at query time Query returns top k matching items sorted by rank Query time is O(t(n)+k)

5. Rank-sensitive – key features Returns items sorted in rank order Query time depends on k and not on l Even if l is small with respect to n, it can still be much too large to process (e.g. web search) Can be used in real-time systems No structure which requires sorting has this feature

6. Overview Motivation Our results Solution outline

7. Our model Tree data structures Result set is obtained from An interval of consecutive leaves or O(polylog(n)) such disjoint intervals

8. Our model – examples (1) Suffix tree (trie) occurrences of „010” 0 1 0

9. Our model – examples (2) Range tree (1D) Items in range  3; 19  12357111317192329 31

10. Our model – examples (3) Range tree (2D) Items in a 2D rectangle

11. Our model – examples (4) Hierarchy Similar items

12. Our results in this model D – data structure as defined above  D  – size of D (in memory words) s(n) – number of items stored in D (incl. copies) O(t(n)+l) – query time D – rank-sensitive version of D (static case) O(t(n)+k) – query time  D  D  + O(s(n)lg  n) – for any  0  1 D – rank-sensitive version of D (dynamic) O(t(n)+k) plus O(lgn/lglgn) per interval  D  D  + O(s(n)lgn/lglgn) O(lgn) per copy – rank change

13. Overview Motivation Our results Solution outline

14. Basic idea 76 67 81 18 43 34 52 25 16782345 12345678 O(nlgn) space

15. Query 76 67 81 18 43 34 52 25 16782345 12345678 Reduced to merging O(lgn) lists on the fly

16. Space reduction in static case 76 67 81 18 43 34 52 25 16782345 12345678 Chazelle 1988 O(nlg  n) space 10101010 10011001 01111000

17. Dynamic case Store explicit values in lists Weight-balanced B-tree Degree proportional to lgn/lglgn Dynamic fractional cascading Multi-Q-heaps Constant-depth hierarchical pipeline of heaps

18. Multi-Q-heaps Similar to Q-heap Stores up to O(lgN/lglgN) integers The integers are from the universe 0...O(N) Search, find-min, insert, delete in O(1) time Requires lookup tables of O(N) space Performs operations on any subset of items Simple implementation, no special instructions

19. Multi-Q-heaps in our solution (1) 3 11 2 6 7 1 26 13 16 O ( l g N ) ® l g N l g l g N ® l g N l g l g N ® l g N l g l g N Multi-Q 2 1 13... Multi-Q 1 Constant depth

20. Multi-Q-heaps in our solution (2) Nodes have non-constant degree 2139115 Multi-Q 3

21. Thank you Questions?