1 Ranked Queries over sources with Boolean Query Interfaces without Ranking Support Vagelis Hristidis, Florida International University Yuheng Hu, Arizona State University Panos Ipeirotis, New York University

2 Motivation: PubMed (and USPTO, and Linked In, and…)  PubMed offers only ranking by date, author, title, or journal  Usually, user like ranking by relevance – Measured by IR ranking function, like tf-idf

3 Problem Definition  Input – Query Q contains term t 1, …t n – Database D contain documents d 1,…,d m  Output – Top-k documents ranked according to a relevance score function  Example of ranking function: tf.idf  Baseline: Submit a disjunctive query with all query keywords, retrieve all the documents, locally re-rank  Problems with Baseline method: Too many results! – “immunodeficiency virus structure”  1,451,446 results

4 Query Relaxation Approach  A tf.idf query has OR semantics  Using queries will AND semantics returns promising documents earlier on  Gradual query relaxation allows fast execution  Key questions:  Which (conjunctive) queries to execute?  When to stop?

Problem Setting and Challenges  Boolean query interface, (e.g PubMed)  Limited data access through web service (quota per day)  No useful ranking functions  No indices to rely on  No statistics exported from database 5

6 Probabilistic Approach  Document Score  Estimate tf (and scores) probabilistically: – The tf of the terms in a database tend to follow a Poisson distribution – Document scores also follow a Poisson tf parameter of Poisson for the term in database idf, (easy part) tf, (challenging part)

7 Probabilistic top-k with query relaxation  Querying strategy – How to pick a good query candidate? – A good query should have good “benefit”  Benefit: Probability that document in results for relaxed query q in top-k. The k-th highest score so far Query candidate We choose the query candidate q with maximum probability Score follows Poisson, function of the λ parameters of query terms in Q Pr{Score Q (D,q) > τ}

8 Estimation of Poisson Parameters  Sample-based estimation: Fetch documents, construct sample, use estimates from sample – Need very extensive sampling size for reliable estimates  Query-based estimation: Combine sampling and query execution – Every query generates a sample and provides candidate top-k docs – Main challenge: Adjust estimates to compensate for querying bias (we are looking for top-k documents, we do not perform random sampling)

9  Document sample returned for each query is not random!  Sample is “conditional” on query terms (guaranteed to appear) – Need to acknowledge in estimates that queries are trying to find the top-k, not intended for random sampling  Without correction, estimates significantly off Query-based Sampling

10 Top-k algorithm using query relaxation 1.Send conjunctive query to the database with all terms 2.Update statistics for each term using estimates from the biased sample 3.Compute benefits for each possible query relaxation 4.If benefit (i.e., probability of finding top-k document) below threshold, stop; else go to step 1

Experiments  Datasets – PubMed – TREC  Quality Measure – Spearman’s Footrule  Algorithms – Baseline – Summary-based – Query-based 11

Experiments: Quality 12  Compared footrule distance compared to baseline (baseline = retrieve everything, fetch locally, rerank)  Lower values better  Query-based sampling consistently better than alternatives

Experiments: Efficiency 13  Measured #documents, queries, and execution time of alternative techniques

Conclusion 14  Technique for top-k queries on top of document databases without ranking support  Introduction of an exploration-exploitation framework for building necessary statistics on-the-fly, during query execution  Order-of-magnitude efficiency improvements, small losses in quality

Thank you ! Questions? 15