The Database and Info. Systems Lab. University of Illinois at Urbana-Champaign RankFP : A Framework for Rank Formulation and Processing Hwanjo Yu, Seung-won Hwang, Kevin Chen-Chuan Chang

AIM 2 The Context: AIMing to the Top Enabling ad-hoc ranking in data retrieval Rank Formulation Rank Processing select * from houses order by [ranking function F] limit 3 ranked results query top-3 houses

AIM 3 Problem: Enabling Ad-hoc Ranking To enable ad-hoc ranking, we observe two major barriers: Usability: Ranking should be “user-friendly”, for ordinary users to easily specify their ranking criteria Efficiency: Ranking should be “DB-friendly” to be amenable to efficient processing  We propose a framework combining user-friendly formulation and DB-friendly processing.

AIM 4 Our Insight: Combining Usability and Efficiency We combine qualitative model for usability and quantitative model for efficiency Qualitative model  Query condition is represented as a relative ordering of objects  User-friendly by alleviating user from specifying the absolute score on each object  Example > Quantitative model  Query condition is represented as a mapping F of objects into absolute numerical scores  DB-friendly, by attaining the absolute score on each object  Example F( )=0.9 F( )=0.5

AIM 5 Our Solution: RankFP (RANK Formulation and Processing) For usability, we propose a qualitative formulation front-end which enables rank formulation by ordering samples For efficiency, we learn a quantitative ranking function F which is readily expressible using order by clause in SQL sample S (unordered) Sample Selection: generate new S Function Learning: learn new F ranking R* over S Over S: R F  R* ? no yes F ranking function Rank Formulation Rank Processing ranked results processing of Q Q: select * from houses order by F limit k

AIM 6 Implementation SVM Learner order by F sampled top results interface PostgreSQL Top-k results if R F  R* ?

AIM 7 Task 1: Rank Formulation Front-end (Ranking  Classification) Challenge: Unlike a conventional learning problem of classifying objects into groups, we need to learn a function inducing a desired ordering of all objects Solution: Transform ranking into a classification on pairwise differences [Herbrich2000] and adopt learning algorithms (e.g., SVM) to learn pairwise classification function F learning algorithms: a binary classifier 1 - F a - b b - c c - d d - e a - c … … ranking view: c > b > d > e > a c b d e a classification view: pairwise diff. classification [Herbrich2000] R. Herbrich, et. al. Large margin rank boundary for ordinal regression. MIT Press, 2000.

AIM 8 Task 2: Rank Processing Back-end (Classification  Ranking) Challenge: While the classification is for each pair of objects, we need to efficiently rank the entire database. Solution: We develop duality connecting a pairwise classification function F, also as a global per-object ranking function. Suppose the rank function F is linear Classification View: Ranking View: F(u i -u j )>0  F(u i )- F(u j )>0  F(u i )> F(u j ) b d e a c F(a-b)? F(a-c)? F(a-d)? ….. F Rank with F(. ) e.g., F(c)>F(b)>F(d)>… Further: Optimization of Top-k Order-by [SIGMOD’05]

AIM 9 Conclusion: Summary To support ranking for data retrieval, we develop RankFP, an iterative learning and processing framework, combining: Usability: Developing a learning front-end, which enables qualitative rank formulation Efficiency: Transforming the classification to a global rank function for efficient processing

AIM 10 Thank You! For more information: The AIM Project: