1. Feature Selection for Ranking
Xiubo Geng, Tie-Yan Liu, Tao Qin, Hang Li
---- SIGIR 2007

2. What’s Ranking? Ranking: a classical problem in information retrieval
Supervised ranking
Ranking SVM
RankNet

3. Feature Selection Well studied in classification Enhance accuracy
Improve efficiency
Avoid over-fitting

4. What’s new for Ranking?
In ranking, a number of ordered categories are used; in classification, the categories are “flat”.
Different evaluation measures:
Ranking focuses more on precision than recall; Classification cares both precision & recall.
Top-n instances are more critical for ranking; whereas instances are weighted equivalently in classification.

5. What’s new for Ranking?
Mean average precision(MAP)
Suppose you submit a query, and it has 4 relevant docs.
Retrieve Ranking: Y, N, N, Y
MAP = 1x1/4+0.5x0/2+0.3x0/2+0.5x1/4
=0.375

6. MAP (A Toy Example) A query has 5 relevant documents,
Retrieve ranking: Y, Y, N, N
Answer?

7. Normalized discount cumulative gain (NDCG)
If have multiple level of relevance, then use NDCG
R(j) denotes score for rank j and Zn is a normalization factor to guarantee that a perfect ranking’s NDCG at position n equals 1.

8. Overview of Algorithm
Goal: select t features from the entire feature set.
Procedure
Define the importance score of each feature
Define the similarity between any two features.
Greedy method to maximize the total importance whereas minimize the total similarity.

9. Importance & Similarity
Rank instances using one feature and using the evaluation measure MAP or NDCG as importance score.
Similarity between features:
Use ranking distance like Kendell’s tao measure.

10. An optimization formula
Wi is the importance score,
ei,j is similarity between two features.
Integer programming, typically not easy to solve.
Thus, adopt an greedy algorithm.

11. Greedy Approach
Construct a undirected graph with each node represent one feature. The node weight is importance score and edges between nodes are the similarity.
For each feature,
Select the node (VKi)with maximum weight
A punishment is conducted to all the other nodes according to their similarities.
Add Vki to the selected features and remove the node and corresponding edges from the graph.

12. Data Sets .gov dataset of Web TREC 2004. OHSUMED data
1,053,110 docs, only a subset is used.
75 queries with binary relevance.
44 features for each doc.
OHSUMED data
a bibliographical document collection, a subset of MEDLINE database.
16,140 query-document pairs
three levels of relevance: definitely/possibly/not relevant
26 features.

13. Experimental Results(1)

14. Experimental Results(2)