Multi-Task Learning for Boosting with Application to Web Search Ranking Olivier Chapelle et al. Presenter: Wei Cheng

Outline Motivation Backgrounds Algorithm – From svm to boosting using L1 regularization – Є-boosting for optimization – Overall algorithm Evaluation Overall review and new research points discussion

Motivation Different/same search engine(s) for different countries? Domain specific engine is better! e.g. ‘gelivable’ (very useful) Domain specific engine is better! e.g. ‘gelivable’ (very useful)

Motivation Should we train ranking model separately? – Corps in some domains might be too small to train a good model – Solution: Multi-task learning

Backgrounds Sinno Jialin Pan and Qiang Yang, TKDE’2010

Backgrounds Sinno Jialin Pan and Qiang Yang, TKDE’2010

Backgrounds Why transfer learning works? Sinno Jialin Pan et al. At WWW’10

Backgrounds Why transfer learning works?(continue)

Backgrounds Why transfer learning works?(continue)

Backgrounds LearnerA Input: Target: LearnerB Dog/human Girl/boy traditional learning

Backgrounds Joint Learning Task Input: Target: Dog/human Girl/boy Multi-task learning

Algorithm The algorithm aims at designing an algorithm based on gradient boosted decision trees Inspired by svm based multi-task solution and boosting-trick. Using Є-boosting for optimization

Algorithm From svm to boosting using L1 regularization Previous svm based multi-task learning:

Algorithm Svm(kernel-trick)---  boosting (boosting trick) Pick set of non-linear functions(e.g., decision trees, regression trees,….) H |H|=J Apply every single function to each data point X  ф(X)

Є-boosting for optimization Using L1 regulization Using Є-boosting

Algorithm

Evaluation Datasets

Evaluation

Overall review and new research point discussion Contributions: – Propose a novel multi-task learning method based on gradient boosted decision tree, which is useful for web-reranking applications. (e.g., personalized search). – Have a thorough evaluation on we-scale datasets. New research points: – Negative transfer: – Effective grouping: flexible domain adaptation

Q&A Thanks!