1. Multi-Task Learning for Boosting with Application to Web Search Ranking Olivier Chapelle et al. Presenter: Wei Cheng

2. Outline Motivation Backgrounds Algorithm – From svm to boosting using L1 regularization – Є-boosting for optimization – Overall algorithm Evaluation Overall review and new research points discussion

3. Motivation Different/same search engine(s) for different countries? Domain specific engine is better! e.g. ‘gelivable’ (very useful) Domain specific engine is better! e.g. ‘gelivable’ (very useful)

4. Motivation Should we train ranking model separately? – Corps in some domains might be too small to train a good model – Solution: Multi-task learning

5. Backgrounds Sinno Jialin Pan and Qiang Yang, TKDE’2010

6. Backgrounds Sinno Jialin Pan and Qiang Yang, TKDE’2010

7. Backgrounds Why transfer learning works? Sinno Jialin Pan et al. At WWW’10

8. Backgrounds Why transfer learning works?(continue)

9. Backgrounds Why transfer learning works?(continue)

10. Backgrounds LearnerA Input: Target: LearnerB Dog/human Girl/boy traditional learning

11. Backgrounds Joint Learning Task Input: Target: Dog/human Girl/boy Multi-task learning

12. Algorithm The algorithm aims at designing an algorithm based on gradient boosted decision trees Inspired by svm based multi-task solution and boosting-trick. Using Є-boosting for optimization

13. Algorithm From svm to boosting using L1 regularization Previous svm based multi-task learning:

14. Algorithm Svm(kernel-trick)---  boosting (boosting trick) Pick set of non-linear functions(e.g., decision trees, regression trees,….) H |H|=J Apply every single function to each data point X  ф(X)

15. Є-boosting for optimization Using L1 regulization Using Є-boosting

16. Algorithm

17. Evaluation Datasets

18. Evaluation

23. Overall review and new research point discussion Contributions: – Propose a novel multi-task learning method based on gradient boosted decision tree, which is useful for web-reranking applications. (e.g., personalized search). – Have a thorough evaluation on we-scale datasets. New research points: – Negative transfer: – Effective grouping: flexible domain adaptation

24. Q&A Thanks!