1. Online Rank Elicitation for Plackett- Luce: A Dueling Bandits Approach Balázs Szörényi Technion, Haifa, Israel / MTA-SZTE Research Group on Artificial Intelligence, Hungary Róbert Busa-Fekete, Adil Paul, Eyke Hüllermeier Department of Computer Science, University of Paderborn Paderborn, Germany Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS 2015)

2. Problem: Rank Elicitation from pairwise preferences > > > 1. 2. 3. 4.

3. Moving to online setting

4. The online ranking problem

5. Stochastic transitivity assumptions

6. Connection with sorting algorithm  Naively apply a sorting algorithm as sampling scheme  Since all the pairwise comparisons are stochastic  A random order will be produced  What can we say about the optimality of such an order? >> > > >

7. Contributions

8. Preliminaries

9. Dueling Bandit Framework Continue or terminate? Prediction

10. Estimation of pairwise probability

11. The Plackett-Luce Model

13. Properties of PL Model  The marginal probabilities are easy to calculate  Satisfies the stochastic transitivity  Most probable ranking:  simply sort the items according to their skill parameters

14. Harmony of QuickSort and Plackett-Luce model

16. Budgeted QuickSort algorithm

18. Random tree of QuickSort algorithm 1 2 3 4 6 7 8 9 1 2 3 47 8 9 1 3 4 4 9 9

19. Random tree of Budgeted QuickSort 1 2 3 4 6 8 7 9 1 2 3 48 7 9 1 3 4 4

20. Pairwise Stability of Budget QuickSort

21. Proof sketch of Proposition 2

22. First Goal of learner: PAC-item

23. PLPAC Algorithm Goal: Finding the PAC item In each iteration,  Generate a partial ranking (line 6)  Translate ranking into pairwise comparisons  Update the estimates of marginal

24. PLPAC Algorithm

25. Sample Complexity analysis of PLPAC 4 1 3 2 6 8 7 9

26. Sample Complexity analysis of PLPAC

27. Second goal of learner: AMPR

29. PLPAC-AMPR algorithm

30. PLPAC-AMPR

31. PLPAC-AMPR algorithm

34. Sample Complexity Analysis of PLPAC- AMPR

35. Experiments – The PAC-item Problem

37. Experiment – The AMPR Problem

39. Conclusion