Bayesian inference for Plackett-Luce ranking models Bayesian inference for Packet-Lube ranking models John Guiver, Edward Snelson MSRC

Distributions over orderings Many problems in ML/IR concern ranked lists of items Data in the form of multiple independent orderings of a set of K items How to characterize such a set of orderings? Need to learn a parameterized probability model over orderings Examples: Meta-search, multi-player games and competitions

Notation Easy to get confused! Ordering: B, A, C Ranking: 2, 1, 3 for (A,B,C)

Distributions Ranking distributions are defined over the domain of all K! rankings (or orderings) A fully parameterised distribution would have a probability for each possible ranking which sum to 1. E.g. For three items: A ranking distribution is a point in this simplex A model is a parameterised family within the simplex

Plackett-Luce: vase interpretation Probability: vr vg vb In this talk we are looking at a particular parameterised ranking model with very nice properties which we’ll look at over the next two slides. In this model there is one parameter per item. Consider an example where we have three items Red Green and Blue with parameters vR, vG, and vB. In this picture these parameters represent the proportions of the different coloured balls in these infinite urns. First we mix all the balls together. Now we’ll describe a sampling procedure which leads to an ordering –i.e. one of the six orderings {RGB,RBG, ...}. This is equivalent to the P-L likelihood for the observed ordering rgb

Plackett-Luce model PL likelihood for a single complete ordering: Picking the item for ranking position k, out of the items that have not yet been ranked. Complete ordering has K factors. More realistically, in ML, we have many items but only some of them have been ranked i.e. Sparse data

Plackett-Luce: vase interpretation Top N Partial orderings Only need as many factors as items ranked for a given data point. PL is internally consistent – marginalising out items leads to the same model form, and the same likelihood as if they were not there. Bradley-Terry model for case of pairs

Luce’s Choice Axiom Probability of choosing wine as favourite drink equals probability of choosing an alcoholic drink as the favourite X probability that wine is the favourite alcoholic drink. Not the same as conditional probablility. P-L is only multi-stage model (choose favourite, then choose next favourite from remaining etc.) whose probabilities satisfy Luce’s choice axiom at every stage.

Gumbel Thurstonian model Each item represented by a score distribution on the real line. Marginal matrix Probability of an item in a position Thustonian model: each item is represented by a score distribution on the real line. Assume a Thurstonian Model with each score having identical distributions except for their means. Then: The score distributions give rise to a Plackett-Luce model if and only the scores are distributed according to a Gumbel distribution (Yellott)

Thurstonian Models, and Yellott’s Theorem Assume a Thurstonian Model with each score having identical distributions except for their means. Then: The score distributions give rise to a Plackett-Luce model if and only the scores are distributed according to a Gumbel distribution (Yellott) Result depends on some nice properties of the Gumbel distribution: Pairwise probability is erf with Gaussian, but logistic with Gumbel. Relation to Plackett Luce: v = exp(mu/beta).

Maximum likelihood estimation Hunter (2004) describes minorize/maximize (MM) algorithm to find MLE Can over-fit with sparse data (especially incomplete rankings) Strong assumption for convergence: “in every possible partition of the items into two nonempty subsets, some item in the second set ranks higher than some item in the first set at least once in the data” MM algorithm is like EM (EM is a special case). The E-step is a specific example of minorization. We’ll show an example of over-fitting later

Bayesian inference: factor graph Gamma priors vA vD vB vC vE B A E D E 5 items. Two data points, incomplete rankings: BAE and DE. Item C not involved in any rankings, so remains as prior.

Fully factored approximation Posterior over P-L parameters, given N orderings : Approximate as fully factorised product of Gammas:

Expectation Propagation [Minka 2001] Iterate until convergence – but only need to do a single pass if tree or chain Marginal is product of incoming messages Computing message from factor to variable requires a sum or integral over factor and variables connected to that factor. Not always tractable. Therefore we use an extension called power EP.

Alpha-divergence Let p,q be two distributions (don’t need to be normalised) Kullback-Leibler (KL) divergence Alpha-divergence ( is any real number) Similarity measures between two probability distributions: p is the truth, and q an approximation

Alpha-divergence – special cases Similarity measures between two distributions (p is the truth, and q an approximation) α Well known special cases. Note p(x) in denominator for alpha = -1

Minimum alpha-divergence q is Gaussian, minimizes D(p||q)  = 0.5  = ∞  = 0  = -∞  = 1

Structure of alpha space inclusive (zero avoiding) zero forcing BP, EP MF  1

Bayesian inference: factor graph Gamma priors vA vD vB vC vE B A E D E A choice of α = -1 leads to a particularly nice simplification for the P-L likelihood Sum of Gammas can be projected back onto single Gamma

Inferring known parameters

Ranking NASCAR drivers Hunter used this example of NASCAR races to illustrate MLE estimation in P-L. 36 Races in NASCAR 2002 season. 83 drivers, but some drivers only competed in a few races, some in all races. Therefore incomplete rankings. Table is ordered by simplest possible ranking method: average place. Show rank given by MLE estimate of P-L, rank given by EP estimate (ordered on means), MLE and EP parameter estimates. Jones, Pruett: only competed in 1 race, but did well. MLE places them first and second. EP places them only 18th, and 19th, and has high uncertainty on P-L parameters. In contrast, EP places Martin at top, who’s competed in all 36 races. Similarly at the bottom end of the scale some drivers such as Morgan Shepherd and Dick Trickle have gone substantially down in the rankings compared to maximum likelihood because there is more evidence that they consistently do badly. And there’s Dick Trickle bringing up the rear......

Posterior rank distributions EP MLE Imagine all drivers did race in one race. Where would you expect them to come? Some visualisations of posterior rank distributions over driver ranks from 1 .. 83. Representation of the P-L distributions over ranks implied by the learnt parameters. Light = high prob Dark = low prob Driver rank : 1 .... 83

Conclusions and future work We have given an efficient Bayesian treatment for P-L models using Power EP Advantage of Bayesian approach is: Avoid over-fitting on sparse data Gives uncertainty information on the parameters Gives estimation of model evidence Future work: Mixture models Feature-based ranking models

Thank you http://www.research.microsoft.com/infernet

Ranking movie genres

Incomplete orderings Internally consistent: “the probability of a particular ordering does not depend on the subset from which the items are assumed to be drawn” Likelihood for an incomplete ordering (only a few items or top-S items are ranked) simple: only include factors for those items that are actually ranked in datum n Suppose we have two sets of items A and B where B in A. This means that the probability of a particular ordering of the items in B, marginalizing over all possible unknown positions of the items left over in A, is exactly the same as the P-L probability of simply ordering those items in B completely independently from A

Power EP for Plackett-Luce A choice of α = -1 leads to a particularly nice simplification for the P-L likelihood An example of the type of calculation in the EP updates, with a factor connecting two items A, E: Sum of Gammas can be projected back onto single Gamma α = -1 power makes this tractable