MULTIDIMENSIONAL RANKING Ashish Das Indian Institute of Technology Bombay India

Outline The Multidimensional Ranking Problem Applications Desired properties Ranking Methods with Illustrations Extensions and Modifications

The Multidimensional Ranking Problem m candidates (or “alternatives”) M = {1,…,m}: set of candidates n voters (or “agents” or “judges”) N = {1,…,n}: set of voters Each voter i, has an ranking i on M i(a) < i(b) means i-th voter prefers a to b The rank aggregation problem: Combine 1,…,n into a single ranking  on M, which represents the “social choice” of the voters. Rank aggregation function: f(1,…,n) = 

WHAT IS MULTIDIMENSIONAL RANKING? Multidimensional Ranking is an algorithm for comprehensive ranking based on more than one variable or opinion. The aim of multidimensional ranking is to combine many different rank orderings on a set of variables, in order to obtain a better ordering. It provides an aggregate or over-all measure by multi criteria decision making or data integration methods.

APPLICATIONS Meta search: Combine results of different web search engines into a better overall ranking Rank items in a database according to multiple criteria Ex: Choose a restaurant by cuisine, distance, price, quality, etc. Ex: Choose a flight ticket by price, # of stops, date and time, frequent flier bonuses, etc. Overall ranking of various mid-size cars available in the market Overall ranking of Banks Ranking service quality of mobile cell providers Overall ranking of universities Compare overall rankings over time

Desired Properties: Unanimity Unanimity (or Pareto optimality): If all voters prefer candidate a to candidate b (i.e., i(a) < i(b) for all i), then also  should prefer a to b (i.e., (a) < (b)). a c b a:b = 3:0

Desired Properties: Condorcet Condorcet Criterion [Condorcet, 1785]: Condorcet winner: a candidate a, which is preferred by most voters to any other candidate b (i.e., for all b, # of i s.t. i(a) < i(b) is at least n/2). Condorcet criterion: If Condorcet winner exists,  should rank it first (i.e., (a) = 1). c b a c b a a:b = 2:1, a:c = 2:1 No Condorcet winner

Desired Properties: XCC Extended Condorcet Criterion (XCC): If most voters prefer candidate a to candidate b (i.e., # of i s.t. i(a) < i(b) is at least n/2), then also  should prefer a to b (i.e., (a) < (b)). Not always realizable c b a c b a (a) < (b) < (c) Not realizable

Desired Properties: Neutrality and Anonymity No candidate should be favored to others. If two candidates switch positions in 1,…,n, they should switch positions also in . Anonymity No voter should be favored to others. If two voters switch their orderings,  should remain the same.

Desired Properties: Monotonicity and Consistency If the ranking of a candidate is improved by a voter, its ranking in  can only improve. Consistency If voters are split into two disjoint sets, S and T, and both the aggregation of voters in S and the aggregation of voters in T prefer a to b, then also the aggregation of all voters should prefer a to b.

METHODS For finding the best option from all the feasible alternatives, we use some sound approaches to aggregate or rank with respect to more than one attributes. A. Positional Rank Aggregation Methods B. TOPSIS METHOD (Technique for Order Preference by Similarity to an Ideal Solution) C. KEMENEY’S METHOD

Positional Rank Aggregation Methods Plurality score(a) = # of voters who chose a as #1 : order candidates by decreasing scores Top-k approval score(a) = # of voters who chose a as one of the top k Borda’s rule [Borda, 1781] score(a) = i i(a) : order candidates by increasing scores

Positional Methods: Example b c a d Borda Top-2 Approval Plurality 1+1+4+4=10 2 a 2+4+2+1=9 3 1 b 3+3+1+3=10 c 4+2+3+2=11 d

The TOPSIS method TOPSIS [Hwang, C.L. and Yoon, K. , 1981], compares each state or candidate with respect to positive and negative ideal solution Composite ranking ranks that alternative the best that has shortest distance from positive ideal solution & longest from the negative ideal solution The method induces an ordering of the solutions based on similarity to the ideal point, guiding the search towards the zone of interest

The TOPSIS method The TOPSIS distances I- f2min dz- az dj- dz+ aj dj+

Steps in TOPSIS computation Suppose S1, S2,…, Sm are m possible alternatives among which decision makers have to choose based on n criteria. Let, C1, C2 ,…, Cn are the criteria with which alternative performance are measured, Xij is the rating of alternative Si with respect to the criterion Cj. Thus, the data matrix is,

Household Consumer Expenditure in India 2006-07

Variables selected Average Monthly Per Capita Expenditure Average per person consumption of non-cereals in total food consumption (%) Average expenditure per person per 30 days on consumption of non-food articles (Rs.) Per 1000 number of hh who use LPG as main source for cooking Per 1000 number of households who use electricity as primary source of energy for lighting Per 1000 number of persons aged 7 & above who are literate Per 1000 number of persons aged 7 & above whose level of education is Higher Secondary or above Average covered area of dwelling unit (sq. m) per household Per 1000 no. of hh who do not stay in self owned dwelling unit

TOPSIS RANK TABLE

Rural Data After Ranking

TOPSIS RANK TABLE (RANKED DATA)

Change in Rank Orderings

Optimal Rank Aggregation d: distance measure among rankings Definition: The optimal rank aggregation for 1,…,n w.r.t. d is the ranking  which minimizes i d(,i). 1 n 2 

Distance Measures Kendall tau distance (or “bubble sort distance”) K(,) = # of pairs of candidates (a,b) on which  and  disagree Ex: K( (a b c d), (a d c b)) = 0 + 2 + 1 = 3 Spearman footrule distance F(,) = a |(a) - (a)| Ex: F((a b c d), (a d c b)) = 0 + 2 + 0 + 2 = 4

Kemeny Optimal Aggregation [Kemeny 1959] Optimal aggregation w.r.t. Kendall-tau distance (NP-hard even for small n) K-distance. Let π and σ be two partial lists of {1,…, m}. The K-distance of π and σ, denoted K(π, σ), is the number of pairs i,j ε {1, …, m} such that π(i) < π(j) but σ(i)>σ(j). Note that if it is not the case that both i and j appear in both lists π and σ, then the pair (i,j) contributes nothing to the K-distance between the two lists. For any two partial lists K(π, σ) = K(σ, π). SK, Kemeny optimal. For a collection of partial lists τ1, τ2,…, τn and a full list π we denote SK(π, τ1, τ2,…, τn) = Σ K(π, τi). We say that a permutation σ is a Kemeny optimal aggregation of a collection of partial lists τ1, τ2,…, τn if it minimizes SK(π, τ1, τ2,…, τn) over all permutations π. Theorem [Young & Levenglick, 1978] [Truchon 1998]: Kemeny optimal aggregation is the only rank aggregation function, which is neutral, consistent, and satisfies the Extended Condorcet principle.

APPLICATION PROCEDURE In Kemeny’s method one finds an optimal ranking on the basis of some given condition (variables). We illustrate through the same data set used in TOPSIS method. We apply KEMENY’S method in two different ways: Direct application of KEMENY’S method Stepwise application of KEMENY’S method

DIRECT APPLICATION OF KEMENY’S METHOD Step1: We first decide on the number of states for applying KEMENY. Suppose we choose m=4 rows (states A=1, B=2, C=3, D=4). We work out the 4! = 24 permutations of 1,2,3,4. A permutation 2314 means that the ranking is BCAD. Step2: We incorporate our data matrix as 4 rows and n = 9 (say) columns. Then we construct the rank (position) matrix involving ranking of the states with respect to each of the 9 variables. Step3: We calculate the number of non-matching pairs between each one of the permutations of 1,2,3,4 and position orderings for each of the 9 variables. Step4: We next calculate the K-distance for each of the permutations. K- distances are defined as the number of non-matching pairs between a permutations and position orderings for each of the 9 variables.

EXAMPLE   Variables -> 1 2 3 4 5 6 7 8 9 A=1 Bihar 541.3 0.57 218.3 14 113 522 39 38 B=2 Orissa 458.6 0.56 193.3 25 302 600 40 47.3 21 C=3 West Bengal 629.9 0.62 258.8 31 373 710 38.3 D=4 Madhya Pradesh 514.9 0.67 251.1 660 611 47.5 19 Rank Matrix 3 2 2 1 1 1 1 1 1 1 1 1 3 2 2 4 3 3 4 3 4 4 3 4 4 2 4 2 4 3 2 4 3 4 4 2 Position Matrix 2 2 2 1 1 1 1 1 1 4 1 1 4 2 2 4 3 4 1 3 4 2 3 4 4 2 2 3 4 3 3 4 3 4 4 3

EXAMPLE 2 4 1 3 Position Matrix S.No. 1 4 3 2 1 2 4 3 1 2 3 4 2 3 1 4 Permutations 1 4 3 2 1 2 4 3 1 2 3 4 2 3 1 4 4 2 1 3 5 4 1 2 3 6 4 1 3 2 7 3 4 2 1 8 3 4 1 2 9 3 2 4 1 10 3 2 1 4 11 3 1 2 4 12 3 1 4 2 13 2 3 4 1 14 2 3 1 4 15 2 4 3 1 16 2 4 1 3 17 2 1 4 3 18 2 1 3 4 19 1 3 2 4 20 1 3 4 2 21 1 2 3 4 22 1 2 4 3 23 1 4 2 3 24 1 4 3 2 Counting K-distance 3 5 4 4 6 5 6 5 4 42 4 6 5 3 5 4 6 4 3 40 2 4 3 3 5 4 6 6 3 36 1 3 2 2 4 3 6 5 2 28 2 4 3 1 3 2 6 4 1 26 3 5 4 2 4 3 6 3 2 32 4 4 5 5 5 6 6 4 5 44 5 5 6 4 4 5 6 3 4 3 3 4 6 4 5 6 3 6 4 2 3 5 3 4 5 2 5 33 5 3 4 4 2 3 5 1 4 31 6 4 5 3 3 4 5 2 3 35 2 2 3 5 3 4 6 4 5 34 3 1 2 4 2 3 5 3 4 27 1 3 2 4 4 3 6 5 4 0 2 1 3 3 2 6 4 3 24 1 1 0 2 2 1 5 3 2 17 2 0 1 3 1 2 5 2 3 19 4 2 3 3 1 2 5 0 3 23 5 3 4 2 2 3 5 1 2 3 1 2 2 0 1 5 1 2 2 2 1 1 1 0 5 2 1 15 3 3 2 0 2 1 5 3 0 4 4 3 1 3 2 5 2 1 25 Position Matrix 2 4 1 3

STEPWISE APPLICATION OF KEMENY’S METHOD Stepwise application of KEMENEY’S method is a new concept of applying the basic KEMENEY’S method. It is very useful for large data set, say for example m=26. Here the direct application of Kemeny is not feasible since there are 26! Combinations to look into. We introduce the concept of stepwise Kemeny. Step1: We have to first fix the number of row (= k) to be taken at a time (say, 3, or 4, or 5, …). After fixing this, we proceed as in Step1 (of Kemeny) taking m = k. Step2: We then incorporate our data matrix as 26 rows and 9 columns (say). Then we recursively choose our inputs from original data matrix. We choose first k (say 6) rows, and then construct rank matrix and position rank matrix as in Step 2 of Kemeny. Step3: We then proceed to Steps 3 and 4 of Kemeny based on these first 6 rows. The optimal Kemeny, thus obtained, is used to identify the Rank 1 state. In the next iteration we remove the Rank 1 state and add the 7th state in order to make a new set of 6 row. This process continues to identify Rank 2 state, Rank 3 state and so on Rank 20 state. The ranking in the final iteration gives Rank 21 through 26. Step4: Repeat the process till the rankings stabilize.

PERFORMANCE OF MODIFIED KEMENY 1 2 3 4 5 6 7 In Kemeny-2 it took 2 iterations to stabilize

COMPARISON BETWEEN TOPSIS & KEMENY METHODS We have described 2 different approaches for TOPSIS method and KEMENY method. We compare the optimum results for each of the above methods. As direct application of KEMENY method is not feasible for a large of data set, for illustration, we take a small data set.

EXAMPLE(SMALL DATA SET) Result based on original TOPSIS Result based on Ranked TOPSIS Result based on KEMENY

EXAMPLE(LARGE DATA SET)