Download presentation
Presentation is loading. Please wait.
Published byPolly Miller Modified over 9 years ago
1
When Do Noisy Votes Reveal the Truth? Ioannis Caragiannis 1 Ariel D. Procaccia 2 Nisarg Shah 2 ( speaker ) 1 University of Patras & CTI 2 Carnegie Mellon University
2
What? Why? What? Alternatives to be compared True order (unknown ground truth) Noisy estimates (votes) drawn from some distribution around it Q: How many votes are needed to accurately find the true order? Why? Practical motivation Theoretical motivation 2 a > b > c > d b > a > c > d a > c > b > d a > b > d > c Alternatives a, b, c, d
3
Practical Motivation 1. Human Computation EteRNA, Foldit, Crowdsourcing … How many users/workers are required? 2. Judgement Aggregation Jury system, experts ranking restaurants, … How many experts are required?
4
Theoretical Motivation Maximum Likelihood Estimator (MLE) View: Is a given voting rule the MLE for any noise model? Problems Only 1 MLE/noise model Strange noise models Noise model is usually unknown Our Contribution MLE is too stringent! Just want low sample complexity Family of reasonable noise models 4 Voting Rules Noise Models
5
Boring Stuff! 5
6
Sample Complexity for Mallows’ Model 6
7
PM-c and PD-c Rules 7 Pairwise Majority Consistent Rules (PM-c) Must match the pairwise majority graph whenever it is acyclic Condorcet consistency for social welfare functions a b c d
8
PM-c and PD-c Rules 8 PD-c rules similar, but focus on positions of alternatives PM-c PD-c KM SL CP RP SC BL PSR
9
The Big Picture 9 Kemeny rule + uniform tie breaking Optimal sample complexity PM-c PM-c O(log m) (m = #alternatives) Any voting rule Ω(log m) Logarithmic Polynomial Exponential Many scoring rules Plurality, veto Strictly exponential
10
Take-Away - I 10 Given any fixed noise model, sample complexity is a clear and useful criterion for selecting voting rules Hey, what happened to the noise model being unknown?
11
Generalization 11 Stronger need Unknown noise model Working well on a family of reasonable noise models Problems 1.What is reasonable? 2.HUGE sample complexity for near-extreme parameter values! Relaxation Accuracy in the Limit Ground truth with probability 1 given infinitely many samples Novel axiomatic property
12
Accuracy in the Limit 12 Voting RulesNoise models for which they are accurate in the limit PM-c + PD-cMallows’ model (probability decreases exponentially in the KT distance) PM-c + PD-cAll KT-monotonic noise models (probability decreases monotonically in the KT distance) PM-cAll d-monotonic iff d = Majority Concentric (MC) PD-cAll d-monotonic iff d = Position Concentric (PC) PM-c + PD-cAll d-monotonic iff d = both MC and PC Monotonicity is reasonable, but why Kendall-Tau distance?
13
Take-Away - II 13 Robustness accuracy in the limit over a family of reasonable noise models d-monotonic noise models reasonable If you believe in PM-c and PD-c rules look for distances that are both MC and PC Kendall-Tau, footrule, maximum displacement Cayley distance and Hamming distance are neither MC nor PC Even the most popular rule – plurality – is not accurate in the limit for any monotonic noise model over either distance ! Lose just too much information for the true ranking to be recovered
14
Distances over Rankings 14 σ*σ* σ*σ*
15
Discussion 15
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.