Amplification of stochastic advantage

Slides:

Advertisements

Similar presentations

Lecture 18: Temporal-Difference Learning

Advertisements

General Statistics Ch En 475 Unit Operations. Quantifying variables (i.e. answering a question with a number) 1. Directly measure the variable. - referred.

Week11 Parameter, Statistic and Random Samples A parameter is a number that describes the population. It is a fixed number, but in practice we do not know.

POINT ESTIMATION AND INTERVAL ESTIMATION

Ch. 8 – Practical Examples of Confidence Intervals for z, t, p.

Randomized Algorithms Kyomin Jung KAIST Applied Algorithm Lab Jan 12, WSAC

Evaluation (practice). 2 Predicting performance  Assume the estimated error rate is 25%. How close is this to the true error rate?  Depends on the amount.

Evaluating Hypotheses

Lehrstuhl für Informatik 2 Gabriella Kókai: Maschine Learning 1 Evaluating Hypotheses.

A) Transformation method (for continuous distributions) U(0,1) : uniform distribution f(x) : arbitrary distribution f(x) dx = U(0,1)(u) du When inverse.

Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement8-1 Additional real-life examples (proportions) Supplement 8: Additional real-life examples.

Chapter 7 Estimation: Single Population

Introduction to Data Analysis Probability Distributions.

STA Lecture 161 STA 291 Lecture 16 Normal distributions: ( mean and SD ) use table or web page. The sampling distribution of and are both (approximately)

Confidence Intervals 1 Chapter 6. Chapter Outline Confidence Intervals for the Mean (Large Samples) 6.2 Confidence Intervals for the Mean (Small.

Prabhas Chongstitvatana 1 Primality Testing Is a given odd integer prime or composite ? No known algorithm can solve this problem with certainty in a reasonable.

Chapter 5 Discrete Probability Distributions

1 G Lect 4a G Lecture 4a f(X) of special interest: Normal Distribution Are These Random Variables Normally Distributed? Probability Statements.

Estimation (Point Estimation)

Amplification of stochastic advantage

General Statistics Ch En 475 Unit Operations. Quantifying variables (i.e. answering a question with a number) 1. Directly measure the variable. - referred.

Sampling Theory The procedure for drawing a random sample a distribution is that numbers 1, 2, … are assigned to the elements of the distribution and tables.

Machine Learning Chapter 5. Evaluating Hypotheses

Prabhas Chongstitvatana1 Las Vegas algorithm The answer obtained is always correct but sometime no answer. Modified deterministic algorithm by using randomness.

The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch.

Chapter 8: Confidence Intervals based on a Single Sample

Prabhas Chongstitvatana1 Monte Carlo integration It is a numerical probabilistic algorithm ab I/(b-a) f.

1 Mean Analysis. 2 Introduction l If we use sample mean (the mean of the sample) to approximate the population mean (the mean of the population), errors.

ICS 353: Design and Analysis of Algorithms

Introduction to Probability – Experimental Probability.

Prabhas Chongstitvatana1 Numerical probabilistic The answer is always an approximation.

Chapter Outline 6.1 Confidence Intervals for the Mean (Large Samples) 6.2 Confidence Intervals for the Mean (Small Samples) 6.3 Confidence Intervals for.

Chapter 6 Large Random Samples Weiqi Luo ( 骆伟祺 ) School of Data & Computer Science Sun Yat-Sen University ：

Chapter 8 Estimation ©. Estimator and Estimate estimator estimate An estimator of a population parameter is a random variable that depends on the sample.

Evaluating Hypotheses. Outline Empirically evaluating the accuracy of hypotheses is fundamental to machine learning – How well does this estimate its.

Rodney Nielsen Many of these slides were adapted from: I. H. Witten, E. Frank and M. A. Hall Data Science Credibility: Evaluating What’s Been Learned Predicting.

Random Variables Lecture Lecturer : FATEN AL-HUSSAIN.

1. 2 At the end of the lesson, students will be able to (c)Understand the Binomial distribution B(n,p) (d) find the mean and variance of Binomial distribution.

Evaluating Hypotheses. Outline Empirically evaluating the accuracy of hypotheses is fundamental to machine learning – How well does this estimate accuracy.

Statistics for Business and Economics 7 th Edition Chapter 7 Estimation: Single Population Copyright © 2010 Pearson Education, Inc. Publishing as Prentice.

Chapter 19 Monte Carlo Valuation. © 2013 Pearson Education, Inc., publishing as Prentice Hall. All rights reserved.19-2 Monte Carlo Valuation Simulation.

usually unimportant in social surveys:

Statistical Estimation

STATISTICAL INFERENCE

Nature of Estimation.

Chapter 19: Unbiased estimators

Point and interval estimations of parameters of the normally up-diffused sign. Concept of statistical evaluation.

Randomized Algorithm (Lecture 2: Randomized Min_Cut)

Lecture 8 Randomized Algorithms

CPSC 531: System Modeling and Simulation

Statistical Methods Carey Williamson Department of Computer Science

Estimating Population Variance

Prabhas Chongstitvatana

CONCEPTS OF ESTIMATION

Numerical probabilistic

Chapter 6 Confidence Intervals.

Evaluating Hypotheses

6.2/6.3 Probability Distributions and Distribution Mean

Sampling Distributions

Probabilistic algorithms

Carey Williamson Department of Computer Science University of Calgary

Monte Carlo integration

9. Limit Theorems.

Probabilistic algorithms

Chapter 6 Confidence Intervals.

Chapter 8 Estimation.

STA 291 Summer 2008 Lecture 12 Dustin Lueker.

STA 291 Spring 2008 Lecture 12 Dustin Lueker.

CS 160: Lecture 16 Professor John Canny 8/2/2019.

Presentation transcript:

Amplification of stochastic advantage Biased : known with certainty one of the possible answer is always correct. Error can be reduced by repeat the algorithm. Unbiased example coin flip for p-correct “advantage” is p - 1/2 Prabhas Chongstitvatana

Prabhas Chongstitvatana Let MC be a 3/4-correct unbiased What is the error prob. Of MC3 (mojority vote) ? MC3 is 27/32-correct ( > 84% ) Prabhas Chongstitvatana

Prabhas Chongstitvatana What is the error prob. of MC with advantage e > 0 in majority vote k times ? Let Xi = 1 if correct answer, 0 otherwise Pr[ Xi = 1 ] >= 1/2 + e assume for simplicity Pr[ Xi = 1 ] = 1/2 + e ; k is odd (no tie) E( Xi ) = 1/2 + e; Var(Xi ) = (1/2 + e) (1/2-e) = 1/4 - e2 Prabhas Chongstitvatana

Prabhas Chongstitvatana Let X is a random variable corresponds to the number of correct answer is k trials. E(X) = (1/2+e)k Var(X) = (1/4 - e2) k Prabhas Chongstitvatana

Prabhas Chongstitvatana error prob. Pr [ X <= k/2 ] can be calculated is normal distributed if k >= 30 Prabhas Chongstitvatana

Prabhas Chongstitvatana If need error < 5% Pr [ X < E(X) - 1.645 sqrt Var(X) ] ~ 5% (from the table of normal distribution) Pr [ X <= k/2 ] < 5% if k/2 < E(X) - 1.645 sqrt Var(X) k > 2.706 ( 1/(4e2) - 1 ) Prabhas Chongstitvatana

Prabhas Chongstitvatana Example e = 5%, which is 55%-correct unbiased Monte Carlo, k > 2.706 ( 1/(4e2) - 1 ) k > 267.894, majority vote repeat 269 times to obtain 95%-correct. Repetition turn 5% advantage into 5% error prob Prabhas Chongstitvatana

Prabhas Chongstitvatana It takes a large number of run for unbiased Compare to biased Run 55%-correct bias MC 4 times reduces the error prob. To 0.454 ~ 0.041 ( 4.1% ) Prabhas Chongstitvatana

Prabhas Chongstitvatana Not much more expensive to obtain more confidence. If we want 0.5% confidence (10 times more) Pr[X < E(X) - 2.576 sqrt Var(X) ] ~ 5% k > 6.636 (1/(4e2) - 1 ) This makes it 99.5%-correct with less than 2.5 times more expensive than 95%-correct. Prabhas Chongstitvatana

Prabhas Chongstitvatana To reduce error prob < del for an unbiased Monte Carlo with advantage e; the number of repetition is proportional to 1/e2, also to log 1/del Prabhas Chongstitvatana