Daniela Stan Raicu School of CTI, DePaul University CSC 323 Quarter: Winter 02/03 Daniela Stan Raicu School of CTI, DePaul University 11/15/2018 Daniela Stan - CSC323

Outline Chapter 4: Probability – The Study of Randomness The Law of Large Numbers Random Variables Means and Variances of Random Variables 11/15/2018 Daniela Stan - CSC323

The Law of Large Numbers Probability describes the long-term proportion with which a certain outcome will occur in situations with short-term uncertainty. Probability is a measure of the likelihood of a random phenomenon or a chance behavior. Example: Flip a coin 1000 times and compute the proportion of heads observed after each toss of the coin. As the number of flips of the coin increases, the graph tends toward a proportion of 0.5 Proportion of heads: 1/1, 1/2, 1/3, 1/4, 3/5, 4/6,…, 504/998, 503/999, 501/1000 11/15/2018 Daniela Stan - CSC323

The Law of Large Numbers Therefore, we say that the probability of observing a head is ½ or 50% because as the number of repetitions of the experiment increases, the proportion of the heads tends towards ½. The phenomenon is referred to as the Law of Large Numbers. The law of large numbers is about regularity in the long run and forms the foundation of gambling casinos and insurance companies. For instance, at the craps table the chance of winning the “don’t pass bet” is 49.29% making it almost a fair game. Some people win and some people lose, but on average the casino takes in 1.4cents per dollar bet. In the long run, as tens of thousands of people play, this 1.4 cents per bet is as predictable as a paycheck. In the same way an insurance company balances the risk of a huge payout by collecting many small premiums. The regularity over a large group of customers makes up for the occasional unpredictable disaster. Other examples: Internet Server Providers use the law of large numbers for resources allocation 11/15/2018 Daniela Stan - CSC323

Random Variable “The longer a random process is repeated under the same conditions, the closer the observed proportion of each outcome occurrence is to the actual probability of occurrence.” A convenient way of representing a random phenomenon is through a random variable. We can associate a variable to each random process. The values of such a variable are the possible outcomes of the random process. 11/15/2018 Daniela Stan - CSC323

Random variables For instance X = number of heads in 4 tosses of a coin. These are the possible outcomes: HHTT HTHT HTTT HTTH HHHT THTT THHT HHTH TTHT THTH HTHH TTTT TTTH TTHH THHH HHHH X=0 X=1 X=2 X=3 X=4 A probability value can be associated to each value of X. 11/15/2018 Daniela Stan - CSC323

If X=0, then no head comes up, so the probability is 1/16 HHTT HTHT HTTT HTTH HHHT THTT THHT HHTH TTHT THTH HTHH TTTT TTTH TTHH THHH HHHH X=0 X=1 X=2 X=3 X=4 If X=0, then no head comes up, so the probability is 1/16 If X=1, then only one head come up, the probability is 4/16 If X=2, then 2 heads come up, the probability is 6/16 If X=3, then 3 heads come up, the probability is 4/16 If X=4, then 4 heads come up, the probability is 1/16 X 1 2 3 4 Probability 0.0625 0.25 0.375 11/15/2018 Daniela Stan - CSC323

Probability Histograms The probability table associated to a random process or a random variable can be displayed as a probability histogram. For example the probability histogram of the number of heads in 4 tosses of a coin is displayed below: X 1 2 3 4 Probability 0.0625 0.25 0.375 0 1 2 3 4 Chance 40 (%) 30 20 10 The number of heads in 4 tosses of a coin would be a number around 2. 11/15/2018 Daniela Stan - CSC323

Expected value & standard error of a random variable Let us assume that we flip a coin for 100 tosses and we expect to get 50 heads; however, we might get 57 heads, which is 7 heads above the expected value of 50. Toss the coin 100 times again, you might get 55, which is 5 heads above 50 Again… you might get 48, which is 2 heads below 50….and so on…. The numbers delivered by the process vary around the expected value, the amount off being similar in size to the standard error. Thus in 100 tosses the expected value for the number of heads is 50. The standard error is a measure of the chance error. We will now define these two quantities: Expected value Standard error 11/15/2018 Daniela Stan - CSC323

The Expected Value The expected value of the number of heads in 4 tosses of a coin is 2. The expected value of the number of heads in 100 tosses of a coin is 50. In statistical terms: It is calculated by multiplying each possible value by its probability, then adding all the products. number of heads in 4 tosses X 1 2 3 4 Probability 0.0625 0.25 0.375 Expected value of X= 0*0.0625+1*0.25+2*0.375+3*0.25+4*0.0625= = 0 + 0.25 + 0.75 +0.75 + 0.25 = 2 11/15/2018 Daniela Stan - CSC323

The expected value The expected value is the mean (average) of the probability histogram. Ex: The expected value of the number of heads in 4 tosses of a coin is 2. 0 1 2 3 4 Chance 40 (%) 30 20 10 X=2 11/15/2018 Daniela Stan - CSC323

Standard Error The standard error measures the spread of the probability histogram. The standard error of the number of heads in 4 tosses of a coin is 1. 0 1 2 3 4 Chance 40 (%) 30 20 10 X–1s.e.=2 – 1=1 X+1s.e.=2+1=3 1 s.e. 1 s.e. X=2 Remark: Observed values are rarely more than 2 or 3 standard errors away!! 11/15/2018 Daniela Stan - CSC323

Mathematical Expressions Given a random variable X with probability table X x1 x2 x3 x4 … xk Probability p1 p2 p3 p4 pk The expected value is The standard error is 11/15/2018 Daniela Stan - CSC323

Expected payoff in gambling A game is fair if the expected value for the net gain equals 0: on the average players neither win or lose. Keno: In the game Keno, there are 80 balls, numbered 1 to 80. On each play, the casino chooses 20 balls at random. Suppose you bet $1 on 17 in each Keno play. When you win, the casino gives you your dollar back and 2 dollars more When you lose, the casino keeps your dollar. The bet pays 3 to 1. Is the bet fair? Event Probability X=+3 You win, 17 is among the 20 balls 20/80=0.25 X=–1 You lose, 17 is not among 20 balls 60/80=0.75 Create the random variable: The expected value of X is –1*0.75+3*0.25=0. The game is fair! 11/15/2018 Daniela Stan - CSC323

Remarks on random processes An observed value should be somewhere around the expected value; the difference is chance error. The likely size of the chance error is the standard error. Observed values are rarely two or three standard errors away from the expected value. The standard error is defined for random processes and measures the chance error. (Subtle difference) The standard error “makes more sense” if the probability histogram of the random variable is bell-shaped, (similar to the normal distribution). 11/15/2018 Daniela Stan - CSC323

Recommended problems Problems 4.60, 4.61/page 334 11/15/2018 Daniela Stan - CSC323