1. Daniela Stan Raicu School of CTI, DePaul University
CSC 323 Quarter: Winter 02/03
Daniela Stan Raicu
School of CTI, DePaul University
11/15/2018
Daniela Stan - CSC323

2. Outline Chapter 4: Probability – The Study of Randomness
The Law of Large Numbers
Random Variables
Means and Variances of Random Variables
11/15/2018
Daniela Stan - CSC323

3. The Law of Large Numbers
Probability describes the long-term proportion with which a certain
outcome will occur in situations with short-term uncertainty.
Probability is a measure of the likelihood of a random phenomenon or a chance behavior.
Example:
Flip a coin 1000 times and compute the proportion of heads observed after each toss of the coin.
As the number of flips of the coin increases, the graph tends toward a proportion of 0.5
Proportion of heads: 1/1, 1/2, 1/3, 1/4, 3/5, 4/6,…, 504/998, 503/999, 501/1000
11/15/2018
Daniela Stan - CSC323

4. The Law of Large Numbers
Therefore, we say that the probability of observing a head is ½ or 50% because as the number of repetitions of the experiment increases, the proportion of the heads tends towards ½. The phenomenon is referred to as the Law of Large Numbers.
The law of large numbers is about regularity in the long run and forms the foundation of gambling casinos and insurance companies.
For instance, at the craps table the chance of winning the “don’t pass bet” is 49.29% making it almost a fair game. Some people win and some people lose, but on average the casino takes in 1.4cents per dollar bet. In the long run, as tens of thousands of people play, this 1.4 cents per bet is as predictable as a paycheck.
In the same way an insurance company balances the risk of a huge payout by collecting many small premiums. The regularity over a large group of customers makes up for the occasional unpredictable disaster.
Other examples: Internet Server Providers use the law of large numbers for resources
allocation
11/15/2018
Daniela Stan - CSC323

5. Random Variable
“The longer a random process is repeated under the same conditions, the closer the observed proportion of each outcome occurrence is to the actual probability of occurrence.”
A convenient way of representing a random phenomenon is through a random variable.
We can associate a variable to each random process. The values of such a variable are the possible outcomes of the random process.
11/15/2018
Daniela Stan - CSC323

6. Random variables
For instance X = number of heads in 4 tosses of a coin. These are the possible outcomes:
HHTT
HTHT
HTTT
HTTH
HHHT
THTT
THHT
HHTH
TTHT
THTH
HTHH
TTTT
TTTH
TTHH
THHH
HHHH
X=0
X=1
X=2
X=3
X=4
A probability value can be associated to each value of X.
11/15/2018
Daniela Stan - CSC323

7. If X=0, then no head comes up, so the probability is 1/16
HHTT
HTHT
HTTT
HTTH
HHHT
THTT
THHT
HHTH
TTHT
THTH
HTHH
TTTT
TTTH
TTHH
THHH
HHHH
X=0
X=1
X=2
X=3
X=4
If X=0, then no head comes up, so the probability is 1/16
If X=1, then only one head come up, the probability is 4/16
If X=2, then 2 heads come up, the probability is 6/16
If X=3, then 3 heads come up, the probability is 4/16
If X=4, then 4 heads come up, the probability is 1/16 X 1 2 3 4
Probability
0.0625
0.25
0.375
11/15/2018
Daniela Stan - CSC323

8. Probability Histograms
The probability table associated to a random process or a random variable can be displayed as a probability histogram.
For example the probability histogram of the number of heads in 4 tosses of a coin is displayed below: X 1 2 3 4
Probability
0.0625
0.25
0.375
Chance 40
(%) 30 20 10
The number of heads in 4 tosses of a coin would be a number around 2.
11/15/2018
Daniela Stan - CSC323

9. Expected value & standard error of a random variable
Let us assume that we flip a coin for 100 tosses and we expect to get 50 heads; however, we might get 57 heads, which is 7 heads above the expected value of 50.
Toss the coin 100 times again, you might get 55, which is 5 heads above 50
Again… you might get 48, which is 2 heads below 50….and so on….
The numbers delivered by the process vary around the expected value, the amount off being similar in size to the standard error.
Thus in 100 tosses the expected value for the number of heads is 50. The standard error is a measure of the chance error.
We will now define these two quantities:
Expected value
Standard error
11/15/2018
Daniela Stan - CSC323

10. The Expected Value
The expected value of the number of heads in 4 tosses of a coin is 2. The expected value of the number of heads in 100 tosses of a coin is 50.
In statistical terms:
It is calculated by multiplying each possible value by its probability, then adding all the products.
number of heads in 4 tosses X 1 2 3 4
Probability
0.0625
0.25
0.375
Expected value of X= 0* *0.25+2* *0.25+4*0.0625=
= = 2
11/15/2018
Daniela Stan - CSC323

11. The expected value
The expected value is the mean (average) of the probability histogram.
Ex: The expected value of the number of heads in 4 tosses of a coin is 2.
Chance 40
(%) 30 20 10
X=2
11/15/2018
Daniela Stan - CSC323

12. Standard Error
The standard error measures the spread of the probability histogram.
The standard error of the number of heads in 4 tosses of a coin is 1.
Chance 40
(%) 30 20 10
X–1s.e.=2 – 1=1
X+1s.e.=2+1=3
1 s.e.
1 s.e.
X=2
Remark: Observed values are rarely more than 2 or 3 standard errors away!!
11/15/2018
Daniela Stan - CSC323

13. Mathematical Expressions
Given a random variable X with probability table X x1 x2 x3 x4 … xk
Probability p1 p2 p3 p4 pk
The expected value is
The standard error is
11/15/2018
Daniela Stan - CSC323

14. Expected payoff in gambling
A game is fair if the expected value for the net gain equals 0: on the average players neither win or lose.
Keno: In the game Keno, there are 80 balls, numbered 1 to 80. On each play, the casino chooses 20 balls at random. Suppose you bet $1 on 17 in each Keno play. When you win, the casino gives you your dollar back and 2 dollars more
When you lose, the casino keeps your dollar. The bet pays 3 to 1.
Is the bet fair?
Event
Probability
X=+3
You win, 17 is among the 20 balls
20/80=0.25
X=–1
You lose, 17 is not among 20 balls
60/80=0.75
Create the random variable:
The expected value of X is –1*0.75+3*0.25=0. The game is fair!
11/15/2018
Daniela Stan - CSC323

15. Remarks on random processes
An observed value should be somewhere around the expected value; the difference is chance error.
The likely size of the chance error is the standard error.
Observed values are rarely two or three standard errors away from the expected value.
The standard error is defined for random processes and measures the chance error. (Subtle difference)
The standard error “makes more sense” if the probability histogram of the random variable is bell-shaped, (similar to the normal distribution).
11/15/2018
Daniela Stan - CSC323

16. Recommended problems Problems 4.60, 4.61/page 334
11/15/2018
Daniela Stan - CSC323