1. Lecture 9 5.3 Discrete Probability

2. 5.3 Bayes’ Theorem We have seen that the following holds: We can write one conditional probability in terms of the other: Bayes’ Theorem

3. 5.3 Example: What is the probability that a family with 2 kids has two boys, given that they have at least one boy? (all possibilities are equally likely).  S: all possibilities: {BB, GB, BG, GG}. E: family has two boys: {BB}. F: family has at least one boy: {BB, GB, BG}. E F = {BB} p(E|F) = (1/4) / (3/4) = 1/3 E BB F GB BG GG Now we compute the probability of P(F|E), what is the probability that a family with two boys has at least one boy ? P(F|E) = P(E|F) P(F) / P(E) = 1/3 * ¾ / ¼ = 1

4. 5.3 Expected Values The definition of an expected value of a random variable is: This equivalent to: Example: What is the expected number of heads if we toss a fair coin n times?  We know that the distribution for this experiment is the Binomial distribution:

5. 5.3 Therefore we need to compute:

6. 5.3 Expectation are linear: Theorem: E(X1+X2) = E(X1) + E(X2) E(aX + b) = aE(X) + b Examples: 1) Expected value for the sum of the values when a pair of dice is rolled:  X1 = value of first die, X2 value of second die: E(X1+X2) = E(X1) + E(X2) = 2 * (1+2+3+4+5+6)/6 = 7. 2) Expected number of heads when a fair coin is tossed n times (see example previous slide)  Xi is the outcome coin toss i. Each has a probability of p of coming up heads. linearity: E(X1+...+Xn) = E(X1) +... + E(xn) = n p.

7. 5.3 More examples: A person checking out coats mixed the labels up randomly. When someone collects his coat, he checks out a coat randomly from the remaining coats. What is the expected number of correctly returned coats? There are n coats checked in.  Xi = 1 of correctly returned, and 0 if wrongly returned. Since the labels are randomly permuted, E(Xi) = 1/n E(X1+...Xn) = n 1/n = 1 (independent of the number of checked in coats)

8. 5.3 Geometric distribution Q: What is the distribution of waiting times until a tail comes up, when we toss a fair coin? A: Possible outcomes: T, HT, HHT, HHHT, HHHHT,.... (infinitely many possibilities) P(T) = p, P(HT) = (1-p) p, P(HHT) = (1-p)^2 p,.... geometric distribution Normalization: (matlab) X(s) = number of tosses before success.

9. 5.3 Geometric Distr. Here is how you can compute the expected value of the waiting time:

10. 5.3 Independence Definition: Two random variables X(s) and Y(s) on a sample space S are independent if the following holds: Examples 1) Pair of dice is rolled. X1 is value first die, X2 value second die. Are these independent?  P(x1=r1) = 1/6 P(X2=r2)=1/6 P(X1=r1 AND X2=r2)=1/36 = P(X1=r1) P(X2=r2):  YES independent. 2) Are X1 and X=X1+X2 independent?  P(X=12) =1/36 P(X1=1)=1/6 P(X=12 AND X1=1)=0 which is not the product: P(X=12) P(X1=1)

11. 5.3 Independence Theorem: If two random variables X and Y are independent over a sample space S then: E(XY)=E(X) E(Y). (proof, read book) Note1: The reverse is not true: Two random variables do not have to be independent for E(XY)=E(X)E(Y) to hold. Note2: If 2 random variables are not independent, it follows that E(XY) does not have to be equal to E(X)E(Y), although it might still happen. Example: X counts number of heads when a coin is tossed twice: P(X=0) =1/4 (TT) P(X=1)=1/2 (HT,TH) P(X=2) =1/4 (HH). E(X) = 1x½+2x1/4=1. Y counts the number of tails: E(Y)=1 as well (symmetry, switch role H,T). However, P(XY=0) = 1/2 (HH,TT) P(XY=1) =1/2 (HT,TH) E(XY) = 0x1/2 + 1x1/2=1/2

12. 5.3 Variance The average of a random variable tells us noting about the spread of a probability distribution. (matlab demo) Thus we introduce the variance of a probability distribution: Definition: The variance of a random variable X over a sample space S is given by: variance standard deviation (this is the width of the distribution)

13. 5.3 Variance Theorem: For independent random variables the variances add: (proof in book) Example: 1) We toss 2 coins, Xi(H)=1, Xi(T)=0. What is the STD of X=X1+X2?  X1 and X2 are independent. V(X1+X2)=V(X1)+V(X2)=2V(X1) E(X1)=1/2 V(X1) = (0-1/2)^2 x ½ + (1-1/2)^2 x ½ =1/4 V(X) = ½ STD(X)=sqrt(1/2).

14. 5.3 Variance What is the variance of the number of successes when n independent Bernoulli trials are performed. V(X) = V(X1+...+Xn)=nV(X1) V(X1) = (0-p)^2 x (1-p) + (1-p)^2 x p = p^2(1-p) + p(1-p)^2=p(1-p) V(X)=np(1-p) (matlab demo)

15. 5.3 Chebyshev’s Inequality Theorem: X is random variable on sample space S, and P(X=r) it’s probability distribution. Then for any positive real number r: (proof in book) In words: the probability of finding a value of X farther away from the mean than r is smaller than the variance divided by r^2. r

16. 5.3 Example: What is the probability that with 100 Bernoulli trials we find more than 89 or less than 11 successes when the prob. of success is ½. X counts number of successes. EX=100 x ½ =50 V(X) = 100 x ½ x ½ = 25. P(|X-50|>=40)<=25/40^2 = 1/64