Lecture Discrete Probability

5.2 Recap Sample space: space of all possible outcomes. Event: subset of of S. p(s) : probability of element s of S: Probability of complement Disjoint events: Overlapping events: Three doors problem (see ex. 39 p matlab demo)

5.2 We flip a coin 3 times with Heads and Tails equal probability. There are 8 possibilities: |S| = 8 E: event that an odd number of tails occurs F: first flip comes up tails HHH HHT HTH HTT THH TTH THT TTT E F S

5.2 HHH HHT HTH HTT THH TTH THT TTT E F S What is the probability of odd number of tails (i.e. of event E) if we know that the first flip was tails (i.e. F happened) ?  If we know that F happened the total number of possible outcomes shrinks to |F| E = THH, THT, TTH, TTT. Of those there are two that make event E happen: THH, TTT. P(E|F) = 1/2 THH TTT THT TTH F E|F

5.2 Theorem: If E and F are two events and p(F)>0, then the conditional probability of E given F is given by: Example: What is the probability that a family with 2 kids has two boys, given that they have at least one boy? (all possibilities are equally likely).  S: all possibilities: {BB, GB, BG, GG}. E: family has two boys: {BB}. F: family has at least one boy: {BB, GB, BG}. E F = {BB} p(E|F) = (1/4) / (3/4) = 1/3 E BB F GB BG GG

5.2 E BB F GB BG GG p(E) = ¼ P(E|F) = 1/3 HHH HHT HTH HTT THH TTH THT TTT E F S P(E)=1/2 P(E|F) = 1/2 By knowing F has happened, I have changed the probability that E has happened: they are dependent By knowing F has happened, I have NOT changed the probability that E has happened: they are independent S

5.2 Equivalent statement: The probability of the intersection od 2 events is the product of the probabilities of the separate events. Example: A family has 2 children (|S|=4). Is the event E that the family has children of both sexes independent from the event F that they have at most one boy? E: {BG, GB} F: {GG, BG, GB} E F: {BG, GB} dependent

5.2 What about a family with 3 kids ? |S| = 8 |E| = 8 – 2 = 6 (only BBB and GGG violate E). |F| =| {GGG, GBB, BGB, BBG} | = 4 =| {GBB, BGB, BBG}| = 3 independent ! (again: it’s hard to get any intuition for this).

5.2 Problem: We toss a coin 7 times, but the coin biased with probability 2/3 heads. What is the probability that we find 4 heads in those 7 tosses? There are C(7,4) ways to generate a sequence with 4 heads and 3 tails. (thinks of a bit-string with 0’s and 1’s). Each one of them has probability (2/3)^4 x (1/3)^3. Total: probability: C(7,4) x (2/3)^4 x (1/3)^3 More general: What is the probability of finding k 1’s in a bit-string of length n, when the probability of finding a 1 is p?  Binomial distribution. (matlab demo)

5.2 Does it sum to 1? Recall: Use q = 1-p to prove the result.

5.2 Random Variables Many problems deal with real numbers rather than set memberships. E.g. What is the sum of the outcomes of 2 dice? How many times do we expect to 7 1 in bit-strings of length 12? We were able to answer those questions before, but we now introduce some formal machinery: Definition: A random variable X is a function (not a variable!) from sample space to the real numbers. It assigns a real number to each possible outcome. (and is not random!). Example: S={apple, pear, banana}  X(apple) = 1, X(pear) = 2, X(banana) = 3. X: the number of times heads comes up when we toss a coin 2 times: X(TT) = 0; X(HT) = 1; X(TH) = 1; X(HH) = 2;

5.2 Definition: Let P(X=r) be the probability that X takes a value r. A distribution of a random variable X on a sample space S is the set of all pairs (r,P(X=r)) for all r in X(S). Thus, we specify the distribution by providing all possible P(X=r). Example: X is the number of heads of 2 coin tosses: P(X=0) = ¼ P(X=1) = ½ P(X=2) = ¼

5.3 Definition: The expected value of a random variable X(s) on the sample space S is given by: Sometimes, it’s more efficient to compute expectations by clumping together all elements of S that result in the same value for X(s). number of elements in S such that X(s)=r only true when all p(s) s.t. X(s)=r have equal probability. generally true

5.3 Example: X is number of heads in 2 coin tosses. Expected value? E(X) = 0 x P(X=0) + 1 x P(X=1) + 2 x P(X=2) = 0 + ½ + 2x ¼ = 1. We expect on average that we see 1 head. Example: X is the value of the number that comes up on a die. Expected value? E(X) = 1 x 1/6 + 2 x 1/6 + 3 x 1/6 + 4 x 1/6 + 5 x 1/6 + 6 x 1/6 = 21/6 = 7/2. (matlab demo2)