Chap 3 Conditional Probability and Independence Ghahramani 3rd edition 2019/2/16

Outline 3.1 Conditional probability 3.2 Law of multiplication 3.3 Law of total probability 3.4 Bayes’ formula 3.5 Independence 3.6 Applications of probability to genetics

3.1 Conditional probability Def If P(B)>0, the conditional probability of A given B, denoted by P(A|B), is

Conditional probability Ex 3.1 In a certain region of Russia, P(one lives at least 80)=0.75, P(one lives at least 90)=0.63. What is the probability that a randomly selected 80-year-old person from this region will survive to become 90? Sol: Let B and A be the events that the person selected survives to become 80 and 90 years old, respectively. P(A|B)=P(AB)/P(B)=P(A)/P(B)=0.63/0.75=0.84

Conditional probability Ex 3.2 From the set of all families with 2 children, a family is selected at random and is found to have a girl. What is the probability that the other child of the family is a girl? Assume that in a 2-child family all sex distributions are equally probable. Sol: Let B and A be the events that the family has a girl (meaning at least a girl) and the family has 2 girls, respectively. possible equally likely outcomes: (b,b),(b,g),(g,b),(g,g) P(A|B)=P(AB)/P(B)=P(A)/P(B)=(1/4)/(3/4)

Conditional probability Ex 3.5 Draw 8 cards from a deck of 52 cards. Given 3 of them are spades, P(the remaining 5 are also spades)=? Sol: B: the event that at least 3 of them are spades A: the event that all 8 cards selected are spades P(A|B)=P(AB)/P(B)=P(A)/P(B) P(A)=C(13,8)/C(52,8) P(B)=C(13,3)C(39,5)/C(52,8) +C(13,4)C(39,4)/C(52,8)+ … +C(13,8)C(39,0)/C(52,8) P(A|B)=5.44 x 10-6

3.2 Law of multiplication

Law of multiplication Ex 3.9 Suppose 5 good fuses and 2 defective ones have been mixed up. To find the defective fuses, we test them 1-by-1, at random and without replacement. What is the probability that we are lucky and find both of the defective fuses in the first two tests? Sol: Let D1 and D2 be the events of finding a defective fuse in the 1st and 2nd tests, respectively. P(D1D2)=P(D1)P(D2|D1) =2/7x1/6=1/21

Law of multiplication Thm 3.2 If P(A1A2A3 … An-1)>0, then P(A1A2A3 … An-1 An) =P(A1)P(A2|A1)P(A3|A1A2) … P(An|A1A2A3 … An-1) Proof: For n=3, if P(AB)>0, then P(ABC)=P(AB)P(C|AB) and P(A)>0 then P(AB)=P(A)P(B|A) so P(ABC)= P(A)P(B|A)P(C|AB) Thm can be proved by math induction.

Law of multiplication Ex 3.11 Suppose 5 good fuses and 2 defective ones have been mixed up. To find the defective fuses, we test them 1-by-1, at random and without replacement. What is the probability that we are lucky and find both of the defective fuses in exactly three tests?

Law of multiplication Sol: Let D1 , D2 and D3 be the events that the 1st, 2nd, and 3rd fuses tested are defective, respectively. Let G1 , G2 and G3 be the events that the 1st, 2nd, and 3rd fuses tested are good, respectively. P(G1D2D3UD1G2D3) =P(G1D2D3)+P(D1G2D3) =P(G1)P(D2|G1)P(D3|G1D2) + P(D1)P(G2|D1)P(D3|D1G2) =5/7x2/6x1/5+2/7x5/6x15 =0.095

3.3 Law of total probability Thm 3.3(Law of total probability) Let P(B)>0, and P(Bc)>0, then P(A)=P(A|B)P(B)+P(A|Bc)P(Bc) Proof: By Thm 1.7, P(A)=P(AB)+P(ABc) and use law of multiplication.

Law of total probability Ex 3.12 An insurance company rents 35% of the cars for its customers from agency I and 65% from agency II. If 8% of the cars of agency I and 5% of the cars of agency II break down during the rental periods, what is the probability that a car rented by this insurance company breaks down? Sol: P(A)=P(A|I)P(I)+ P(A|II)P(II) =(0.08)(0.35)+(0.05)(0.65)=0.0605

Law of total probability Ex 3.14(Gambler’s ruin problem) 2 gamblers play the game of “heads or tails,” in which each time a fair coin lands heads up, player A wins $1 from B, and each time it lands tails up, player B wins $1 from A. Suppose that the player A initially has a dollars and player B has b dollars. If they continue to play this game successively, what is the probability that (a) A will be ruined; (b) the game goes forever with nobody winning?

Law of total probability Sol: (a) Let E be the event that A will be ruined if he starts with i dollars, and let pi=P(E). Our aim is to calculate pa. To do so we define F to be the event that A wins the 1st game. Then P(E)=P(E|F)P(F)+P(E|Fc)P(Fc) => pi = pi+1(1/2)+pi-1(1/2) => pi+1 – pi = pi – pi-1

Law of total probability p0=1, pa+b=0, and let p1- p0 = x pi+1 – pi = pi – pi-1 = … = p2- p1 = p1- p0 = x Thus p1 = p0 + x p2 = p0 + 2x … pi = p0 + ix and pa+b = p0 + (a+b)x => x = -1/(a+b) So pi = 1 – i/(a+b). In particular pa = b/(a+b)

Law of total probability The same method can be used with obvious modifications to calculate qi, the probability that B is ruined if he starts with i dollars. qi = 1 – i/(a+b) Since B starts with b dollars, he will be ruined with probability qb = a/(a+b). Thus the probability that the game goes on forever with nobody winning is 1-(qb+pa)=0.

Law of total probability Definition Let {B1, B2, … , Bn} be a set of nonempty subsets of the sample space S of an experiment. If the events B1, B2, … , Bn are mutually exclusive and , the set {B1, B2, … , Bn} is called a partition of S.

Law of total probability Thm 3.4(Law of total probability) If {B1, B2, … , Bn} is a partition of the sample space S of an experiment and P(Bi)>0 for all i. Then for any event A of S, P(A)=P(A|B1)P(B1)+P(A|B2)P(B2)+ … +P(A|Bn)P(Bn) Thm 3.5 Thm 3.4 can be generalized to n=

Law of total probability Ex 3.15 Suppose that 80% of the seniors, 70% of the juniors, 50% of the sophomores, and 30% of the freshmen of a college use the library of their campus frequently. If 30% of all students are freshmen, 25% are sophomores, 25% are juniors, and 20% are seniors, what percent of all students use the library frequently? Sol: P(A)=P(A|F)P(F)+P(A|O)P(O) +P(A|J)P(J)+P(A|E)P(E) =(0.30)(0.30)+(0.50)(0.25) +(0.70)(0.25)+(0.80)(0.20) =0.55

3.4 Bayes’ formula In a bolt factory, 30, 50, and 20% of production is manufactured by machines I, II, and III, respectively. If 4, 5, and 3% of the output of these respective machines is defective, what is the probability that a randomly selected bolt that is found to be defective is manufactured by machine III?

Bayes’ formula To solve this problem, let A be the event that a random bolt is defective and Bi be the event that it is manufactured by machine i. We are asked to find P(B3|A). Now P(B3|A)= P(B3A)/P(A) and P(B3A)=P(A|B3)P(B3) What is P(A)? By law of total probability P(A)=P(A|B1)P(B1)+P(A|B2)P(B2)+P(A|B3)P(B3)

Bayes’ formula So Bayes’ formula: P(B3|A) = P(B3A)/P(A) = P(A|B3)P(B3)/ [P(A|B1)P(B1)+P(A|B2)P(B2)+P(A|B3)P(B3)]

Bayes’ formula Ex 3.19 A box contains 7 red and 13 blue balls. 2 balls are selected at random and are discarded without their colors being seen. If a 3rd ball is drawn randomly and observed to be red, what is the probability that both of the discarded balls were blue?

Bayes’ formula Sol: Let BB, BR, and RR be the events that the discarded balls are blue and blue, blue and red, red and red, respectively. By Bayes’formula: P(BB|R)=P(R|BB)P(BB)/ [P(R|BB)P(BB)+P(R|BR)P(BR)+ P(R|RR)P(RR)]

Bayes’ formula Now P(BB)=(13/20)(12/19)=39/95 P(RR)=(7/20)(6/19)=21/190 P(BR)=(13/20)(7/19)+(7/20)(13/19)=91/190 Thus P(BB|R)=(7/18)(39/95)/ [(7/18)(39/95)+(6/18)(91/190)+(5/18)(21/190)]=0.46

3.5 Independence In general, the conditional probability of A given B is not the probability of A. However, if it is, that is P(A|B)=P(A), we say that A is independent of B. P(A|B)=P(A) P(AB)/P(B)=P(A) P(AB)=P(A)P(B) Definition Two events A and B are called independent if

Independence Ex 3.24 An urn contains 5 red and 7 blue balls. Suppose that 2 balls are selected at random and with(without) replacement. Let A and B be the events that the first and the second ball are red, respectively. Are A and B independent?

Independence Sol: (a) with replacement P(AB)=P(A)P(B)=(5/12)(5/12) So A and B are independent (b) without replacement P(B|A)=4/11 while P(B)=P(B|A)P(A)+P(B|Ac)P(Ac) =(4/11)(5/12)+(5/11)(7/12)=5/12 So A and B are dependent

Independence Thm 3.6 If A and B are independent, then A and Bc are independent as well. Proof: P(A)=P(AB)+P(ABc) Therefore P(ABc)=P(A)-P(AB) =P(A)-P(A)P(B) =P(A)[1-P(B)] =P(A)P(Bc) Coro If A and B are independent, then Ac and Bc are independent as well.

Independence Ex 3.28(Jailer’s paradox) The jailer of a prison in which Alex, Bill, and Tim are held is the only person, other than the judge, who knows which of these prisoners is condemned to death, and which two will be freed. Alex has written a letter to his fiancee and wants to give it to either Bill or Tim, whoever goes free, to deliver. So Alex asks the jailer to tell him which of the two will be freed. The jailer refuses to give that information to Alex, explaining that, if he does, the probability of Alex dying increases from 1/3 to 1/2. Really?

Independence No. The jailer is wrong! We will show that even if the jailer tells the name of which of the other two who will go free, the probability of Alex dying is still 1/3. Sol-1: Let A, B, and T be the events that “Alex dies,” ”Bill dies,” and “Tim dies.” Let w1=(T, the jailer tells Alex that Bill goes free) w2=(B, the jailer tells Alex that Tim goes free) w3=(A, the jailer tells Alex that Bill goes free) w4=(A, the jailer tells Alex that Tim goes free)

Independence The sample space S = {w1, w2, w3, w4} P(w1)=P(w2)=1/3 Let J be the event that “the jailer tells Alex that Tim goes free” then

Independence Sol-2(Zweifel, in June 1986 issue of Mathematics Magazine, page 156) He analyzes this paradox by using Bayes’ formula:

Independence We now extend the concept of independent to finitely many events. Definition The events A, B, and C are called independent if P(AB)=P(A)P(B) P(AC)=P(A)P(C) P(BC)=P(B)P(C) P(ABC)=P(A)P(B)P(C)

Independence Definition The set of events {A1, A2, …, An} is called independent if for every subset {Ai1, Ai2, …, Aik}, k>1, P(Ai1 Ai2 … Aik)=P(Ai1)P(Ai2)…P(Aik)

Independence Ex 3.31 We draw cards, one at a time, at random and successively from an ordinary deck of 52 cards with replacement. What is the probability that an ace appears before a face card?

Independence Sol: Let E be the event of an ace appearing before a face card. Let A, F, and B be the events of ace, face card, and neither in the first experiment, respectively. Then P(E)=P(E|A)P(A)+P(E|F)P(F)+P(E|B)P(B) =1(4/52)+0(12/52)+P(E|B)(36/52) P(E|B)=P(E) so P(E)=4/52+P(E)(36/52) so P(E)=1/4

Skip 3.6 Applications of probability to genetics