Lectures prepared by: Elchanan Mossel Yelena Shvets Introduction to probability Stat 134 FAll 2005 Berkeley Lectures prepared by: Elchanan Mossel Yelena Shvets Follows Jim Pitman’s book: Probability Sections 1.4-1.5 5/31/2019

Three draws from a magic hat. 5/31/2019

Three draws from a magic hat. Space of possible ‘3 draws’ from the hat: Note: Draws are without replacement 5/31/2019

Three draws from a magic hat. What is the chance that we get an on the 2nd draw, if we got a on the 1st draw? 5/31/2019

Three draws from a magic hat. ** * * * P   p p p p p P(*I*|R**)=1/2 p 5/31/2019

Three draws from a magic hat. What is the chance that we get an on the 1st draw, if we got a on the 2nd draw? 5/31/2019

Three draws from a magic hat. ** * * * P   p p p p p P(I**|*R*)=1/2 p 5/31/2019

Counting formula. Under the uniform measure: P(A|B)= #(AB)/#(B) W A B w1 X w2 w3 w4 w5 w6 AB X 5/31/2019

Frequency interpretation In a long sequence of trials, among those which belong to B, the proportion of those that also belong to A should be about P(A|B). 5/31/2019

Three draws from a magic hat. Let’s make 3 draws from the magic hat many times: #(AB)/#B = 4/7¼ 1/2 5/31/2019

For a uniform measure we have: 5/31/2019

Conditional probability in general: Let A and B be two events in a general probability space. The conditional probability of A given B is denoted by P(A | B). It is given by: P(A | B) = P(A Å B) / P(B) 5/31/2019

Example: Rich & Famous Example: Rich & Famous Neither Famous Rich In a certain town 10% of the inhabitants are rich, 5% are famous and 3% are rich and famous. Neither If a town’s person is chosen at random and she is rich what is the probability she is famous? Famous Rich 5/31/2019

Example: Rich & Famous Example: Rich & Famous Neither Famous Rich In a certain town 10% of the inhabitants are rich, 5% are famous and 3% are rich and famous. Neither P(R) = 0.1 P(R & F) = 0.03 P(F | R) = =0.03/0.1 = 0.3 Famous Rich 5/31/2019

Example: Relative areas A point is picked uniformly at random from the big rectangle whose area is 1. Suppose that we told that the point is in B, what is the chance that it is in A? A B 5/31/2019

Example: Relative areas In other words: Given that the point is in B, what is the conditional probability that it is in A? AB ( ) Area A B = 1/2 B ( ) Area 5/31/2019

Multiplication Rule P(AB)=P(B)P(A|B) For all events A and B such that P(B)  0: P(AB)=P(B)P(A|B) 5/31/2019

Box then ball Consider the following experiment: we first pick one of the two boxes; , and next we pick a ball from the boxed that we picked. What’s the chance of getting a 2? 1 3 5 2 4 5/31/2019

Tree diagrams P(Box 1) = 1/2 P(Box 2) = 1/2 2 4 1 3 5 1/3 1/3 1/3 1 3 1/4 1/4 1/6 1/6 1/6 P(2) = P(2 Å Box2) = P(Box2) P(2|Box2) = ½*1/2 = 1/4 5/31/2019

Consider the Partition B1 t B2 t … t Bn = W B1 B2 B3 B4 Bn Bn-1 A AB1 t AB2 t … t ABn = A P(AB1)+ P(AB2)+…+P(ABn) = P(A) 5/31/2019

Rule of Average Conditional Probabilities If B1,…,Bn is a disjoint Partition of  then P(A)= P(AB1) + P(AB2) +…+ P(ABn) = P(A|B1)P(B1) + P(A|B2)P(B2)+…+P(A|Bn)P(Bn) 5/31/2019

Box then ball We make the following experiment: we first pick one of the two boxes; , and next we pick a ball from the boxed that we picked. What’s the probability getting a number smaller than 3.5? 1 3 5 2 4 5/31/2019

Independence When the probability for A is unaffected by the occurrence of B we say that A and B are independent. In other words, A and B are independent if P(A|B)=P(A|Bc) = p Note that if A and B are independent then: P(A) = P(A|B)P(B) + P(A|Bc)P(Bc) = p P(B) + p P(Bc) = p 5/31/2019

Independence Obvious: If A is independent of B then A is also independent of Bc Question: If A is independent of B, is B independent of A? 5/31/2019

Independence Consider a ball picked uniformly: Independent: 1 1 2 1 1 Then Color=Red/Green and Number=1 or 2 are Independent: P(Red|#=1)=1/2=P(Red|#=2); P(Green|#=1)=1/2=P(Green|#=2); 5/31/2019

Independence Also: 1 1 2 1 1 2 P(#=2|Green)=1/3=P(#=2|Red); 5/31/2019

Independence Consider a ball picked uniformly: 1 2 2 1 1 2 Then Color and Number are not independent: P(#=2|Green)=1/3 ¹ P(#=2|Red)=2/3; 5/31/2019

Independence X Y Points from a figure have coordinates X and Y. If a point is picked at uniformly from a rectangle then the events {X > a} and {Y > b} are independent! X Y 5/31/2019

P(X>a & Y>b) = P(X>a) P(Y>b) = (1-a)(1-b) Independence P(X>a & Y>b) = P(X>a) P(Y>b) = (1-a)(1-b) Y 1 ¾ ½ X ½ ¾ 1 5/31/2019

Independence Points from a figure have coordinates X and Y. If a point is picked uniformly from a cat shape then {X > a} and {Y > b} are not independent! (at least for some a & b) Y X 5/31/2019

Points from a figure have coordinates X and Y. Independence Points from a figure have coordinates X and Y. Are the events {X > a} and {Y>b} independent if a point is picked uniformly at random from a disc? X Y 5/31/2019

Dependence P(X> )= B/p = P(X> ) Y 2 B 1 B Area = p X 1 2 1 2 5/31/2019

Dependence P(X> & Y> ) = 0 ¹ B2/p2 Y 2 B 1 B X 1 2 So the events Are not independent for a = b = B 1 B X 1 2 5/31/2019

Independence Follow up question: Are there values of a and b for which the event {X > a} and {X > b} are independent? X Y 5/31/2019

Recall: Multiplication Rule P(AB)=P(A|B)P(B) =P(A) P(B) This holds even if A and B are independent ! 5/31/2019

Picking a Box then a Ball If a ball is drawn from a randomly picked box comes out to be red, which box would you guess it came from and what is the chance that you are right? 5/31/2019

Box then Ball 1/3 1/3 1/3 2/3 1/3 1/2 1/2 3/4 1/4 5/31/2019

By the Rule of Average Conditional Probability P(R Ball) = P(R Ball | W Box) P(W Box) + P(R Ball| Y Box) P(Y Box) + P(R Ball | B Box) P(B Box) = ½ * 1/3 + 2/3*1/3 + ¾*1/3 = 23/36 5/31/2019

P( , ) P( | ) = = P( ) P( , ) P( | ) = = P( ) P( , ) P( | ) = = P( ) 1/3*1/2 = 23/36 P( ) P( , ) P( | ) = 1/3*2/3 = 23/36 P( ) P( , ) P( | ) = 1/3*3/4 = 23/36 P( ) 5/31/2019

Picking a Box then a Ball If a ball is drawn from a randomly picked box comes out to be red, which box would you guess it came from and what is the chance that you are right? A: Guess -> last box. Chances you are right: 9/23. 5/31/2019

Bayes’ Rule For a partition B1, …, Bn of all possible outcomes, 5/31/2019

Ac Bc Cc C A B Sequence of Events Multiplication rule for 3 Events P(ABC) = P(AB)P(C|AB) = P(A) P(B|A) P(C|AB) Ac Bc Cc P(A) P(B|A) P(C|AB) C A B 5/31/2019

Multiplication rule for n Events P(A1 A2 … An) = P(A1 … An-1)P(An|A1 … An-1) = P(A1) P(A2|A1) P(A3|A1 A2)… P(An| A1 … An-1) A1c A2c Anc P(A1) P(A2|A1) P(An| A1 … An-1) A1 A2… An 5/31/2019

Shesh Besh Backgammon We roll two dice. What is the chance that we will roll out Shesh Besh: for the first time on the n’th roll? 5/31/2019

This is a Geometric Distribution Shesh Besh Backgammon This is a Geometric Distribution with parameter p=1/36. 5/31/2019

The Geometric distribution In Geom(p) distribution the probability of outcome n is given by p (1-p)n-1 5/31/2019

P(at least 2 have same birthday) = 1 – P(No coinciding birthdays). The Birthday Problem n students in the class, what is the chance that at least two of them have the same birthday? P(at least 2 have same birthday) = 1 – P(No coinciding birthdays). We arrange the students’ birthdays in some order: B1, B2,…, Bn. We need: P(B2 Ï {B1} & B3 Ï {B1,B2} & … & Bn Ï {B1,…,Bn-1}). 5/31/2019

Use multiplication rule to find P(B2 Ï {B1} & B3 Ï {B1,B2} & … & Bn Ï {B1,…,Bn-1}). B2=B1 B32 {B1,B2} Bn2 {B1, … Bn-1} … B2Ï{B1} B3Ï{B1,B2} BnÏ{B1, … Bn-1} 5/31/2019

P(at least 2 have same birthday) = 1 – P(No coinciding birthdays) = The Birthday Problem P(at least 2 have same birthday) = 1 – P(No coinciding birthdays) = This is hard to compute for large n. So we can use an approximation. 5/31/2019

log(P(No coinciding birthdays))= The Birthday Problem log(P(No coinciding birthdays))= 5/31/2019

P(No coinciding birthdays) The Birthday Problem P(No coinciding birthdays) P(At least 2 have same birthday) 5/31/2019

Probabilities in the Birthday Problem. Shesh Besh Backgammon Probabilities in the Birthday Problem. n 5/31/2019

Independence of n events P(B|A)=P(B|Ac) = P(B); P(C|AB)= P(C|AcB) = P(C|AcBc) = P(C|ABc) =P(C) Multiplication rule for three independent events P(ABC) = P(A) P(B) P(C) 5/31/2019

Pair-wise independence does not imply independence I pick one of these people at random. If I tell you that it’s a girl, there is an equal chance that she is a blond or a brunet; she has blue or brown eyes. Similarly for a boy. However, if a tell you that I picked a blond and blue eyed person, it has to be a boy. So sex, eye color and hair color, for this group, are pair-wise independent, but not independent. 5/31/2019