1. Lectures prepared by: Elchanan Mossel Yelena Shvets
Introduction to probability
Stat FAll 2005
Berkeley
Lectures prepared by: Elchanan Mossel Yelena Shvets
Follows Jim Pitman’s book:
Probability
Sections
5/31/2019

2. Three draws from a magic hat.
5/31/2019

3. Three draws from a magic hat.
Space of possible ‘3 draws’ from the hat:
Note: Draws are without replacement
5/31/2019

4. Three draws from a magic hat.
What is the chance that
we get an on the 2nd draw,
if we got a on the 1st draw?
5/31/2019

5. Three draws from a magic hat.**
* * * P p p p p p
P(*I*|R**)=1/2 p
5/31/2019

6. Three draws from a magic hat.
What is the chance that
we get an on the 1st draw,
if we got a on the 2nd draw?
5/31/2019

7. Three draws from a magic hat.**
* * * P p p p p p
P(I**|*R*)=1/2 p
5/31/2019

8. Counting formula. Under the uniform measure: P(A|B)= #(AB)/#(B)W A B w1 X w2 w3 w4 w5 w6 AB X
5/31/2019

9. Frequency interpretation
In a long sequence of trials, among those which belong to B, the proportion of those that also belong to A should be about P(A|B).
5/31/2019

10. Three draws from a magic hat.
Let’s make 3 draws from the magic hat many times:
#(AB)/#B = 4/7¼ 1/2
5/31/2019

11. For a uniform measure we have:
5/31/2019

12. Conditional probability in general:
Let A and B be two events in a general probability space.
The conditional probability of A given B is denoted by P(A | B).
It is given by:
P(A | B) = P(A Å B) / P(B)
5/31/2019

13. Example: Rich & Famous Example: Rich & Famous Neither Famous Rich
In a certain town 10% of
the inhabitants are rich,
5% are famous and
3% are rich and famous.
Neither
If a town’s person is chosen at random and she is rich what is the probability she is famous?
Famous
Rich
5/31/2019

14. Example: Rich & Famous Example: Rich & Famous Neither Famous Rich
In a certain town 10% of
the inhabitants are rich,
5% are famous and
3% are rich and famous.
Neither
P(R) = 0.1
P(R & F) = 0.03
P(F | R) = =0.03/0.1
= 0.3
Famous
Rich
5/31/2019

15. Example: Relative areas
A point is picked uniformly at random from the big rectangle whose area is 1.
Suppose that we told that the point is in B, what is the chance that it is in A? A B
5/31/2019

16. Example: Relative areas
In other words: Given that the point is in B, what is the conditional probability that it is in A? AB ( )
Area A B
= 1/2 B ( )
Area
5/31/2019

17. Multiplication Rule P(AB)=P(B)P(A|B)
For all events A and B such that P(B)  0:
P(AB)=P(B)P(A|B)
5/31/2019

18. Box then ball Consider the following experiment:
we first pick one of the two boxes; ,
and next we pick a ball from the boxed that we picked.
What’s the chance of getting a 2? 1 3 5 2 4
5/31/2019

19. Tree diagrams P(Box 1) = 1/2 P(Box 2) = 1/2 2 4 1 3 5 1/3 1/3 1/3 1 3
1/4
1/4
1/6
1/6
1/6
P(2) = P(2 Å Box2) = P(Box2) P(2|Box2) = ½*1/2 = 1/4
5/31/2019

20. Consider the Partition
B1 t B2 t … t Bn = W B1 B2 B3 B4 Bn
Bn-1 A
AB1 t AB2 t … t ABn = A
P(AB1)+ P(AB2)+…+P(ABn) = P(A)
5/31/2019

21. Rule of Average Conditional Probabilities
If B1,…,Bn is a disjoint Partition of  then
P(A)= P(AB1) P(AB2) …+ P(ABn)
= P(A|B1)P(B1) + P(A|B2)P(B2)+…+P(A|Bn)P(Bn)
5/31/2019

22. Box then ball We make the following experiment:
we first pick one of the two boxes; ,
and next we pick a ball from the boxed that we picked.
What’s the probability getting a number smaller
than 3.5? 1 3 5 2 4
5/31/2019

23. Independence
When the probability for A is unaffected by the occurrence of B we say that A and B are independent.
In other words, A and B are independent if
P(A|B)=P(A|Bc) = p
Note that if A and B are independent then:
P(A) = P(A|B)P(B) + P(A|Bc)P(Bc)
= p P(B) + p P(Bc)
= p
5/31/2019

24. Independence
Obvious: If A is independent of B then A is also independent of Bc
Question: If A is independent of B, is B independent of A?
5/31/2019

25. Independence Consider a ball picked uniformly: Independent: 1 1 2 1 1
Then Color=Red/Green and Number=1 or 2 are
Independent:
P(Red|#=1)=1/2=P(Red|#=2);
P(Green|#=1)=1/2=P(Green|#=2);
5/31/2019

26. Independence Also: 1 1 2 1 1 2 P(#=2|Green)=1/3=P(#=2|Red);
5/31/2019

27. Independence Consider a ball picked uniformly:1 2 2 1 1 2
Then Color and Number are
not independent:
P(#=2|Green)=1/3 ¹ P(#=2|Red)=2/3;
5/31/2019

28. Independence X Y Points from a figure have coordinates X and Y.
If a point is picked at uniformly from a rectangle then
the events {X > a} and {Y > b} are
independent! X Y
5/31/2019

29. P(X>a & Y>b) = P(X>a) P(Y>b) = (1-a)(1-b)
Independence
P(X>a & Y>b) = P(X>a) P(Y>b) = (1-a)(1-b) Y 1 X 1
5/31/2019

30. Independence Points from a figure have coordinates X and Y.
If a point is picked uniformly from a cat shape then
{X > a} and {Y > b} are
not independent! (at least for some a & b) Y X
5/31/2019

31. Points from a figure have coordinates X and Y.
Independence
Points from a figure have coordinates X and Y.
Are the events {X > a} and {Y>b} independent if a point is picked uniformly at random from a disc? X Y
5/31/2019

32. Dependence P(X> )= B/p = P(X> ) Y 2 B 1 B Area = p X 1 21 2
5/31/2019

33. Dependence P(X> & Y> ) = 0 ¹ B2/p2 Y 2 B 1 B X 1 2 So the events
Are not independent
for a = b = B 1 B X 1 2
5/31/2019

34. Independence
Follow up question: Are there values of a and b for which the event {X > a} and {X > b} are independent? X Y
5/31/2019

35. Recall: Multiplication Rule
P(AB)=P(A|B)P(B)
=P(A) P(B)
This holds even if A and B are independent !
5/31/2019

36. Picking a Box then a Ball
If a ball is drawn from a randomly picked box comes out to be red, which box would you guess it came from and what is the chance that you are right?
5/31/2019

37. Box then Ball
1/3
1/3
1/3
2/3
1/3
1/2
1/2
3/4
1/4
5/31/2019

38. By the Rule of Average Conditional Probability
P(R Ball) =
P(R Ball | W Box) P(W Box) +
P(R Ball| Y Box) P(Y Box) +
P(R Ball | B Box) P(B Box)
= ½ * 1/3 +
2/3*1/3 +
¾*1/3
= 23/36
5/31/2019

39. P( , ) P( | ) = = P( ) P( , ) P( | ) = = P( ) P( , ) P( | ) = = P( )
1/3*1/2 =
23/36 P( ) P( , ) P( | ) =
1/3*2/3 =
23/36 P( ) P( , ) P( | ) =
1/3*3/4 =
23/36 P( )
5/31/2019

40. Picking a Box then a Ball
If a ball is drawn from a randomly picked box comes out to be red, which box would you guess it came from and what is the chance that you are right?
A: Guess -> last box. Chances you are right: 9/23.
5/31/2019

41. Bayes’ Rule For a partition B1, …, Bn of all possible outcomes,
5/31/2019

42. Ac Bc Cc C A B Sequence of Events Multiplication rule for 3 Events
P(ABC) = P(AB)P(C|AB) = P(A) P(B|A) P(C|AB) Ac Bc Cc
P(A)
P(B|A)
P(C|AB) C A B
5/31/2019

43. Multiplication rule for n Events
P(A1 A2 … An) = P(A1 … An-1)P(An|A1 … An-1)
= P(A1) P(A2|A1) P(A3|A1 A2)… P(An| A1 … An-1)
A1c
A2c
Anc
P(A1)
P(A2|A1)
P(An| A1 … An-1) A1
A2… An
5/31/2019

44. Shesh Besh Backgammon
We roll two dice. What is the chance that we will roll out Shesh Besh: for the first time on the n’th roll?
5/31/2019

45. This is a Geometric Distribution
Shesh Besh Backgammon
This is a Geometric Distribution
with parameter p=1/36.
5/31/2019

46. The Geometric distribution
In Geom(p) distribution the probability of outcome n is given by
p (1-p)n-1
5/31/2019

47. P(at least 2 have same birthday) = 1 – P(No coinciding birthdays).
The Birthday Problem
n students in the class, what is the chance that at least two of them have the same birthday?
P(at least 2 have same birthday) =
1 – P(No coinciding birthdays).
We arrange the students’ birthdays in some order:
B1, B2,…, Bn.
We need:
P(B2 Ï {B1} & B3 Ï {B1,B2} & … & Bn Ï {B1,…,Bn-1}).
5/31/2019

48. Use multiplication rule to find
P(B2 Ï {B1} & B3 Ï {B1,B2} & … & Bn Ï {B1,…,Bn-1}).
B2=B1
B32 {B1,B2}
Bn2 {B1, … Bn-1} …
B2Ï{B1}
B3Ï{B1,B2}
BnÏ{B1, … Bn-1}
5/31/2019

49. P(at least 2 have same birthday) = 1 – P(No coinciding birthdays) =
The Birthday Problem
P(at least 2 have same birthday) =
1 – P(No coinciding birthdays) =
This is hard to compute for large n. So we can use an approximation.
5/31/2019

50. log(P(No coinciding birthdays))=
The Birthday Problem
log(P(No coinciding birthdays))=
5/31/2019

51. P(No coinciding birthdays)
The Birthday Problem
P(No coinciding birthdays)
P(At least 2 have same birthday)
5/31/2019

52. Probabilities in the Birthday Problem.
Shesh Besh Backgammon
Probabilities in the Birthday Problem. n
5/31/2019

53. Independence of n events
P(B|A)=P(B|Ac) = P(B);
P(C|AB)= P(C|AcB) = P(C|AcBc) = P(C|ABc) =P(C)
Multiplication rule for three independent events
P(ABC) = P(A) P(B) P(C)
5/31/2019

54. Pair-wise independence does not imply independence
I pick one of these people at random. If I tell you that it’s a girl, there is an equal chance that she is a blond or a brunet; she has blue or brown eyes. Similarly for a boy.
However, if a tell you that I picked a blond and
blue eyed person, it has to be a boy.
So sex, eye color and hair color, for this group, are
pair-wise independent, but not independent.
5/31/2019