ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Midterm 22 nd Mar One and a half hour. Bring calculator and blank papers. Close-book and close-note exam. Coverage: – Lecture 1 to 15. – Tutorial 1 to 7. – Homework 1 to 3. kshumENGG2012B2

Derangements of n=4 objects Let  be the set of all 24 permutations of 1,2,3,4. For i = 1,2,3,4, let A i be the set of permutations which fix i. A 1 = {1234, 1243, 1324, 1342, 1423, 1432}. A 2 = {1234, 1243, 3214, 3241, 4213, 4231}. A 3 = {1234, 1432, 2134, 2431, 4132, 4231}. A 4 = {1234, 1324, 2134, 2314, 3124, 3214}. A permutation of {1,2,3,4} not in A 1 to A 4 has no fixed point, and is called a derangement. kshumENGG2012B3

Venn diagram kshumENGG2012B4  A1A1 A2A2 A3A3 A4A

The case n=4 (cont’d) A 1  A 2 = {1234, 1243}. A 1  A 3 = {1234, 1432}. A 1  A 4 = {1234, 1324}. A 2  A 3 = {1234, 4231}. A 2  A 4 = {1234, 3214}. A 3  A 4 = {1234, 2134}. A 1  A 2  A 3 = A 1  A 2  A 4 = A 1  A 3  A 4 =A 2  A 3  A 4 = A 1  A 2  A 3  A 4 ={1,2,3,4}. By PIE, the number of derangements for n=4 is equal to 4! – 4   2 – 4  = 9. Indeed, the nine derangements for n=4 are 2143, 2341, 2413, 3142, 3412, 3421, 4123, 4312, kshumENGG2012B5

Probability of no fixed number For n=4, if we pick a permutation randomly, – the probability of no fixed number is 9/24= – the probability of at least one fixed number is 15/24= For large n, it can be shown that the probability of no fixed number is approximately 1/e = … kshumENGG2012B6 nProbability that a random permutation of length n is a derangement

John Venn ( ) British logician and philosopher kshumENGG2012B7

Classical definition of probability From Laplace’s Théorie analytique des probabilités (1812) – “The probability of an event is the ratio of the number of cases favourable to it, to the number of all possible cases.” kshumENGG2012B8

BASIC PROBABILITY kshumENGG2012B9

Disjoint events Two events are called disjoint if the intersection is empty, i.e., if they have no overlap. The calculation of union of events in general is complicated when the events overlap. It is much easier if we want to compute the probability of a union of disjoint event. We simply add the probability of the events kshumENGG2012B10

Union of disjoint events kshumENGG2012B11  A B C Pr(A  B  C) = Pr(A) + Pr(B) + Pr(C). A, B, C mutually disjoint

Partition of sample space kshumENGG2012B12  A B C D Pr(A) + Pr(B) + Pr(C) + Pr(D)= Pr(A  B  C  D) = 1 A  B  C  D = , A, B, C, D mutually disjoint

Law of total probability kshumENGG2012B13  A B C D Pr(E) = Pr(E  A)+ Pr(E  B)+ Pr(E  C)+ Pr(E  D). A  B  C  D = , A, B, C, D mutually disjoint E

CONDITIONAL PROBABILITY kshumENGG2012B14

Example There are four cards, two of them are red and two of them are black. Shuffle the four cards and lay out the cards on the table with the front of each card facing downwards. What is the probability that the first card is red? We reveal the last card. – If it turns out that the last card is red, what is the probability that the first card is red? – If it turns out that the last card is black, what is the probability that the first card is red? kshumENGG2012B15

Conditional probability If we are given an addition information that the outcome is in event B, then we can update the likelihood to |A  B|/ |B|. If Pr(B) is nonzero, then we define the conditional probability of A given B by Pr(A|B) = Pr(A  B)/Pr(B). kshumENGG2012B16  A B At the beginning, the probability of event A is |A| / |  |, assuming that all outcomes in  are equally likely.

The law of total probability in terms of conditional probability kshumENGG2012B17  A B C D Pr(E) = Pr(E  A)+ Pr(E  B)+ Pr(E  C)+ Pr(E  D) = Pr(E|A)Pr(A)+ Pr(E|B)Pr(B)+ Pr(E|C)Pr(C)+ Pr(E|D)Pr(D). A  B  C  D = , A, B, C, D mutually disjoint E

Example Alice rolls a fair dice two times. Suppose Bob is told that the face value of the first roll is 6. What is the probability the second roll is also 6? Suppose Carol is told that the face value of one of the roll is 6. What is the probability that the other roll is also 6? kshumENGG2012B18

Independent events If P(A|B) = P(A), then it means that the likelihood of the event A does not change after we are told that event B has occurred. In this case, event A is said to be independent of event B. As Pr(A|B) = Pr(A  B)/Pr(B), event A is independent of B if Pr(A  B) = Pr(A) Pr(B). Two events A and B are said to be independent, or statistically independent, if Pr(A  B) = Pr(A) Pr(B). kshumENGG2012B19

Example Roll two fair dice. Let A be the event that the face value of the first die is even. Let B be the event that the sum of the dice is odd. Are events A and B independent? kshumENGG2012B First number is even Sum is odd

Pairwise independent does not imply Pr(A  B  C)=Pr(A)Pr(B)Pr(C). Let  be the sample space {0,1,2,3}. The four outcomes are equally likely. Let A be the event {0,1}, B be the event {0,2}, and C be the event {0,3}. The three events A, B and C are pairwise independent, because – Pr(A) Pr(B) = (1/2) 2 = 1/4 = Pr(A  B) – Pr(B) Pr(C) = (1/2) 2 = 1/4 = Pr(B  C) – Pr(C) Pr(A) = (1/2) 2 = 1/4 = Pr(C  A) However, – Pr(A  B  C) = 1/4 – Pr(A) Pr(B) Pr(C) = (1/2) 3 =1/8 kshumENGG2012B21  A B C

Example There are two boxes. The first box contains 3 red balls and 7 blue balls. The second box contains 16 red balls and 4 blue balls. Perform the following random experiment – Throw a fair die. – If the number is 1, pick a ball randomly from the first box. – If the number is 2 to 5, pick a ball randomly from the second box. What is the probability that a red ball is drawn? What is the probability that a blue ball is drawn? Given that a red ball is picked, what is the probability that the first box is chosen? kshumENGG2012B22

The Bayes’ rule Let A and B be two events. Suppose that the probability of B is nonzero. Probability of A given B can be computed from the probability of B given A by kshumENGG2012B23

Thomas Bayes ( ) English minister and mathematician kshumENGG2012B24

The Monty Hall Problem Quote from wikipedia – Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice? An explanation from youtube – kshumENGG2012B25

Explanation using conditional probability We use – “100” to represent “prize behind the first door”. – “010” to represent “prize behind the second door”. – “001” to represent “prize behind the third door”. The host may need to flip a coin in order to open the door. We use “H” and “T” for the outcome of the coin toss. Set up a sample space  of six elements  = {100H, 100T, 010H, 010T, 001H, 001T}. Each outcome is equally probable. kshumENGG2012B26

Before any door is opened by the host Suppose you choose door 1 kshumENGG2012B /3 The probability that there is a car behind the first door is 1/3.

After a door is opened Suppose you choose door 1. A door is opened by the host of the game. kshumENGG2012B /3 H T H T H T Door 3 is opened Door 2 is opened Door 3 is opened Door 2 is opened

Probability given that door 3 is opened Suppose you choose door 1. Suppose door 3 is opened Pr(Prize is behind door 1 | door 3 is opened) = 1/3 If you switching to door 2, you will win with probability 2/3 kshumENGG2012B /3 H T H T H T Door 3 is opened

Probability given that door 2 is opened Suppose you choose door 1. Suppose door 3 is opened Pr(Prize is behind door 1 | door 2 is opened) = 1/3 If you switching to door 3, you will win with probability 2/3 kshumENGG2012B /3 H T H T H T Door 2 is opened