1. ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

2. Midterm 22 nd Mar One and a half hour. Bring calculator and blank papers. Close-book and close-note exam. Coverage: – Lecture 1 to 15. – Tutorial 1 to 7. – Homework 1 to 3. kshumENGG2012B2

3. Derangements of n=4 objects Let  be the set of all 24 permutations of 1,2,3,4. For i = 1,2,3,4, let A i be the set of permutations which fix i. A 1 = {1234, 1243, 1324, 1342, 1423, 1432}. A 2 = {1234, 1243, 3214, 3241, 4213, 4231}. A 3 = {1234, 1432, 2134, 2431, 4132, 4231}. A 4 = {1234, 1324, 2134, 2314, 3124, 3214}. A permutation of {1,2,3,4} not in A 1 to A 4 has no fixed point, and is called a derangement. kshumENGG2012B3

4. Venn diagram kshumENGG2012B4  2143 2341 2413 3142 3412 3421 4123 4312 4321 A1A1 A2A2 A3A3 A4A4 1234 1243 2134 1432 4231 3214 1324 1342 1423 3241 4213 2431 4132 2314 3124

5. The case n=4 (cont’d) A 1  A 2 = {1234, 1243}. A 1  A 3 = {1234, 1432}. A 1  A 4 = {1234, 1324}. A 2  A 3 = {1234, 4231}. A 2  A 4 = {1234, 3214}. A 3  A 4 = {1234, 2134}. A 1  A 2  A 3 = A 1  A 2  A 4 = A 1  A 3  A 4 =A 2  A 3  A 4 = A 1  A 2  A 3  A 4 ={1,2,3,4}. By PIE, the number of derangements for n=4 is equal to 4! – 4  6 + 6  2 – 4  1 + 1 = 9. Indeed, the nine derangements for n=4 are 2143, 2341, 2413, 3142, 3412, 3421, 4123, 4312, 4321. kshumENGG2012B5

6. Probability of no fixed number For n=4, if we pick a permutation randomly, – the probability of no fixed number is 9/24= 0.375. – the probability of at least one fixed number is 15/24= 0.625. For large n, it can be shown that the probability of no fixed number is approximately 1/e = 0.3679… kshumENGG2012B6 nProbability that a random permutation of length n is a derangement 30.3333 40.3750 50.3667 60.3681 70.3679

7. John Venn (1834-1923) British logician and philosopher http://en.wikipedia.org/wiki/John_Venn kshumENGG2012B7

8. Classical definition of probability From Laplace’s Théorie analytique des probabilités (1812) – “The probability of an event is the ratio of the number of cases favourable to it, to the number of all possible cases.” kshumENGG2012B8 http://en.wikipedia.org/wiki/Pierre-Simon_Laplace

9. BASIC PROBABILITY kshumENGG2012B9

10. Disjoint events Two events are called disjoint if the intersection is empty, i.e., if they have no overlap. The calculation of union of events in general is complicated when the events overlap. It is much easier if we want to compute the probability of a union of disjoint event. We simply add the probability of the events kshumENGG2012B10

11. Union of disjoint events kshumENGG2012B11  A B C Pr(A  B  C) = Pr(A) + Pr(B) + Pr(C). A, B, C mutually disjoint

12. Partition of sample space kshumENGG2012B12  A B C D Pr(A) + Pr(B) + Pr(C) + Pr(D)= Pr(A  B  C  D) = 1 A  B  C  D = , A, B, C, D mutually disjoint

13. Law of total probability kshumENGG2012B13  A B C D Pr(E) = Pr(E  A)+ Pr(E  B)+ Pr(E  C)+ Pr(E  D). A  B  C  D = , A, B, C, D mutually disjoint E

14. CONDITIONAL PROBABILITY kshumENGG2012B14

15. Example There are four cards, two of them are red and two of them are black. Shuffle the four cards and lay out the cards on the table with the front of each card facing downwards. What is the probability that the first card is red? We reveal the last card. – If it turns out that the last card is red, what is the probability that the first card is red? – If it turns out that the last card is black, what is the probability that the first card is red? kshumENGG2012B15

16. Conditional probability If we are given an addition information that the outcome is in event B, then we can update the likelihood to |A  B|/ |B|. If Pr(B) is nonzero, then we define the conditional probability of A given B by Pr(A|B) = Pr(A  B)/Pr(B). kshumENGG2012B16  A B At the beginning, the probability of event A is |A| / |  |, assuming that all outcomes in  are equally likely.

17. The law of total probability in terms of conditional probability kshumENGG2012B17  A B C D Pr(E) = Pr(E  A)+ Pr(E  B)+ Pr(E  C)+ Pr(E  D) = Pr(E|A)Pr(A)+ Pr(E|B)Pr(B)+ Pr(E|C)Pr(C)+ Pr(E|D)Pr(D). A  B  C  D = , A, B, C, D mutually disjoint E

18. Example Alice rolls a fair dice two times. Suppose Bob is told that the face value of the first roll is 6. What is the probability the second roll is also 6? Suppose Carol is told that the face value of one of the roll is 6. What is the probability that the other roll is also 6? kshumENGG2012B18

19. Independent events If P(A|B) = P(A), then it means that the likelihood of the event A does not change after we are told that event B has occurred. In this case, event A is said to be independent of event B. As Pr(A|B) = Pr(A  B)/Pr(B), event A is independent of B if Pr(A  B) = Pr(A) Pr(B). Two events A and B are said to be independent, or statistically independent, if Pr(A  B) = Pr(A) Pr(B). kshumENGG2012B19

20. Example Roll two fair dice. Let A be the event that the face value of the first die is even. Let B be the event that the sum of the dice is odd. Are events A and B independent? kshumENGG2012B20 111213141516 212223242526 313233343536 414243444546 515253545556 616263646566 111213141516 212223242526 313233343536 414243444546 515253545556 616263646566 First number is even Sum is odd

21. Pairwise independent does not imply Pr(A  B  C)=Pr(A)Pr(B)Pr(C). Let  be the sample space {0,1,2,3}. The four outcomes are equally likely. Let A be the event {0,1}, B be the event {0,2}, and C be the event {0,3}. The three events A, B and C are pairwise independent, because – Pr(A) Pr(B) = (1/2) 2 = 1/4 = Pr(A  B) – Pr(B) Pr(C) = (1/2) 2 = 1/4 = Pr(B  C) – Pr(C) Pr(A) = (1/2) 2 = 1/4 = Pr(C  A) However, – Pr(A  B  C) = 1/4 – Pr(A) Pr(B) Pr(C) = (1/2) 3 =1/8 kshumENGG2012B21  1 2 3 0 A B C

22. Example There are two boxes. The first box contains 3 red balls and 7 blue balls. The second box contains 16 red balls and 4 blue balls. Perform the following random experiment – Throw a fair die. – If the number is 1, pick a ball randomly from the first box. – If the number is 2 to 5, pick a ball randomly from the second box. What is the probability that a red ball is drawn? What is the probability that a blue ball is drawn? Given that a red ball is picked, what is the probability that the first box is chosen? kshumENGG2012B22

23. The Bayes’ rule Let A and B be two events. Suppose that the probability of B is nonzero. Probability of A given B can be computed from the probability of B given A by kshumENGG2012B23 http://en.wikipedia.org/wiki/Bayes'_theorem

24. Thomas Bayes (1701-1761) English minister and mathematician http://en.wikipedia.org/wiki/Thomas_Bayes kshumENGG2012B24

25. The Monty Hall Problem Quote from wikipedia – http://en.wikipedia.org/wiki/Monty_Hall_problem http://en.wikipedia.org/wiki/Monty_Hall_problem Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice? An explanation from youtube – http://www.youtube.com/watch?v=mhlc7peGlGg http://www.youtube.com/watch?v=mhlc7peGlGg kshumENGG2012B25

26. Explanation using conditional probability We use – “100” to represent “prize behind the first door”. – “010” to represent “prize behind the second door”. – “001” to represent “prize behind the third door”. The host may need to flip a coin in order to open the door. We use “H” and “T” for the outcome of the coin toss. Set up a sample space  of six elements  = {100H, 100T, 010H, 010T, 001H, 001T}. Each outcome is equally probable. kshumENGG2012B26

27. Before any door is opened by the host Suppose you choose door 1 kshumENGG2012B27 100 010 001 1/3 The probability that there is a car behind the first door is 1/3.

28. After a door is opened Suppose you choose door 1. A door is opened by the host of the game. kshumENGG2012B28 100 010 001 1/3 H T H T H T 100 010 001 Door 3 is opened Door 2 is opened Door 3 is opened Door 2 is opened

29. Probability given that door 3 is opened Suppose you choose door 1. Suppose door 3 is opened Pr(Prize is behind door 1 | door 3 is opened) = 1/3 If you switching to door 2, you will win with probability 2/3 kshumENGG2012B29 100 010 001 1/3 H T H T H T 100 010 001 Door 3 is opened

30. Probability given that door 2 is opened Suppose you choose door 1. Suppose door 3 is opened Pr(Prize is behind door 1 | door 2 is opened) = 1/3 If you switching to door 3, you will win with probability 2/3 kshumENGG2012B30 100 010 001 1/3 H T H T H T 100 010 001 Door 2 is opened