ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 2 – Conditional probability Farinaz Koushanfar ECE Dept., Rice University Aug 27, 2009

ELEC 303, Koushanfar, Fall’09 Lecture outline Reading: Sections 1.3, 1.4 Review Conditional probability – Multiplication rule – Total probability theorem – Bayes rule

ELEC 303, Koushanfar, Fall’09 Probability theory -- review Mathematically characterizes random events Defined a sample space of possible outcomes Probability axioms: 1.(Nonnegativity) 0≤P(A)≤1 for every event A 2.(Additivity) If A and B are two disjoint events, then the probability P(A  B)=P(A)+P(B) 3.(Normalization) The probability of the entire sample space  is equal to 1, i.e., P(  )=1  A B

ELEC 303, Koushanfar, Fall’09 Discrete/continuous models -- review Discrete: finite number of possible outcomes – Enumerate the possible scenarios and count Continuous: the sample space is continuous – The probability of a point event is zero – Probability=area in the sample space

ELEC 303, Koushanfar, Fall’09 Conditional probability example 1 Student stage: (FR), (SO), (JU), (SI) Student standing: probation (P), acceptance (A), honor (H) Example courtesy of

ELEC 303, Koushanfar, Fall’09 Example (cont’d) What is the probability of a JU student? What is the probability of honors standing if the student is a FR? What is the probability of probation for a SU? P(A|B) is probability of A given B B is the new universe

ELEC 303, Koushanfar, Fall’09 Conditional probability Definition: Assuming P(B)  0, then P(A|B) = P(A  B) / P(B) Consequences: If P(A)  0 and P(B)  0, then P(A  B) = P(B).P(A|B) = P(A).P(B|A)  A B

ELEC 303, Koushanfar, Fall’09 Conditional probability example 2 1.What is the probability of both dices showing odd numbers given that their sum is 6? {1,5},{5,1},{2,4},{4,2},{3,3} {1,5},{5,1},{3,3} 2.Let B is the event: min (X,Y)=3, Let M = max{X,Y}. What are the probabilities for M over all of its possible values?

ELEC 303, Koushanfar, Fall’09 Conditional probability example 3 Radar detection vs. airplane presence What is the probability of having an airplane? What is the probability of airplane being there if the radar reads low? When should we decide there is an airplane and when should be decide there is none? Slide courtesy of Prof. Dahleh, MIT

ELEC 303, Koushanfar, Fall’09 Sequential description (A): aircraft present, (A c ): aircraft absent (L): low, (M): medium, (H): high P(A)=0.3 P(A c )=0.7 P(L|A)=0.066 P(M|A)=0.26 P(H|A)=0.66 P(L|A c )=0.643 P(M|A c )=0.286 P(H|A c )=0.071 Multiplication rule: Assuming that all of the conditioning events have positive probability,

ELEC 303, Koushanfar, Fall’09 Conditional probability example 4 Three cards are selected from a deck of 52 cards without replacement. Find the probability that none of the drawn cards is a picture, i.e., (J,Q,K)  drawn set

ELEC 303, Koushanfar, Fall’09 The Monty hall problem You are told that a prize is equally likely behind any of the 3 closed doors You randomly point to one of the doors A friend opens one of the remaining 2 doors, after ensuring that the prize is not behind it Consider the following strategies: – Stick to your initial choice – Switch to the other unopened door Picture courtesy of

ELEC 303, Koushanfar, Fall’09 Total probability theorem Divide and conquer Partition the sample space into A 1, A 2, A 3 For any even B: P(B) = P(A 1  B) + P(A 2  B) + P(A 3  B) = P(A 1 )P(B|A 1 )+P(A 2 )P(B|A 2 )+P(A 3 )P(B|A 3 ) A1A1 A2A2 A3A3 B B B BcBc BcBc BcBc A1BA1B A2BA2B A3BA3B Figure courtesy of Bertsekas&Tsitsiklis, Introduction to Probability, 2008

ELEC 303, Koushanfar, Fall’09 Radar example 3 (cont’d) P(Present) = 0.3 P(Medium|Present)=0.08/0.3 P(Present|low)=0.02/0.47 Example courtesy of Prof. Dahleh, MIT

ELEC 303, Koushanfar, Fall’09 Radar example 3 (cont’d) Given the radar reading, what is the best decision about the plane? Criterion for decision: minimize “probability of error” Decide absent or present for each reading Example courtesy of Prof. Dahleh, MIT

ELEC 303, Koushanfar, Fall’09 Radar example 3 (cont’d) Error={Present and decision is absent} or {Absent and decision is present} Disjoint events P(error)= Example courtesy of Prof. Dahleh, MIT

ELEC 303, Koushanfar, Fall’09 Extended radar example P(Threat)=Prior probability of threat = p Threat alert affects the outcome Example courtesy of Prof. Dahleh, MIT P(…|Threat) P(…|No threat)

ELEC 303, Koushanfar, Fall’09 Extended radar example A=Airplane, R=Radar reading If we let p=P(Threat), then we get: Example courtesy of Prof. Dahleh, MIT

ELEC 303, Koushanfar, Fall’09 Extended radar example Given the radar registered high and a plane was absent, what is the probability that there was a threat? How does the decision region behave as a function of p? Example courtesy of Prof. Dahleh, MIT

ELEC 303, Koushanfar, Fall’09 Extended radar example Example courtesy of Prof. Dahleh, MIT

ELEC 303, Koushanfar, Fall’09 Bayes rule The total probability theorem is used in conjunction with the Bayes rule: Let A 1,A 2,…,An be disjoint events that form a partition of the sample space, and assume that P(A i ) > 0, for all i. Then, for any event B such that P(B)>0, we have