1. Probability Refresher

2. Events Events as possible outcomes of an experiment Events define the sample space (discrete or continuous) – Single throw of a dice: {1, 2, 3, 4, 5, 6} Combinations (union & intersection) of events also define events – E = E1 OR E2 = E1  E2 – E = E1 AND E2 = E1  E2 – Mutually exclusive events: E1  E2 =  – Partition (of sample space  ): Set of mutually exclusive events that cover the entire sample space – Complementary events: E & F are complements of each other if E  F =  and E  F =  2

3. Events & Probabilities – (1) P{E}: Odds that event E occurs (P{  } = 1) Union Law: P{E  F} = P{E}+P{F}-P{E  F}  P{E  F} ≤ P{E}+P{F} with equality only if E and F are mutually exclusive (complementary events are mutually exclusive) Conditional probability – P{E|F} = P{E  F}/P{F}  P{E  F} = P{E|F}  P{F} Independence – E & F are independent if P{E  F} = P{E}  P{F} Conditional independence – E & F are conditionally independent given G if P{E  F|G} = P{E|G}  P{F|G} 3

4. Events & Probabilities – (2) Law of total probability: For any partition F 1,…,F N, of the sample space (  i F i =  ) Bayes Law – Prove using definition of conditional probability Combining Bayes Law and Law of total probability 4

5. Example – Anti-virus s/w Test We know that our s/w is 95% accurate, i.e., – P{positive | virus} = 0.95 (true positive) and P{negative | virus} = 0.05 – P{negative | no virus} = 0.95 (true negative) and P{positive | no virus} = 0.05 We also know that on average 1 out every 1,000 computers is infected with a virus, i.e., P{virus} = 0.001 What are the odds that a computer that tests positive is infected with a virus? – p = P{has virus| positive} – Bayes Law: p = [P{positive | virus}  P{virus}]/P{positive} – Bayes Law + Total Probability Law: Replace P{positive} with P{positive} = P{positive | virus}  P{virus}+P{positive | no virus}  P{no virus} P{positive} = 0.95  0.001+0.05  0.999 = 0.0509 – This gives p = 0.95  0.001/0.509 = 0.0187, i.e., less than 2% 5

6. Random Variables Basically a mapping of events to numbers – Typical notation: X (random variable), x (value) Cumulative distribution function (c.d.f.): F X (a) = P{X ≤ a} – Note that by definition F X (  ) = 1 Complementary distribution: F X (a) = 1- F X (a) = P{X > a} Discrete and continuous r.v.’s – Probability mass function (p.m.f) vs. probability density function (p.d.f.) – Expectation & higher moments Discrete r.v.: Continuous r.v.: Variance: Var(X) = E[(X – E[X]) 2 ] = E[X 2 ] – E 2 [X] (by linearity of expectation – more on this soon) 6

7. Joint Probability Discrete r.v.: Joint probability mass function P X,Y (x,y) – P X,Y (x,y) = P{X=x AND Y=y) – P X (x) = Σ y P X,Y (x,y) and P Y (y) = Σ x P X,Y (x,y) Continuous r.v.: Joint density function f X,Y (x,y) If X and Y are independent r.v.’s (X  Y) – Discrete: P X,Y (x,y) = P X (x)  P Y (y) – Continuous: f X,Y (x,y) = f X (x)  f Y (y) – E[XY] = E[X]E[Y] 7

8. Conditional Probabilities & Expectations Discrete r.v.: – Conditional p.m.f. of X given A – Conditional expectation of X given A Continuous r.v.: – Conditional p.d.f. – Conditional expectation of X given A 8

9. More on Expectation Expected value from conditional expectation – Discrete r.v.: E[X] = Σ y E[X|Y=y]P{Y=y} More generally: E[g(X)] = Σ y E[g(X)|Y=y]P{Y=y} – Continuous r.v.: E[X] =  y E[X|Y=y]f Y (y)dy More generally: E[g(X)] =  y E[g(X)|Y=y]P{Y=y} Linearity of expectation: E[X+Y] = E[X] + E[Y] Linearity of variance for independent r.v.’s – If X  Y then Var(X+Y) = Var(X) + Var(Y) 9

10. Random Sum of Random Variables Let X 1, X 2, X 3,… be i.i.d. random variables and N be a non-negative, integer-valued random variable, independent of the X i ’s Define Find expressions for E[S] and Var(S) – Condition on N and use linearity of expectation – For variance, use the fact that Var(S|N = n) = nVar(X) 10