Probability Refresher

Events Sample space  consists of all possible outcomes (discrete or continuous) of an experiment Single throw of a dice: {1, 2, 3, 4, 5, 6} Events are any sub-space of the sample space Combinations (union & intersection) of events are also events E = E1 OR E2 = E1  E2 – (OR  ) E = E1 AND E2 = E1  E2 – (AND  ) Mutually exclusive events: E1  E2 =  Partition (of sample space ): Set of mutually exclusive events that cover the entire sample space Complementary events: E & F are complements of each other if E  F =  and E  F = 

Events & Probabilities – (1) P{E}: Odds that event E occurs (P{} = 1) Union Law: P{EF} = P{E}+P{F}-P{EF} P{EF} ≤ P{E}+P{F} with equality only if E and F are mutually exclusive (complementary events are mutually exclusive) Conditional probability P{E|F} = P{EF}/P{F}  P{EF} = P{E|F}P{F} Fair dice example: 6 possible outcomes with the same probability of 1/6 Probability of a 2 given that outcome is an even number? (1/6)/(1/6+1/6+1/6) = 1/3 2 is one of three possible outcomes that yield an even number

Events & Probabilities – (2) Independence E & F are independent if P{EF} = P{E}P{F} Examples: Dice throwing P{ODD & Multiple of 3}= P{3}=1/6; and P{ODD}=1/2, P{Multiple of 3}=1/3, so that P{ODD}*P{Multiple of 3}=1/6 as well P{ODD & 5}=P{1}+P{3}+P{5}=1/2; and P{ODD}=1/2, P{5}=P{1}+P{2}+P{3}+P{4}+P{5}=5/6, so that P{ODD}*P{5}=5/121/2 Conditional independence E & F are conditionally independent given G if P{EF|G} = P{E|G}P{F|G} P{≥2 & 4|EVEN}=P{2|EVEN}+P{4|EVEN}=2/3; and P{≥2 |EVEN}=1, P{4|EVEN}=2/3, so that P{≥2 |EVEN}*P{4|EVEN}=2/3 as well Note that P{≥2 & 4}=1/2P{4}*P{≥2)=4/6*5/6=5/9

Events & Probabilities – (3) Law of total probability: For any partition F1,…,FN, of the sample space (iFi = ) Bayes Law Prove using definition of conditional probability Combining Bayes Law and Law of total probability

Bayes Law Conditional probability P{E|F} = P{EF}/P{F}  P{EF} = P{E|F}P{F} P{F|E} = P{EF}/P{E} Using the expression for P{EF} from (1) in (2) gives P{F|E} = P{E|F}P{F}/P{E}

Example – Anti-virus s/w Test We know that our s/w is 95% accurate, i.e., P{positive | virus} = 0.95 (true positive) and P{negative | virus} = 0.05 P{negative | no virus} = 0.95 (true negative) and P{positive | no virus} = 0.05 We also know that on average 1 out every 1,000 computers is infected with a virus, i.e., P{virus} = 0.001 What are the odds that a computer that tests positive is infected with a virus? p = P{has virus| positive} Bayes Law: p = [P{positive | virus}P{virus}]/P{positive} Bayes Law + Total Probability Law: Replace P{positive} with P{positive} = P{positive | virus}P{virus}+P{positive | no virus} P{no virus} P{positive} = 0.950.001+0.050.999 = 0.0509 This gives p = 0.950.001/0.509 = 0.0187, i.e., less than 2%

Random Variables Basically a mapping of events to numbers Typical notation: X (random variable), x (value) Cumulative distribution function (c.d.f.): FX(a) = P{X ≤ a} Note that by definition FX() = 1 Complementary distribution: FX(a) = 1- FX(a) = P{X > a} Discrete and continuous r.v.’s Probability mass function (p.m.f) vs. probability density function (p.d.f.) Expectation & higher moments Discrete r.v.: Continuous r.v.: Variance: Var(X) = E[(X – E[X])2] = E[X2] – E2[X] (by linearity of expectation – more on this soon)

Joint Probability Discrete r.v.: Joint probability mass function PX,Y(x,y) PX,Y(x,y) = P{X=x AND Y=y) PX(x) = ΣyPX,Y(x,y) and PY(y) = ΣxPX,Y(x,y) Continuous r.v.: Joint density function fX,Y(x,y) If X and Y are independent r.v.’s (X  Y) Discrete: PX,Y(x,y) = PX(x)PY(y) Continuous: fX,Y(x,y) = fX(x)fY(y) E[XY] = E[X]E[Y]

Conditional Probabilities & Expectations Discrete r.v.: Conditional p.m.f. of X given A Conditional expectation of X given A Continuous r.v.: Conditional p.d.f.

More on Expectation Expected value from conditional expectation Discrete r.v.: E[X] = ΣyE[X|Y=y]P{Y=y} More generally: E[g(X)] = ΣyE[g(X)|Y=y]P{Y=y} Continuous r.v.: E[X] = yE[X|Y=y]fY(y)dy More generally: E[g(X)] = yE[g(X)|Y=y]P{Y=y} Linearity of expectation: E[X+Y] = E[X] + E[Y] Linearity of variance for independent r.v.’s If X  Y then Var(X+Y) = Var(X) + Var(Y)

Random Sum of Random Variables Let X1, X2, X3,… be i.i.d. random variables and N be a non-negative, integer-valued random variable, independent of the Xi’s Define Find expressions for E[S] and Var(S) Condition on N and use linearity of expectation For variance, use the fact that Var(S|N = n) = nVar(X)

Mean of Random Sum of Random Variables

Variance of Random Sum of Random Variables Start with Var(S|N=n) So that Hence