1. 1 Review of Probability Theory [Source: Stanford University]

2. 2  A random experiment with set of outcomes  Random variable is a function from set of outcomes to real numbers Random Variable

3. 3 Example  Indicator random variable:  A : A subset of is called an event

4. 4 CDF and PDF  Discrete random variable:  The possible values are discrete (countable)  Continuous random variable:  The rv can take a range of values in R  Cumulative Distribution Function (CDF):  PDF and PMF:

5. 5 Expectation and higher moments  Expectation (mean):  if X>0 :  Variance:

6. 6 Two or more random variables  Joint CDF:  Covariance:

7. 7 Independence  For two events A and B :  Two random variables  IID : Independent and Identically Distributed

8. 8 Useful Distributions

9. 9 Bernoulli Distribution  The same as indicator rv:  IID Bernoulli rvs (e.g. sequence of coin flips)

10. 10 Binomial Distribution  Repeated Trials:  Repeat the same random experiment n times. (Experiments are independent of each other)  Number of times an event A happens among n trials has Binomial distribution  (e.g., number of heads in n coin tosses, number of arrivals in n time slots,…)  Binomial is sum of n IID Bernoulli rvs

11. 11 Mean of Binomial  Note that:

12. 12 Binomial - Example n=4 n=40 n=10 n=20 p=0.2

13. 13 Binomial – Example (ball-bin)  There are B bins, n balls are randomly dropped into bins.  : Probability that a ball goes to bin i  : Number of balls in bin i after n drops

14. 14 Multinomial Distribution  Generalization of Binomial  Repeated Trails (we are interested in more than just one event A )  A partition of  into A 1,A 2,…,A l  X i shows the number of times A i occurs among n trials.

15. 15 Poisson Distribution  Used to model number of arrivals

16. 16 Poisson Graphs =10 =.5 =1 =4

17. 17 Poisson as limit of Binomial  Poisson is the limit of Binomial(n, p) as  Let

18. 18 Poisson and Binomial Poisson(4) n=5,p=4/5 n=20, p=.2 n=10,p=.4 n=50,p=.08

19. 19 Geometric Distribution  Repeated Trials: Number of trials till some event occurs

20. 20 Exponential Distribution  Continuous random variable  Models lifetime, inter-arrivals,…

21. 21 Minimum of Independent Exponential rvs  : Independent Exponentials

22. 22 Memoryless property  True for Geometric and Exponential Dist.:  The coin does not remember that it came up tails l times  Root cause of Markov Property.

23. 23 Proof for Geometric

24. 24  Moment Generating Function (MGF)  For continuous rvs (similar to Laplace transform)  For Discrete rvs (similar to Z-transform): Characteristic Function

25. 25 Characteristic Function  Can be used to compute mean or higher moments:  If X and Y are independent and T=X+Y

26. 26 Useful CFs  Bernoulli( p ) :  Binomial( n, p ) :  Multinomial :  Poisson :