1. COMP 170 L2 L18: Random Variables: Independence and Variance Page 1

2. COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 2

3. COMP 170 L2 Distribution Functions of RVs l P(X=k) viewed as a function of k: The distribution function of X. l In general Page 3

4. COMP 170 L2 Probability Weight and Distribution Function Page 4

5. COMP 170 L2 Distribution Functions of RVs Page 5

6. COMP 170 L2 Distribution Functions of RVs Page 6 l For coin example n D(1, 9) = P(1 <= X <=9 ) ~= 1

7. COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 7

8. COMP 170 L2 Independence Between Events l E is independent of F If P(E|F) = P(E) n The information that “Event F occurred” does not change the probability of E

9. COMP 170 L2 Random Variable and Event l Given a rand variable, we can define many difference events Page 9

10. COMP 170 L2 Independence between RVs Page 10

11. COMP 170 L2 Independence between RVs Page 11

12. COMP 170 L2 Independence between RVs Page 12 l No need to further check n P(X=0, Y=1) ?= P(X=0)P(Y=1) n P(X=1, Y=0) ? = P(X=1)P(Y=0) n P(X=1, Y=1) ?= P(X=1)P(Y=1)

13. COMP 170 L2 Independence between RVs l Draw two cards from a deck of 52 cards n X: number on first card n Y: number on second card n X and Y are independent when drawing with replacement n X and Y are not independent when drawing without replacement Page 13

14. COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 14

15. COMP 170 L2 Independence and Expectation Page 15

16. COMP 170 L2 Independence and Expectation Page 16

17. COMP 170 L2 Page 17

18. COMP 170 L2 Page 18

19. COMP 170 L2 Illustrating Proof via Example Page 19

20. COMP 170 L2 Illustrating Proof via Example Page 20

21. COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Sum of independent RVs n The Trend n Central limit theorem Page 21

22. COMP 170 L2 Variance of RVs l Probability starts with a process whose outcome is uncertain l Sample space: the set of all possible outcomes l A random variable (RV) is a function defined on the sample space l Different runs of the process might yield different outcomes l The RV might take different values in different runs n In other words, the value of RV varies across different runs l Some RVs vary more and some vary less n Number of heads in 1 coin flip n Number of heads in 10 coin flips n Number of heads in 100 coin flips Page 22

23. COMP 170 L2 Variance of RVs l Variance of an RV X n Measures how much it varies (across different runs of process) n Relative to the mean value Page 23

24. COMP 170 L2 Page 24

25. COMP 170 L2 Page 25

26. COMP 170 L2 Which RV vary the most? Flip fair coin l Number of heads in 1 flip l Number of heads in 10 flips l Number of heads in 100 flips Page 26 l Variance 1/4 10/4 100/4

27. COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 27

28. COMP 170 L2 Calculating Variance Page 28

29. COMP 170 L2 Example 1: We have already seen l Number of heads in n flips n n=1 n n=10 (x1+X2+…+X10) n n=100 (X1+X2+…+X100) Page 29 l Variance 1/4 10/4 = 10 * 1/4 100/4 = 100 * 1/4 l Additivity is true here.

30. COMP 170 L2 Page 30 l Additivity is true here.

31. COMP 170 L2 Example 3 (Counter Example) Page 31

32. COMP 170 L2 Example 3 (Counter Example) Page 32

33. COMP 170 L2 Page 33

34. COMP 170 L2 Two Lemmas Page 34

35. COMP 170 L2 Page 35

36. COMP 170 L2 Page 36

37. COMP 170 L2 Page 37 An Application of Theorem 5.29

38. COMP 170 L2 A Corollary Page 38

39. COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 39

40. COMP 170 L2 Standard Deviation l Standard deviation is another measure of how much a rv varies or how much a distribution spread out l It is the square root of variance. Page 40 l Next page l Shows several distributions, variances and standard deviations l Highlights the differences between variance and standard deviation

41. COMP 170 L2 Distributions, Variances, and Standard Deviations Page 41

42. COMP 170 L2 l The examples on the previous page show n Standard deviation is a natural measure of “spread” of distribution l Variance is easier to manipulate mathematically. Will see this in further study of probability theory and statistics. Page 42 Distributions, Variances, and Standard Deviations

43. COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 43

44. COMP 170 L2 A Pattern l If we flip of a coin a large number of times, n “The number of heads” has bell-shaped distribution. l This phenomenon is not unique to coin flipping Page 44

45. COMP 170 L2 Page 45 Test with.8 probability of getting correct answer

46. COMP 170 L2 Page 46

47. COMP 170 L2 Page 47

48. COMP 170 L2 Normal Distribution Page 48

49. COMP 170 L2 Page 49

50. COMP 170 L2 Page 50

51. COMP 170 L2 Recap: 13-05-2010

52. COMP 170 L2 Recap: 13-05-2010 l E is independent of F If P(E|F) = P(E) n The information that “Event F occurred” does not change the probability of E

53. COMP 170 L2 Recap: 13-05-2010 Flip fair coin l Number of heads in 1 flip: Variance = 1/4 l Number of heads in 10 flips: Variance = 10/4 l Number of heads in 100 flips: Variance = 100/4

54. COMP 170 L2 Number of times until first success l Throw a fair die n How many times, on average, do you need to throw the die until you see a 1? n P(getting 1 at each throw) = 1/6 n Answer: 1/(1/6) = 6 Page 54

55. COMP 170 L2 Number of times until first success l Throw a fair die n How many times, on average, do you need to throw the die until you see a 2 and 3 in that order? n Expected number of throws to see 2: 6 n After that, expected number of throws to see 3: 6 n Answer: 6+6 = 12 Page 55

56. COMP 170 L2 Number of times until first success l Throw a fair die n How many times, on average, do you need to throw the die until you see a 2 and 3, where the order does not matter? n P( get 2 or 3 in one throw ) = 1/3 n Expected number throws until you see one of 2 or 3: 3 n After that, P( get the other number in each throw) = 1/6 n Expected number of throws until you see the other number: 6 n Answer: 3+6 = 9 Page 56

57. COMP 170 L2 Old Exam Question 1 Page 57 P( at least one copy of T1 in 20 weeks) = 1 – P( no copy of T1 in 20 weeks) = 1 – P ( no T1 in week1 AND no T1 in week 2 AND …) = 1- P(no T1 in week1) P(no T1 in week 2) … =

58. COMP 170 L2

59. l Can we do this? P(all 10 types of toys in 20 weeks) = P(copy of T1 in 20 weeks AND copy of T2 in 20 weeks AND …) = P(copy of T1 in 20 weeks) P(copy of T2 in 20 weeks)… NO, events not independent l Correct way Page 59 P(all 10 types of toys in 20 weeks) = 1 – P(A) Use inclusion-exclusion to calculate P(A)

60. COMP 170 L2

61. Page 61 Old Exam Question 2

62. COMP 170 L2