1. Lecture 15 Parameter Estimation Using Sample Mean Last Time Sums of R. V.s Moment Generating Functions MGF of the Sum of Indep. R.Vs Sample Mean (7.1) Deviation of R. V. from the Expected Value (7.2) Law of Large Numbers (part of 7.3) Probability & Stochastic Processes Yates & Goodman (2nd Edition) NTUEE SCC_06_2009 15 - 1

2. Lecture 15: Parameter Estimation Using Sample Mean Today Law of Large Numbers (Cont.) Central Limit Theorem (CLT) Application of CLT The Chernoff Bound Point Estimates of Model Parameters Confidence Intervals Reading Assignment: 6.6 – 6.8, 7.3-7.4 Probability & Stochastic Processes Yates & Goodman (2nd Edition) NTUEE SCC_06_2009 15 - 2

3. Lecture 15: Parameter Estimation Using Sample Mean Next Time: Final Exam Scope: Chapters 4 – 7 Time: 15:30 -17:30 Reading Assignment: 6.6 – 6.8, 7.3-7.4 Probability & Stochastic Processes Yates & Goodman (2nd Edition) NTUEE SCC_06_2009 15 - 3

4. Law of Large Numbers: Strong and Weak Jakob Bernoulli, Swiss Mathematician, 1654-1705 [Ars Conjectandi, Basileae, Impensis Thurnisiorum, Fratrum, 1713 The Art of Conjecturing; Part Four showing The Use and Application of the Previous Doctrine to Civil, Moral and Economic Affairs Translated into English by Oscar Sheynin, Berlin 2005] Bernoulli and Law of Large Number.pdf S&WLLN.doc

5. Visualization of Law of Large Numbers 15 - 5

7. Who are they?

10. For Sum of iid Uniform RVs

11. For Sum of iid Binomial RVs

13. 15 - 13 Table 1: Normal Distribution Table (from Ulberg, 1987)

29. Probability and Stochastic Processes A Friendly Introduction for Electrical and Computer Engineers SECOND EDITION Roy D. YatesDavid J. Goodman Chapter 7 Parameter Estimation Using the Sample Mean

43. 12 - 43 Theorem 7.7 If X has finite variance, then the sample mean M N (X) is a sequence of consistent estimates of E[X].

46. Strong Law of Large Numbers Please refer to the supplementary material

50. 12 - 50 Theorem 7.10 E[V N (X)] = (n-1)/n Var[X]

59. Application to Histogram Construction Application of P(A) estimation to historgram construction

60. Approximation of CDF 1. Discretization of RV Values

69. Lecture 15: Parameter Estimation Using Sample Mean Next Time: Final Exam Scope: Chapters 4 – 7 Time: 15:30 -17:30 Reading Assignment: 6.6 – 6.8, 7.3-7.4 Probability & Stochastic Processes Yates & Goodman (2nd Edition) NTUEE SCC_06_2009 15 - 69