1. P robability 1 09. Limit Infinity 郭俊利 2010/05/11

2. Probability 2 Outline Estimation error Inequalities Large numbers Convergence Central limit theorem 4.6 ~ 5.4

3. Probability 3 Linear Estimator ^ X = E[X| Y] ≒ a Y + b I f the measurement Y = X 2, X is uniformly in [0, 2], what is the optimal estimate after a measured value = 2.25 ? ^ X = Y ½ ≒ 1/3 Y + 2/3 ^ X = 1.5 ≒ 1.416

4. Probability 4 Linear Least Estimator ^ X = E[X| Y] ≒ a Y + b a = cov(X, Y) / var(Y) = ρ(σ X / σ Y ) b = E[X] – a E[Y] E[ ~ X] = (1 – ρ 2 ) σ X 2 σ is standard deviations, ρ is correlation coefficient

5. Probability 5 Linear Least Squares Estimator X = aY + b X – aY = b E[b] = E[X – aY] ↓ b = E[X] – a E[Y] ↓ ~ X = X – ^ X Liner = X – aY – b E[ ~ X] = E[X – aY – b] ↓ = var(X – aY) ↓ a = cov(X, Y) / var(Y) = ρ(σ X / σ Y ) E[ ~ X] = (1 – ρ 2 ) σ X 2 E[(X – ^ X Liner ) Y] = 0

6. Probability 6 Linear Estimated Example (1/2) X = Y ½  ^ X Liner = 1/2 Y +1/3 a = ρ(σX / σY) ρ = cov(X, Y) / σXσY σ = √var = √E[ 2 ] – E[] 2, cov(X, Y) = E[XY] – E[X] E[Y] E[X] = 1, E[Y] = E[X 2 ] = 4/3, E[X 3 ] = 2, E[X 4 ] = 16/5 cov(X, Y) = 2 – 1*4/3 = 2/3 var(X) = 1/3, var(Y) ≒ 1.42 ρ ≒ 0.97  a ≒ 1/2 b = E[X] – a E[Y] = 1/3 ^ X Liner = 1/2 Y +1/3 E[ ~ X] = (1 – ρ 2 ) σ X 2 ≒ 0.02

7. Probability 7 Linear Estimated Example (2/2) 00.6666670.33333300.6666670.333333 0.010.670.3383330.10.570.238333 0.040.680.3533330.20.480.153333 0.090.6966670.3783330.30.3966670.078333 0.160.720.4133330.40.320.013333 0.250.750.4583330.50.25-0.04167 0.360.7866670.5133330.60.186667-0.08667 0.490.830.5783330.70.13-0.12167 0.640.880.6533330.80.08-0.14667 0.810.9366670.7383330.90.036667-0.16167 110.83333310-0.16667 1.211.070.9383331.1-0.03-0.16167 1.441.1466671.0533331.2-0.05333-0.14667 1.691.231.1783331.3-0.07-0.12167 1.961.321.3133331.4-0.08-0.08667 2.251.4166671.4583331.5-0.08333-0.04167 2.561.521.6133331.6-0.080.013333 2.891.631.7783331.7-0.070.078333 3.241.7466671.9533331.8-0.053330.153333 3.611.872.1383331.9-0.030.238333 422.333333200.333333 YX 1/3 Y + 2/31/2 Y +1/3 ~X~X

8. Probability 8 Inequalities (1/4) Markov Chebyshev When 0 ≤ E[X] ≤ a, Markov is informative; When 0 ≤ σ ≤ c, Chebyshev is informative. If Chebyshev is informative, Chebyshev is exacter than Markov P(X ≥ a) ≤ E[X] a P(|X – μ| ≥ c) ≤ σ2c2σ2c2

9. Probability 9 Inequalities (2/4) X is uniformly distributed in [0, 4] Exact probabilities P(X ≥ 2) = 0.5 P(X ≥ 3) = 0.25 P(X ≥ 4) = 0 Markov P(X ≥ 2) ≤ 1 P(X ≥ 3) ≤ 2/3 P(X ≥ 4) ≤ 1/2 Chebyshev P(X ≥ 3) ≤ 4/3 ! P(X ≥ 4) ≤ 1/3

10. Probability 10 Inequalities (3/4) Xiao-Ming can type X words/minute and X is random variable. The average is 50 words/minute. (a) P(X ≥ 75) ≤ Real probability < 1/4 (b) If σ X = 5, P(40 ≤ X ≤ 60) ≥

11. Probability 11 Inequalities (4/4) Both Markov and Chebyshev are suitable for exponential distribution, but not for normal distribution. X is exponentially distributed with λ = 1. For c > 1, to find P(X > c). Exact probability= e – c Markov≤ 1 / c Chebyshev≤ 1 / (c – 1) 2

12. Probability 12 Convergence (1/4) Sequence X n  Constant c n  ∞  lim X n = c M n = (Σ X i ) / n = S n / n E[ M n ] = μ; var( M n ) = σ 2 / n From Chebyshev P(| M n – μ| ≥ε) ≤  0 for all ε > 0, M n converges to μ σ2nε2σ2nε2

13. Probability 13 Convergence (2/4) lim P(| Y n – a | ≥ε) = 0 Xiao-Bao calculate Y n = min{X 1, …, X n }, X i is uniformly in [0, 1], find the converged value of Y n. n  ∞ P(|Y n – 0| ≥ ε) = P(X 1 ≥ ε, …, X n ≥ ε) = P(X 1 ≥ ε) … P(X n ≥ ε) = (1 – ε) n ∵ ∴ lim P(| Y n – 0 | ≥ε) = lim (1 – ε) n = 0 For all ε,ε = 0 ~ 1

14. Probability 14 Convergence (3/4) If Xiao-Bao guess wrong, he gets the converged value = ½, what happened after finishing calculating? P(|Y n – ½| ≥ ε) = P(Y n – ½ ≥ ε ∪ ½ – Y n ≥ ε) = P(Y n ≥ ε+ ½) + P(Y n ≤ ½ – ε) = (½ – ε) n + (½ – ε) n ∵ ε = 0 ~ ½ ∴ lim P(| Y n – 0 | ≥ε) when ε = ½ ~ 1 cannot support.

15. Probability 15 Convergence (4/4) Y n = max{X 1, …, X n }, X i is uniformly in [-1, 1], find the converged value of Y n. = P(Y n ≥ ε + 1) + P(Y n ≤ 1 – ε) = P(max(X i ) ≥ ε + 1) + P(max(X i ) ≤ 1 – ε) = P(max(X i ) ≤ 1 – ε) = [ P(X ≤ 1 – ε) ] n = (1 – ½ ε) n P(|Y n – 1| ≥ ε) ε = 0 ~ 1 lim (1 – ½ ε) n = 0

16. Probability 16 Central Limit Let X1, X2, … be a sequence of independent random variables with mean μ and variance σ 2 CDF Z n converges to the standard normal CDF Z n = X 1 + … + X n – nμ σ√n N(z) =1/ √2π ∫e –x /2 dx 2 lim ∞ P(Z n ≤ z) = N(z)

17. Probability 17 De Moivre - Laplace P(k ≤ S n ≤ l) ≒ N( ) – N( ) Xiao-Hua answers 100 questions and the probability of his correctness is 0.8. P(X ≥ 70) ≒ P(S ≥ 69.5) ≒ N(–2.875) P(X > 70) ≒ P(S ≥ 70.5) P(X ≤ 85) ≒ N(1.375) P(X = 88) ≒ N(8.5/4) – N(7.5/4) l – np +1/2 √np(1 – p) k – np – 1/2 √np(1 – p)