1 Schaum’s Outline PROBABILTY and STATISTICS Chapter 6 ESTIMATION THEORY Presented by Professor Carol Dahl Examples from D. Salvitti J. Mazumdar C. Valencia

2 Outline Chapter 6 Trader in energy stocks random variable Y = value of share want estimates µ y, σ Y Y = ß 0 + ß 1 X+  want estimates Ŷ, b 0, b 1 Properties of estimators unbiased estimates efficient estimates

3 Outline Chapter 6 Types of estimators Point estimates µ = 7 Interval estimates µ = 7+/-2 confidence interval

4 Outline Chapter 6 Population parameters and confidence intervals Means Large sample sizes Small sample sizes Proportions Differences and Sums Variances Variances ratios

5 Properties of Estimators - Unbiased Unbiased Estimator of Population Parameter estimator expected value = to population parameter

6 Unbiased Estimates Population Parameters: Sample Parameters: are unbiased estimates Expected value of standard deviation not unbiased

7 Properties of Estimators - Efficient Efficient Estimator – if distributions of two statistics same more efficient estimator = smaller variance efficient = smallest variance of all unbiased estimators

8 Unbiased and Efficient Estimates Target Estimates which are efficient and unbiased Not always possible often us biased and inefficient easy to obtain

9 Types of Estimates for Population Parameter Point Estimate single number Interval Estimate between two numbers.

10 Estimates of Mean – Known Variance Large Sample or Finite with Replacement X = value of share sample mean is $32 volatility is known σ 2 = $4.00 confidence interval for share value Need estimator for mean need statistic with mean of population estimator

11 Estimates of Mean- Sampling Statistic P(-1.96 < <1.96) = 95% 2.5%

12 Confidence Interval for Mean P(-1.96 < <1.96) = 95% P(  X < -µ <  X ) = 95% Change direction of inequality P(  X > µ >  X ) = 95%

13 Confidence Interval for Mean P(  X > µ >  X ) = 95% Rearrange P(  X < µ <  X ) = 95% Plug in sample values and drop probabilities X = value of share, sample = 64 sample mean is $32 volatility is σ 2 = $4 {32 – 1.96*2/  64, *2/  64} = {31.51,32.49}

14 Estimates for Mean for Normal Take a sample point estimate compute sample mean interval estimate – 0.95 (95%+) = ( )  X +/-1.96  X +/-Z c (Z<Z c) = = (1 – 0.05/2) 95% of intervals contain 5% of intervals do not contain

15 Estimates for Mean for Normal interval estimate – 0.95 (95%+) = ( )  X +/-Z c (Z<Z c) = = (1 – 0.05/2) interval estimate – (1-  ) %  X +/-Z c (Z<Z c) = (1 –  /2)  % of intervals don’t contain (1-  )% of intervals do contain (Z<Zc) = = (1 – 0.05/2)

16 Common values for corresponding to various confidence levels used in practice are: Confidence Interval Estimates of Population Parameters

17 Functions in EXCEL Menu Click on  Insert  Function or =confidence( ,stdev,n) =confidence(0.05,2,64)= 0.49  X+/-confidence(0.05,2,64) =normsinv(1-  /2) gives Z c value  X+/-normsinv(1-  /2) 32 +/- 1.96*2/  64 Confidence Interval Estimates of Population Parameters

18 Confidence intervalConfidence level Confidence Intervals for Means Finite Population (N) no Replacement

19 Evaluate density of oil in new reservoir 81 samples of oil (n) from population of 500 different wells samples density average is 29°API standard deviation is known to be 9 °API  = 0.05 Example: Finite Population without replacement

20 X = 29, N= 500, n = 81, σ = 9,  = 0.05 Z c = 1.96 Confidence Intervals for Means Finite Population (N) no Replacement Known Variance

21 = N(0.1) df  2/df = t df t-Distribution But don’t know Variance =

22 Confidence Intervals of Means t- distribution = = ==

23 Confidence Intervals of Means Normal compared to t- distribution Normal t distribution  X +/-Z c  X +/-t c

24 Example: Eight independent measurements diameter of drill bit 3.236, 3.223, 3.242, 3.244, 3.228, 3.253, 3.253, % confidence interval for diameter of drill bit Confidence Interval Unknown Variance  X +/-t c

25 Confidence Intervals for Means Unknown Variance  X = ΣX i /n  X = ŝ 2 = Σ(X i -  X) = (  X) (  X) 2 (n-1) (8-1) ŝ =  X +/-t c

26  X = 3.239, n = 8, ŝ = ,  =0.01, 1-  /2=0.995  From the t-table with 7 degrees of freedom, we find t c = t 7,0.995 =3.50 Confidence Intervals for Means Unknown Variance  X +/-t c.005% tctc -t c Find t c from Table of Excel 1-  /2=.975

27 Confidence Intervals for Means Unknown Variance  /2= 0.005% tctc -t c Depends on Table  1-  /2=.975 GHJ  /2 =  t c = Schaums 1-  /2 =  tc = 2.35 Excel =tinv(0.01,7) =

28  X = 3.239, n = 8, ŝ = ,  =0.01, Confidence Intervals for Means Unknown Variance  X +/-t c

engineers surveyed 250 in favor of drilling a second exploratory well 95% confidence interval for proportion in favor of drilling the second well Approximate by Normal in large samples Solution: n=600, X=250 (successes),  = 0.05 z c = 1.96 and Example Confidence Intervals for Proportions

engineers surveyed 250 in favor of drilling a second exploratory well. 95% confidence interval for proportion in favor of drilling the second well Approximate by Normal in large samples Solution: n=600, X=250 (successes),  = 0.05 z c = 1.96 and Example Confidence Intervals for Proportions

31 Confidence Intervals for Proportions sampling from large population or finite one with replacement

32 Confidence Intervals Differences and Sums Known Variances Samples are independent

33 Example sample of 200 steel milling balls average life of 350 days - standard deviation 25 days new model strengthened with molybdenum sample of 150 steel balls average life of 250 days - standard deviation 50 days samples independent Find 95% confidence interval for difference μ 1 -μ 2 Confidence Interval for Differences and Sums – Known Variance

34 Example Solution: X1=350, σ 1 =25, n 1 =200, X2=250, σ 2 =50, n 2 =150 Confidence Intervals for Differences and Sums

35 Where: P 1, P 2 two sample proportions, n 1, n 2 sizes of two samples Confidence Intervals for Differences and Sums – Large Samples

36 random samples 200 drilled holes in mine 1, 150 found minerals 300 drilled holes in mine 2, 100 found minerals c Construct 95% confidence interval difference in proportions Solution: P 1 =150/200=0.75, n1=200, P2=100/300=0.33,n2=300 Example With 95% of confidence the difference of proportions {0.42, 0.08} Confidence Intervals for Differences and Sums

37 Solution: P 1 =150/200=0.75, n 1 =200, P 2 =100/300=0.33, n 2 =300 Example 95% of confidence the difference of proportions [0.08, 0.42] Confidence Intervals Differences and Sums

38 Confidence Intervals for Variances Need statistic with population parameter  2 estimate for population parameter ŝ 2 its distribution -  2

39 Confidence Intervals for Variances has a chi-squared distribution n-1 degrees of freedom. Find interval such that σ lies in the interval for 95% of samples 95% confidence interval

40 Confidence Intervals for Variances Rearrange Take square root if want confidence interval for standard deviation

41 Confidence Intervals for Variances and Standard Deviations Drop probabilities when substitute in sample values 1 -  confidence interval for variance 1 -  confidence interval for standard deviation

42 Variance of amount of copper reserves 16 estimates chosen at random ŝ 2 = 2.4 thousand million tons Find 99% confidence interval variance Solution: ŝ 2 =2.4, n=16, degrees of freedom = 16-1= 15 Example Confidence Intervals for Variance

43 How to get  2 Critical Values  /2 Not symmetric  2 lower  2 upper

44 How to get  2 Critical Values  /2 1-  /2 GHJ area above  ,  , Schaums area below  ,  , 32.8 Excel = chiinv(0.995,15) = Excel = chiinv(0.005,15) = Not symmetric 1-  /2

45 99% confidence interval variance of reserves Solution: ŝ=2.4 (n-1)=15  2 lower = 4.60,  2 upper = 32.8 Example Confidence Intervals for Variances and Standard Deviations

46 Two independent random samples size m and n population variances estimated variances ŝ 2 1, ŝ 2 2 interested in whether variances are the same  2 1 /  2 2 Confidence Intervals for Ratio of Variances

47 Need statistic with population parameter  2 1 /  2 2 estimate for population parameter ŝ 2 1 / ŝ 2 2 its distribution - F Confidence Intervals for Ratio of Variances

48 F-Distribution df1 df2

49 F-Distribution

50 Need statistic with population parameter  2 1 /  2 2 estimate for population parameter ŝ 2 1 / ŝ 2 2 its distribution - F Confidence Intervals for Ratio of Variances

51 Confidence Intervals for Ratio of Variances Rearrange

52 Confidence Intervals for Ratio of Variances Put smallest first, largest second When substitute in values drop probabilities 1-  confidence interval for  2 1 /  2 2

53 Example Two nickel ore samples of sizes 16 and 10 unbiased estimates of variances 24 and 18 Find 90% confidence limits for ratio of variances Solution: ŝ 2 1 = 24, n 1 = 16, ŝ 2 2 = 18, n 2 = 10, Confidence Intervals for Variances

54 Confidence Intervals for Ratio of Variances  /2 F upper F lower GHJ area above F 0.95,15,9, F 0.05,15,9 ? 3.01 Schaums area below F 0.05,15,9, F 0.95,15,9 ? 3.01 Area above Excel = Finv(0.95,15,9) = Excel = Finv(0.05,15,9) = Tables df1  df2 

55 Confidence Intervals for Ratio of Variances  /2 F upper F lower GHJ area above F 0.95,15,9 P(F 15,9 >F c ) = 0.95 P(1/F 15,9 <1/F c ) = 0.95 But 1/F 15,9 = F 9,15 P(F 9,15 <1/Fc) = 0.95 P(F 9,15 <1/Fc) = /F c = 2.59  F c =

56 Example Two nickel ore samples Solution: ŝ 2 1 = 24, n 1 = 16, ŝ 2 2 = 18, n 2 = 10, Confidence Intervals for Variances

57 Maximum Likelihood Estimates Point Estimates x is population with density function f(x,  ) if know  - know the density function  2  where  = degrees of freedom Poisson λ x e -λ /x!  = λ (the mean) If sample independently from f n times x 1, x 2,...x n a sample if consider all possible samples of n a sampling distribution

58 Maximum Likelihood Estimates If sample independently from f n times x 1, x 2,...x n a sample if consider all possible samples of n a sampling distribution called likelihood function

59  which maximizes the likelihood function Derivative of L with respect to  and setting it to 0 Solve for  Usually easier to take logs first log(L) = log(f(x 1,  ) + log(f(x 2,  )+...+ log(f(x n,  ) Maximum Likelihood Estimates

60  log(L) =  log(f(x 1,  ) +  log(f(x 2,  )  log(f(x n,  )     Solution of this equation is maximum likelihood estimator work out example 6.25 work out example 6.26 Maximum Likelihood Estimates

61 Sum Up Chapter 6 Y = ß 0 + ß 1 X Ŷ, b 0, b 1 Properties of estimators unbiased estimates efficient estimates Types of estimators Point estimates Interval estimates

62 Sum Up Chapter 6 Y- µ Y,  Y,  Y, ŝ 2 In Y = ß 0 + ß 1 X Ŷ, b 0, b 1 Properties of estimators unbiased estimates efficient estimates Types of estimators Point estimates Interval estimates

63 Sum Up Chapter 6 Need statistic with population parameter estimate for population parameter its distribution

64 Sum Up Chapter 6 Population parameters and confidence intervals Mean – Normal Know variance and population normal Large sample size can use estimated variance

65 Sum Up Chapter 6 Proportions large sample approximate by normal Differences of means (known variance)

66 Sum Up Chapter 6 Mean population normal - unknown variance

67 Sum Up Chapter 6 Variances

68 Sum Up Chapter 6 Variances ratios

69 Sum Up Chapter 6 Maximum Likelihood Estimators Pick  which maximizes the function

70 End of Chapter 6!