1. South Dakota School of Mines & Technology Introduction to Probability & Statistics Industrial Engineering

2. South Dakota School of Mines & Technology Estimation Industrial Engineering

3. Estimation Point Estimates Industrial Engineering Estimation Point Estimates Industrial Engineering

4. Overview u Point versus interval estimates u Estimators of a population mean u Estimators of a population proportion u Estimators of a population variance u Estimating other parameters Method of Moments Maximum Likelihood Estimates

5. Statistics u Descriptive Statistics is used to summarize a collection of data in a clear and understandable way. Ex: Histograms, box plots, stem and leaf plots. u Inferential Statistics are used to draw inferences about a population from a sample. Ex: 10 subjects perform a task after 3 hours of training. They score 12 points higher than 10 subjects who perform the same task with no training. Is the difference real or could it be due to chance?

6. Point Estimators u Suppose we have a new type of light bulb and we wish to test the bulbs for mean time to burn out.

7. Point Estimators u We select 10 bulbs at random and test to burn out.

8. Point Estimators    n i i n x X 1 10 1462...14831569   = 1,596.2

9. Point Estimators  X 1,596.2 Is called a point estimator of the true but unknown mean .

10. Point Estimators u Suppose we now select a new sample of ten bulbs.

11. Point Estimators X 10 1571...15671739   = 1,603.0

12. Point Estimators X 10 1571...15671739   = 1,603.0 Different samples yield different point estimates of the population mean.

14. South Dakota School of Mines & Technology Estimation Industrial Engineering

15. Estimation Interval Estimates Industrial Engineering Estimation Interval Estimates Industrial Engineering

16. Interval Estimates  Suppose our light bulbs have some underlying distribution f(x) with finite mean  and variance  2. Regardless of the distribution, recall from that central limit theorem that X n ),(N  

17. Interval Estimates u Recall that for a standard normal distribution,  z  /2  z  /2  /2 1 -  )(1 2/2/   zZzP 

18. Interval Estimates But, so, Then, X n ),(N   )1,0(N n X Z      x )(1 2/2/     z n zP   

19. Interval Estimates x )(1 2/2/     z n zP    n )( 2/2/ n zxzP      n xx n )( 2/2/ zzP     

20. 1 -  n xx n )( 2/2/ zzP      xx)( 2/2/ n z n zP     

21. x Interval Estimates 1 -  xx)( 2/2/ n z n zP      In words, we are (1 - a)% confident that the true mean lies within the interval n z   2/ 

22. Example  Suppose we know that the variance of the bulbs is given by  2 = 10,000. A sample of 25 bulbs yields a sample mean of 1,596. Then a 90% confidence interval is given by 25 100 645.1596,1  9.32596,1 

23. Example or 1,563.1 <  < 1,628.9 1,563.1 1,596 1,628.9 32.9

24. Example or 1,563.1 <  < 1,628.9 1,563.1 1,596 1,628.9 32.9 32.9 is called the precision (E) of the interval and is given by n zE   2/ 

25. Example u Suppose we repeat this process 4 times and get 4 sample means of 1596, 1578, 1612, and 1584. Computing confidence intervals then gives 1,612 1,596 1,578 1,584

26. Interpretation  Either the mean is in the confidence interval or it is not. A 90% confidence interval says that if we construct 100 intervals, we would expect 90 to contain the true mean  and 10 would not. 1,612 1,596 1,578 1,584

27. A Word on Confidence Int. u Suppose instead of a 90% confidence, we wish to be 99% confident the mean is in the interval. Then 25 100 575.2596,1  5.51596,1 

28. A Word on Confidence Int. u That is, all we have done is increase the interval so that we are more confident that the true mean is in the interval. 1,563.1 1,596 1,628.9 32.9 1,544.5 1,596 1,647.5 51.5 90% Confidence 99% Confidence

29. Sample Sizes u Suppose we wish to compute a sample size required in order to have a specified precision. In this case, suppose we wish to determine the sample size required in order to estimate the true mean within + 20 hours.

30. Sample Sizes u Recall the precision is given by Solving for n gives n zE   2/  2 2/        E z n  

31. Sample Sizes u We wish to determine the sample size required in order to estimate the true mean within + 20 hours with 90% confidence. 6865.67 20 )100(645.1 2         n

32. Sample Sizes u We wish to determine the sample size required in order to estimate the true mean within + 20 hours with 90% confidence. 6865.67 20 )100(645.1 2         n Greater precision requires a larger sample size.

34. South Dakota School of Mines & Technology Estimation Industrial Engineering

35. Estimation Interval Estimates (  unknown) Industrial Engineering Estimation Interval Estimates (  unknown) Industrial Engineering

36. Confidence Intervals  unknown u Suppose we do not know the true variance of the population, but we can estimate it with the sample variance. x 1 1 2 2 2      n nx s n i i

37. Confidence Intervals (  unknown) u Suppose we do not know the true variance of the population, but we can estimate it with the sample variance. For large samples (>30), replace  2 with s 2 and compute confidence interval as before. x 1 1 2 2 2      n nx s n i i

38. Confidence Intervals (  unknown) u For small samples we need to replace the standard normal, N(0,1), with the t- distribution. Specifically, n 1     n t s x t 

39. Confidence Interval (  unknown)  t n-1,  /2  t n-1,  /2  /2 1 -  t n-1 n 1     n t s x t 

40. Confidence Interval (  unknown)  t n-1,  /2  t n-1,  /2  /2 1 -  t n-1 n 1     n t s x t  Assumption: x is normally distributed

41. Confidence Interval (  unknown) )(1 2/,12/,1        nn t n s x tP  t n-1,  /2  t n-1,  /2  /2 1 -  t n-1

42. Confidence Interval (  unknown) )(1 2/,12/,1        nn t n s x tP n x s t n 2/,1   Miracle 17b occurs

43. Example u Suppose in our light bulb example, we wish to estimate an interval for the mean with 90% confidence. A sample of 25 bulbs yields a sample mean of 1,596 and a sample variance of 10,000. n x s t n 2/,1   25 100 711.1596,1 

44. Example u Suppose in our light bulb example, we wish to estimate an interval for the mean with 90% confidence. A sample of 25 bulbs yields a sample mean of 1,596 and a sample variance of 10,000. n x s t n 2/,1   25 100 711.1596,1  1,596 + 34.2

45. Example u Note that lack of knowledge of s gives a slightly bigger confidence interval (we know less, therefore we feel less confident about the same size interval). 1,563.1 1,596 1,628.9 32.9 1,561.8 1,596 1,630.2 34.2  known  unknown

46. A Final Word -4.00-2.000.002.004.00 N(0,1) t 10

47. A Final Word -4.00-2.000.002.004.00 N(0,1) t 20

48. A Final Word -4.00-2.000.002.004.00 N(0,1) t 30

49. A Final Word u Note that on the t-distribution chart, as n becomes larger, hence, for larger samples (n > 30) we can replace the t-distribution with the standard normal. 2/2/,1  zt n  

51. South Dakota School of Mines & Technology Estimation Industrial Engineering

52. Estimation Estimates for Proportions Industrial Engineering Estimation Estimates for Proportions Industrial Engineering

53. Estimating a Proportion u Suppose we sample 100 circuit boards and find that 8 are defective. We would like to make an inference about the true percentage defective given a sample defective of p = 0.08.

54. Estimating a Proportion u Suppose we sample 100 circuit boards and find that 8 are defective. We would like to make an inference about the true percentage defective given a sample defective of p = 0.08. Recall that for a large sample (n>30) the binomial may be approximated by the normal distribution. We also know that the mean of the binomial is np and the variance is npq. ),(npqnpNx 

55. Estimating a proportion Now if then, or ),(npqnpNx  n x p  ˆ ),( ˆ npqnpNpn  )1,0( ˆ N npq nppn  

56. Estimating a Proportion Divide through by n and replace pq by gives )1,0( ˆ N npq nppn   qp ˆˆ )1,0( / ˆˆ ˆ N nqp pp  

57. Estimating a Proportion )1,0( / ˆˆ ˆ N nqp pp   ) / ˆˆ ˆ (1 2/2/   z nqp pp zP   

58. ) / ˆˆ ˆ (1 2/2/   z nqp pp zP    Miracle 21c occurs nqpzp/ ˆˆˆ 2/  

59. Example u Returning to our circuit board example, suppose a sample of 100 boards yields 8% defective. Compute a 90% confidence interval for the true but unknown proportion defective. nqpzp/ ˆˆˆ 2/   100/)92(.08.645.108.0 

60. Example u Returning to our circuit board example, suppose a sample of 100 boards yields 8% defective. Compute a 90% confidence interval for the true but unknown proportion defective. 0.035 < p < 0.125

62. South Dakota School of Mines & Technology Estimation Industrial Engineering

63. Estimation Interval Estimates (variance) Industrial Engineering Estimation Interval Estimates (variance) Industrial Engineering

64. Estimator for a Variance  Suppose in a sample of 25 light bulds, we compute a sample variance of 10,000. We would now like to make an inference about the true but unknown population variance  2. If the underlying distribution is normal, then the distribution of the sample variance is chi- square.

65. Estimator for a Variance  /2   2 n-1  2 n-1,  /2  2 n-1,1-  /2 2 1 2 2 )1(   n s n  

66. Estimator for Variance ))1((1 2 2/1,1 2 2 2 2/,1       nn s nP Miracle 21c occurs 2 2/,1 2 2 2 2/1,1 2 )1()1(         nn snsn

67. Example u Suppose in our sample of 25 light bulbs we compute a sample variance of 10,000. Compute a 90% confidence for the true variance.

68. Example 2 2/,1 2 2 2 2/1,1 2 )1()1(         nn snsn 484.13 )000,10(24 415.36 )000,10(24 2 

69. Example 6,591 <  2 < 17,799 484.13 )000,10(24 415.36 )000,10(24 2 

70. Example 6,591 <  2 < 17,799 484.13 )000,10(24 415.36 )000,10(24 2  Note that the confidence interval for  2 is not symmetric. 6,591 10,000 17,799

71. Summary u To make probabilistic statements about  (  known)N(0,1)  (  unknown)t n-1 normal  (  unknown)N(0,1)n >> 30  2 given s 2 normal  1 2 given  2 2 F n1-1,n2-1 normal p N(0,1)n >> 30 2 1  n 

73. South Dakota School of Mines & Technology Estimation Industrial Engineering

74. Estimation Method of Moments Industrial Engineering Estimation Method of Moments Industrial Engineering

75. Method of Moments u Recall from Data Analysis, we had three measures for failure time data s 2 = 302.76 X1.19 

76. Method of Moments u For Failure Time data, we now have three measures for the data Exponential ?? s 2 = 302.76 X1.19 

77. Method of Moments u Recall that for the exponential distribution  = 1/ s 2 = 1/ 2 If E[ X ] =  and E [s 2 ] = s 2, then 1/ = 19.1 s 2 = 302.76 X1.19  0524. ˆ 

78. Method of Moments u Recall that for the exponential distribution  = 1/ s 2 = 1/ 2 If E[ X ] =  and E [s 2 ] = s 2, then 1/ = 19.1 or 1/ 2 = 302.76 s 2 = 302.76 X1.19  0575. ˆ  0524. ˆ 

79. Estimation Maximum Likelihood Estimates Industrial Engineering Estimation Maximum Likelihood Estimates Industrial Engineering

80. Discrete Case u Suppose we have hypothesized a discrete distribution from which our data which has some unknown parameter. Let denote the probability mass function for this distribution. The likelihood function is  px  () pxpxpx n ()()()L()    12

81. Discrete Case u Suppose we have hypothesized a discrete distribution from which our data which has some unknown parameter. Let denote the probability mass function for this distribution. The likelihood function is  px  () pxpxpx n ()()()L()    12 L()  is just the joint probability mass function

82. Discrete Case u Since is just the joint probability, we want to choose some which maximizes this joint probability mass function. L()    LLforallpossible(  )() 

83. Continuous Case  Suppose we have a set of nine observations x 1, x 2,... X 9 which have underlying distribution exponential (in this case scale parameter = 2.0). 0.0530.458 0.1120.602 0.1780.805 0.2551.151 0.347

84. Continuous Case  Suppose we have a set of nine observations x 1, x 2,... X 9 which have underlying distribution exponential (in this case scale parameter = 2.0). Our object is to estimate the true but unknown parameter. Lfxfxfx n ()()()()  12

85. MLE (Exponential) Lfxfxfx n ()()()()  12    e n x i   eee xxx n12

86. MLE (Exponential) Lfxfxfx n ()()()()  12    e n x i   eee xxx n12   93961 e.

87. MLE (Exponential)

88.  We can use the plot to graphically solve for the best estimate of. Alternatively, we can find the maximum analytically by using calculus. Specifically,   L() ()  0

89. Log Likelihood u The natural log is a monotonically increasing function. Consequently, maximizing the log of the likelihood function is the same as maximizing the likelihood function itself. LnLNL()() 

90. MLE (Exponential) Lfxfxfx n ()()()()  12    e n x i

91. MLE (Exponential) Lfxfxfx n ()()()()  12    e n x i Lnnlnx i ()()  

92. MLE (Exponential) Lnnlnx i ()()       Ln nx i ()ln()     n x i = 0

93. MLE (Exponential) n x i   n x i   0

94. MLE (Exponential) n x i   n x i   0x  1

95. MLE (Exponential) n x i   n x i   0x  1 x  1 

96. MLE (Exponential) x  1  iX i 10.053 20.112 30.178 40.255 50.347 60.458 70.602 80.805 91.151 Sum =3.961 X-bar =0.440

97. MLE (Exponential) x  1  iX i 10.053 20.112 30.178 40.255 50.347 60.458 70.602 80.805 91.151 Sum =3.961 X-bar =0.440 ..  1 044 227

98. Experimental Data u Suppose we wish to make some estimates on time to fail for a new power supply. 40 units are randomly selected and tested to failure. Failure times are recorded follow: X1.19 

99. Failure Data