1 Development of a Valid Model of Input Data Collection of raw data Identify underlying statistical distribution Estimate parameters Test for goodness of fit

2 Identifying the Distribution Histograms Notes: Histograms may infer a known pdf or pmf. Example: Exponential, Normal, and Poisson distributions are frequently encountered, and less difficult to analyze. Probability plotting (good for small samples)

3 Sample Histograms (Figure 1) (1) Original Data - Too ragged Coarse, ragged, and appropriate histogram

4 Sample Histograms (cont.) (Figure 1) (2) Combining adjacent cells - too coarse Coarse, ragged, and appropriate histogram

5 Sample Histograms (cont.) (Figure 1) (3) Combining adjacent cells - appropriate Coarse, ragged, and appropriate histogram

6 Discrete Data Example The number of vehicles arriving at the northwest corner of an intersection in a 5-minute period between 7:00 a.m. and 7:05 a.m. was monitored for five workdays over a 20-week period. Following table shows the resulting data. The first entry in the table indicates that there were minute periods during which zero vehicles arrived, 10 periods during which one vehicle arrived, and so on.

7 Discrete Data Example (cont.) Arrivals per Period Frequencyper Period Frequency Since the number of automobiles is a discrete variable, and since there are ample data, the histogram can have a cell for each possible value in the range of data. The resulting histogram is shown in Figure 2

8 Histogram of number of arrivals per period (Figure 2) Number of arrivals per period

9 Continuous Data Example Life tests were performed on a random sample of 50 PDP-11 electronic chips at 1.5 times the normal voltage, and their lifetime (or time to failure) in days was recorded:

10 Continuous Data Example (cont.) Chip Life (Days) Frequency 0  x i <  x i <  x i <  x i <  x i <  x i <  x i <  x i <  x i <  x i <  x i <  x i <  x i <  x i <  x i <  x i < Electronic Chip Data

11 Continuous Data Example (cont.) (Figure 3) Histogram of chip life

12 Parameter Estimation The sample mean, X, is defined by --- (Eq 1) And the sample variance, S 2, is defined by --- (Eq 2)

13 Parameter Estimation (cont.) If the data are discrete and grouped in a frequency distribution, Eq1 and Eq2 can be modified to provide for much greater computational efficiency. The sample mean can be computed by --- (Eq 3) And the sample variance, S 2, is defined by --- (Eq 4)

14 Suggested Estimators for distr. often used in Simulation Distribution Parameter(s)Suggested Estimator(s) Poisson  X Exponential X Gamma  see(Table A.8)  X Uniform b b = {(n + 1) / n }  [max(X)] on (0, b) (unbiased) Normal    X   = S  (unbiased)

15 Suggested Estimators for distr. often used in Simulation Distribution Parameter(s)Suggested Estimator(s) Weibull    X / S with v = 0  j  j-1  f(  j-1 ) / f ‘(  j-1 ) Iterate until convergence  n  } 

16 Goodness-of-Fit Tests The Kolmogorov-Smirnov test and the chi-square test were introduced. These two tests are applied in this section to hypotheses about distributional forms of input data.

17 Goodness-of-Fit Tests Chi-Square Test This test is valid for large sample sizes, for both discrete and continuous distributional assumptions when parameters are estimated by maximum likelyhood. The test procedure begins by arranging the n observations into a set of k class intervals or cells. The test statistic is given by --- (Eq 5) where O i is the observed frequency in the ith class interval and E i is the expected frequency in that class interval.

18 Goodness-of-Fit Tests Chi-Square Test (cont.) The hypotheses are: H 0 : the random variable, X, conforms to the distributional assumption with the parameter(s) given by the parameter estimate(s) H 1 : the random variable, X, does not conform The critical value  is found in Table A.6. The null hypothesis, H 0, is rejected if

19 Goodness-of-Fit Tests Chi-Square Test (cont.) (Table 1) Recommendations for number of class intervals for continuous data Sample Size,Number of Class Intervals, nk 20Do not use the chi-square test 50 5 to to 20 >100  n to n/5

20 Goodness-of-Fit Tests Chi-Square Test (cont.) (Example:) (Chi-square test applied to Poisson Assumption) In the previous example, the vehicle arrival data were analyzed. Since the histogram of the data, shown in Figure 2, appeared to follow a Poisson distribution, the parameter,  = 3.64, was determined. Thus, the following hypotheses are formed: H 0 : the random variable is Poisson distributed H 1 : the random variable is not Poisson distributed

21 Goodness-of-Fit Tests Chi-Square Test (cont.) The pmf for the Poisson distribution was given:  e -   x  / x!, x = 0, 1, 2... p(x) =  (Eq 6)  0, otherwise For  = 3.64, the probabilities associated with various values of x are obtained using equation 6 with the following results. p(0) = p(3) = p(6) = p(9) = p(1) = p(4) = p(7) = p(10) = p(2) = p(5) = p(8) = p(11) = 0.001

22 Goodness-of-Fit Tests Chi-Square Test (cont.) Observed Frequency,Expected Frequency, (O i - E i ) 2 / E i x i O i E i (Table 2) Chi-square goodness-of fit test for example

23 Goodness-of-Fit Tests Chi-Square Test (cont.) With this results of the probabilities, Table 2 is constructed. The value of E 1 is given by np 1 = 100 (0.026) = 2.6. In a similar manner, the remaining E i values are determined. Since E 1 = 2.6 < 5, E 1 and E 2 are combined. In that case O 1 and O 2 are also combined and k is reduced by one. The last five class intervals are also combined for the same reason and k is further reduced by four.

24 Goodness-of-Fit Tests Chi-Square Test (cont.) The calculated  is The degrees of freedom for the tabulated value of  2 is k-s-1 = = 5. Here, s = 1, since one parameter was estimated from the data. At  = 0.05, the critical value is Thus, H 0 would be rejected at level of significance The analyst must now search for a better-fitting model or use the empirical distribution of the data.

25 Chi-Square Test with Equal Probabilities Continuous distributional assumption ==> Class intervals equal in probability P i = 1 / k since E i = nP i  5 ==> n / k  5(substitution) and solve for k yields k   n / 5

26 Chi-Square Test for Exponential Distribution (Example) Since the histogram of the data, shown in Figure3 (histogram of chip life), appeared to follow an exponential distribution, the parameter = 1/X = was determined. Thus, the following hypotheses are formed: H 0 : the random variable is exponentially distributed H 1 : the random variable is not exponentially distributed

27 Chi-Square Test for Exponential Distribution (cont.) In order to perform the chi-square test with intervals of equal probability, the endpoints of the class intervals must be determined. The number of intervals should be less than or equal to n/5. Here, n=50, so that k  10. In table 1, it is recommended that 7 to 10 class intervals be used. Let k = 8, then each interval will have probability p = The endpoints for each interval are computed from the cdf for the exponential distribution, as follows:

28 Chi-Square Test for Exponential Distribution (cont.) F(a i ) = 1 - e - a i (Eq 7) where a i represents the endpoint of the ith interval, i = 1, 2,..., k. Since F(a i ) is the cumulative area from zero to a i, F(a i ) = ip, so Equation 7 can be written as ip = 1 - e - a i or e - a i = 1 - ip

29 Chi-Square Test for Exponential Distribution (cont.) Taking the logarithm of both sides and solving for a i gives a general result for the endpoints of k equiprobable intervals for the exponential distribution, namely a i = {-1/ } ln(1 - ip), i = 0, 1,..., k(Eq 8) Regardless of the value of, equation 8 will always result in a 0 = 0 and a k = . With = and k = 8, a 1 is determined from equation 8 as a 1 = {-1/0.084}ln( ) = 1.590

30 Chi-Square Test for Exponential Distribution (cont.) Continued application of equation 8 for i = 2, 3,... 7 results in a 2,... a 7 as 3.425, 5.595, 8.252, , , and Since k = 8, a 8 =  The first interval is [0, 1.590), the second interval is [1.590, 3.425), and so on. The expectation is that of the observations will fall in each interval. The observations, expectations, and the contributions to the calculated value of  are shown in Table 3.

31 Chi-Square Test for Exponential Distribution (cont.) Class Observed Frequency,Expected Frequency, (O i - E i ) 2 / E i Intervlas O i E i [0, 1.590) [1.590, 3.425) [3.425, 5.595) [5.595, 8.252) [8.252, ) [11.677, ) [16.503, ) [24.755,  (Table 3) Chi-Square Goodness-of-fit test

32 Chi-Square Test for Exponential Distribution (cont.) The calculated value of  is The degrees of freedom are given by k - s - 1 = = 6. At  = 0.05, the tabulated value of  is Since, the null hypothesis is rejected. (The value of  is 16.8, so the null hypothesis would also be rejected at level of significance  = 0.01.)

33 Simple Linear Regression Suppose that it is desired to estimate the relationship between a single independent variable x and a dependent variable y. Suppose that the true relationship between y and x is a linear relationship, where the observation, y, is a random variable and x is a mathematical variable. The expected value of y for a given value of x is assumed to be E(y|x) =   +   x(Eq 9) where   = intercept on the y axis; an unknown constant;   = slope, or change in y for a unit change in x; an unknown constant.

34 Simple Linear Regression (cont.) It is assumed that each observation of y can be described by the model y =   +   x +  (Eq 10) where  is a random error with mean zero and constant variance  . The regression model given by equation 10 involves a single variable x and is commonly called a simple linear regression model.

35 Simple Linear Regression (cont.)

36 Simple Linear Regression (cont.)

37 Simple Linear Regression (cont.) The appropriate test statistic for significance of regression is given by t  =   /  (MS E /S xx ) where MS E is the mean squared error. The error is the difference between the observed value y i, and the predicted value, y i, at x i, or e i = y i - y i. The squared error is given by and the mean squared error, given by is an unbiased estimator of  2  = V(  i ).