1. MS 305 Recitation 11 Output Analysis I 16.05.2013

2. Half Width 2  We enforce an upper bound on error h, satisfying the following inequality:  When the above probability is equal to (1 - α), h is called as half-width.  In our analyses, we make use of the fact that the statistic below follows a t-distribution:  The half width can be computed by plugging T into the equation below.

3. Half Width 3  Then, the formula for half-width is given by:  Keep in mind that the t-distribution assumption is only valid when the outputs are independent, identically distributed, and follows a normal distribution.  i.i.d. assumption is justified by ARENA since we are choosing initialize statistics and initialize system.  Normality assumption is justified using the Central Limit Theorem. Half-width =

4. Number of Replications 4

5. 5  Once the approximate number of replications is obtained, you should re-run the model in order to check whether the desired half-width is obtained.  Example:  Suppose that the initial number of replication is 10 and it leads to an initial s² value,  Using this we compute n,  We re-run the model,  If the calculated half-width is less than or equal to h (the specified value), we are done,  Otherwise we take the recent s² as our initial guess and reuse the approximation to recalculate the new value of n…

6. Number of Replications 6  A reasonable strategy to specify an upper bound on half width (h) would be to use a proportion of the sample mean.  i.e. where γ denotes the percentage of deviation from sample mean  When the formula of half-width is given, you should know how to approximate the number of replications.  In other words, the approximation formulas will not be given in the exam since they are intuitive and can be obtained by simply manipulating the half-width formula.

7. Exercise 6.03 7  In exercise 5.03, about how many replications would be required to bring the half-width of a 95% confidence interval for the expected average cycle time for both trimmers down to one minute?  Start with 10 replications.

8. Exercise 6.03 8  We run the simulation with 10 replications and obtain the half-width associated with the 95% confidence interval.  Important: α=0.05 is the default value used in the half-width computation of Arena.  The half-widths for the expected average cycle time for primary and secondary trimmer are 2.11, and 14.77, respectively.  To reduce the half-widths to 1, we work with worst (14.77) and compute the approximate number of required replications (using the second approximation for sample size) as (10)x(14.77/1)²= 2181.529.  Rounding up to the nearest integer gives 2182 replications.

9. Confidence Interval 9  Using the results on half width, we can construct a 100(1-α)% confidence interval on mean as below: where

10. Statistical Comparison of Outputs 10  In output analysis, we are often concerned with the comparison of outcomes obtained from different sources.  Examples for these sources could be:  Two simulation models that could represent the current system and a modified version of the same system,  Data from the real life, and data from the simulation model.  To prove the validity of the model.  More specifically, we want to decide whether these two outcomes are significantly different in terms of the specified mean performance measure.

11. Statistical Comparison of Outputs 11  Such a comparison can be made by performing the following hypothesis test: where μ 1 and μ 2 denote the mean of the random outcome from data source 1 and 2.  Note: t-test is a two-sided test; both positive and negative deviations from the hypothesized value (0 in our case) matter.

12. Statistical Comparison of Outputs 12  In order to make such a comparison, one can utilize the confidence intervals.  Let X and Y represent the random outcome obtained from source 1 and source 2, we define the following random variable:.  We construct a confidence interval for d:  If 0 lies inside this interval, then we cannot claim that the mean differences are significantly different from 0.

13. Example 13  Suppose we have gathered the following (average total time) statistics from 10 replications of Model 1 and Model 2.  Test whether these two models are significantly different from each other with respect to the average total time performance measure.  Carry out the hypothesis test  Construct the C.I. and observe that the same conclusion is obtained due to the duality of confidence intervals and hypotheses tests.  Check ttest.xlsx for calculations & result. #Model 1Model 2 1 14.144313.3307 2 11.506810.2171 3 10.611710.6563 4 14.263610.2678 5 23.077210.3082 6 28.271312.1404 7 11.180411.3499 8 15.05679.0096 9 11.87659.0139 10 16.008910.1748

14. Output Analyzer 14  Main uses:  Comparing means / variances (hypothesis tests)  Computing confidence intervals  Plotting histograms, charts, moving averages  Consider Call Center Model.  Compute confidence intervals for  Average number in Process Product Type 1’s Queue  Average time in system  Let the level of significance be 0.05.

15. Call Center Example 15

16. Call Center Example 16  Suppose that we are allowed to hire 7 new personnels.  Scenario 1: Hire 3 Sales person, and 4 Tech All  Scenario 2: Hire 1 Sales person, and 6 Tech All  Since “0” is an element of the C.I, there is no statistical evidence that one scenario is better than the other.