Steady-State Statistical Analysis By Dr. Jason Merrick

Simulation with Arena - Steady-state Output Analysis C7/2 Warm Up and Run Length Most models start empty and idle –Empty: No entities present at time 0 –Idle: All resources idle at time 0 –In a terminating simulation this is OK if realistic –In a steady-state simulation, though, this can bias the output for a while after startup Bias can go either way Usually downward (results are biased low) in queueing-type models that eventually get congested Depending on model, parameters, and run length, the bias can be very severe

Simulation with Arena - Steady-state Output Analysis C7/3 Warm Up and Run Length (cont’d.) The period up to 1500 minutes is less congested Thus average output measures will be biased down How can we get rid of this bias?

Simulation with Arena - Steady-state Output Analysis C7/4 Intelligent Initial Conditions Collect data –Observe an actual state of the real system that has been running for a reasonable period of time –Use this state as the initial conditions –Not possible if system does not exist or you are modifying the system Use another model –Queuing models, inventory models etc. –Give steady-state results under more restrictive assumptions than simulation –Use these results as initial conditions

Simulation with Arena - Steady-state Output Analysis C7/5 Warm-up Define some time t W until which no statistics are collected –Suppose m W observations are collected up to time t W –Suppose m observations are collected after time t W The idea is that Y m w +1,…,Y m+m w are drawn from the “steady state” distribution, while Y m,…,Y m w are from a different warm-up distribution –So truncating the warm-up observations removes the bias

Simulation with Arena - Steady-state Output Analysis C7/6 Determining Warm-up Times Ensemble averages –The average across replications of the first, second, third, … observations –Each ensemble average is an iid sample from the distribution of that observation –Put t-distribution confidence interval around each average –See when the ensemble averages settle down Series Averages Ensemble Averages

Simulation with Arena - Steady-state Output Analysis C7/7 Determining Warm-up Times

Simulation with Arena - Steady-state Output Analysis C7/8 Determining Warm-up Times

Simulation with Arena - Steady-state Output Analysis C7/9 Truncated Replications If you can identify appropriate warm-up and run- length times, just make replications as for terminating simulations –Only difference: Specify Warm-Up Period in Simulate module

Simulation with Arena - Steady-state Output Analysis C7/10 Batching in a Single Run If model warms up very slowly, truncated replications can be costly –Have to “pay” warm-up on each replication –Throw away

Simulation with Arena - Steady-state Output Analysis C7/11 Batching in a Single Run Alternative: Just one R E A L L Y long run –Only have to “pay” warm-up once –Problem: Have only one “replication” and you need more than that to form a variance estimate (the basic quantity needed for statistical analysis) Big no-no: Use the individual points within the run as “data” for variance estimate Usually correlated (not indep.), variance estimate biased throw away sample

Simulation with Arena - Steady-state Output Analysis C7/12 Batching in a Single Run (cont’d.) Break each output record from the run into a few large batches Batch 1Batch 2Warm-up …...

Simulation with Arena - Steady-state Output Analysis C7/13 The batch means will not actually be independent –The idea is to reduce the correlation to a level that it will not introduce a significant bias in the estimate of the standard deviation –The individual observations in the series are correlated with the previous observations –The correlogram shows that the correlation reduces the higher the lag –So if the batch size is long enough then most of the observations making up batch 1 will be approximately independent of those making up batch 2 –Only the observations near the end of the batches will be correlated Batching in a Single Run (cont’d.) Batch 1Batch 2

Simulation with Arena - Steady-state Output Analysis C7/14 Batching in a Single Run (cont’d.) Rules of thumb –Schmeiser (1982) found that for a given run length, there was little benefit in more than 30 batches –However, less than 10 batches was too few –There may well be correlation between all lags, looking at the lag 1 correlation is usually enough to ascertain independence –Auto-correlation estimates are not very good for sample sizes like 30 –Use smaller batches (say c > 100) and if the independence test is passed then the bigger batches will be fine

Simulation with Arena - Steady-state Output Analysis C7/15 Examining # of Batches Consider the Simple Processing System –t = 1,000,000 minutes, increase number of batches CI too smallCI doesn’t change much

Simulation with Arena - Steady-state Output Analysis C7/16 Examining Run Length Consider the Simple Processing System –n = 25 replications, increase the run length

Simulation with Arena - Steady-state Output Analysis C7/17 Consider the Simple Processing System –t = 15 minutes, increase the number of replications Examining # of Replications