MS 305 Recitation 11 Output Analysis I 16.05.2013.

Slides:



Advertisements
Similar presentations
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 9 Inferences Based on Two Samples.
Advertisements

Statistics Review – Part II Topics: – Hypothesis Testing – Paired Tests – Tests of variability 1.
CHAPTER 21 Inferential Statistical Analysis. Understanding probability The idea of probability is central to inferential statistics. It means the chance.
The Independent- Samples t Test Chapter 11. Independent Samples t-Test >Used to compare two means in a between-groups design (i.e., each participant is.
Chapter 11- Confidence Intervals for Univariate Data Math 22 Introductory Statistics.
© 2010 Pearson Prentice Hall. All rights reserved Least Squares Regression Models.
Two Sample Hypothesis Testing for Proportions
10-1 Introduction 10-2 Inference for a Difference in Means of Two Normal Distributions, Variances Known Figure 10-1 Two independent populations.
Stat 301 – Day 28 Review. Last Time - Handout (a) Make sure you discuss shape, center, and spread, and cite graphical and numerical evidence, in context.
Sample size computations Petter Mostad
Topic 2: Statistical Concepts and Market Returns
Evaluating Hypotheses
1 Validation and Verification of Simulation Models.
Lecture 10 Comparison and Evaluation of Alternative System Designs.
Experimental Evaluation
IENG 486 Statistical Quality & Process Control
Inferences About Process Quality
Lehrstuhl für Informatik 2 Gabriella Kókai: Maschine Learning 1 Evaluating Hypotheses.
5-3 Inference on the Means of Two Populations, Variances Unknown
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 7 Statistical Intervals Based on a Single Sample.
Inferential Statistics
Review of normal distribution. Exercise Solution.
1 Terminating Statistical Analysis By Dr. Jason Merrick.
Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 12 Analyzing the Association Between Quantitative Variables: Regression Analysis Section.
Confidence Intervals and Hypothesis Testing - II
Fundamentals of Hypothesis Testing: One-Sample Tests
Comparing Systems Using Sample Data Andy Wang CIS Computer Systems Performance Analysis.
Copyright © 2013, 2010 and 2007 Pearson Education, Inc. Chapter Inference on the Least-Squares Regression Model and Multiple Regression 14.
Confidence Intervals and Two Proportions Presentation 9.4.
The paired sample experiment The paired t test. Frequently one is interested in comparing the effects of two treatments (drugs, etc…) on a response variable.
Inferences for Regression
Verification & Validation
Inference for a Single Population Proportion (p).
Lecture 14 Sections 7.1 – 7.2 Objectives:
Topics: Statistics & Experimental Design The Human Visual System Color Science Light Sources: Radiometry/Photometry Geometric Optics Tone-transfer Function.
Chapter 9 Hypothesis Testing and Estimation for Two Population Parameters.
1 1 Chapter 2: Comparing Means 2.1 One-Sample t -Test 2.2 Paired t -Test 2.3 Two-Sample t -Test.
Copyright 2011 by W. H. Freeman and Company. All rights reserved.1 Introductory Statistics: A Problem-Solving Approach by Stephen Kokoska Chapter 8: Confidence.
Maximum Likelihood Estimator of Proportion Let {s 1,s 2,…,s n } be a set of independent outcomes from a Bernoulli experiment with unknown probability.
Copyright © 2005 Brooks/Cole, a division of Thomson Learning, Inc Chapter 12 Inference About A Population.
Copyright © 2013, 2010 and 2007 Pearson Education, Inc. Section Inference about Two Means: Independent Samples 11.3.
1 Chapter 10: Introduction to Inference. 2 Inference Inference is the statistical process by which we use information collected from a sample to infer.
Chapter 7 Sampling and Sampling Distributions ©. Simple Random Sample simple random sample Suppose that we want to select a sample of n objects from a.
1 Terminating Statistical Analysis By Dr. Jason Merrick.
Tests of Hypotheses Involving Two Populations Tests for the Differences of Means Comparison of two means: and The method of comparison depends on.
Lecture 9 Chap 9-1 Chapter 2b Fundamentals of Hypothesis Testing: One-Sample Tests.
Chapter 9: Inferences Based on Two Samples: Confidence Intervals and Tests of Hypotheses Statistics.
Section A Confidence Interval for the Difference of Two Proportions Objectives: 1.To find the mean and standard error of the sampling distribution.
1 OUTPUT ANALYSIS FOR SIMULATIONS. 2 Introduction Analysis of One System Terminating vs. Steady-State Simulations Analysis of Terminating Simulations.
Machine Learning Chapter 5. Evaluating Hypotheses
Inen 460 Lecture 2. Estimation (ch. 6,7) and Hypothesis Testing (ch.8) Two Important Aspects of Statistical Inference Point Estimation – Estimate an unknown.
Various Topics of Interest to the Inquiring Orthopedist Richard Gerkin, MD, MS BGSMC GME Research.
Comparing Systems Using Sample Data Andy Wang CIS Computer Systems Performance Analysis.
Central Limit Theorem Let X 1, X 2, …, X n be n independent, identically distributed random variables with mean  and standard deviation . For large n:
Hypothesis Tests. An Hypothesis is a guess about a situation that can be tested, and the test outcome can be either true or false. –The Null Hypothesis.
Fundamentals of Data Analysis Lecture 4 Testing of statistical hypotheses pt.1.
6-1 Copyright © 2014, 2011, and 2008 Pearson Education, Inc.
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 7 Inferences Concerning Means.
SECTION 1 TEST OF A SINGLE PROPORTION
4-1 Statistical Inference Statistical inference is to make decisions or draw conclusions about a population using the information contained in a sample.
Class Six Turn In: Chapter 15: 30, 32, 38, 44, 48, 50 Chapter 17: 28, 38, 44 For Class Seven: Chapter 18: 32, 34, 36 Chapter 19: 26, 34, 44 Quiz 3 Read.
Copyright © Cengage Learning. All rights reserved. 8 Tests of Hypotheses Based on a Single Sample.
Inference for a Single Population Proportion (p)
3. The X and Y samples are independent of one another.
Chapter 9: Inferences Involving One Population
Hypothesis Testing and Confidence Intervals (Part 1): Using the Standard Normal Lecture 8 Justin Kern October 10 and 12, 2017.
CONCEPTS OF ESTIMATION
Chapter Nine Part 1 (Sections 9.1 & 9.2) Hypothesis Testing
CHAPTER 12 More About Regression
Lecture 7 Sampling and Sampling Distributions
Presentation transcript:

MS 305 Recitation 11 Output Analysis I

Half Width 2  We enforce an upper bound on error h, satisfying the following inequality:  When the above probability is equal to (1 - α), h is called as half-width.  In our analyses, we make use of the fact that the statistic below follows a t-distribution:  The half width can be computed by plugging T into the equation below.

Half Width 3  Then, the formula for half-width is given by:  Keep in mind that the t-distribution assumption is only valid when the outputs are independent, identically distributed, and follows a normal distribution.  i.i.d. assumption is justified by ARENA since we are choosing initialize statistics and initialize system.  Normality assumption is justified using the Central Limit Theorem. Half-width =

Number of Replications 4

5  Once the approximate number of replications is obtained, you should re-run the model in order to check whether the desired half-width is obtained.  Example:  Suppose that the initial number of replication is 10 and it leads to an initial s² value,  Using this we compute n,  We re-run the model,  If the calculated half-width is less than or equal to h (the specified value), we are done,  Otherwise we take the recent s² as our initial guess and reuse the approximation to recalculate the new value of n…

Number of Replications 6  A reasonable strategy to specify an upper bound on half width (h) would be to use a proportion of the sample mean.  i.e. where γ denotes the percentage of deviation from sample mean  When the formula of half-width is given, you should know how to approximate the number of replications.  In other words, the approximation formulas will not be given in the exam since they are intuitive and can be obtained by simply manipulating the half-width formula.

Exercise  In exercise 5.03, about how many replications would be required to bring the half-width of a 95% confidence interval for the expected average cycle time for both trimmers down to one minute?  Start with 10 replications.

Exercise  We run the simulation with 10 replications and obtain the half-width associated with the 95% confidence interval.  Important: α=0.05 is the default value used in the half-width computation of Arena.  The half-widths for the expected average cycle time for primary and secondary trimmer are 2.11, and 14.77, respectively.  To reduce the half-widths to 1, we work with worst (14.77) and compute the approximate number of required replications (using the second approximation for sample size) as (10)x(14.77/1)²=  Rounding up to the nearest integer gives 2182 replications.

Confidence Interval 9  Using the results on half width, we can construct a 100(1-α)% confidence interval on mean as below: where

Statistical Comparison of Outputs 10  In output analysis, we are often concerned with the comparison of outcomes obtained from different sources.  Examples for these sources could be:  Two simulation models that could represent the current system and a modified version of the same system,  Data from the real life, and data from the simulation model.  To prove the validity of the model.  More specifically, we want to decide whether these two outcomes are significantly different in terms of the specified mean performance measure.

Statistical Comparison of Outputs 11  Such a comparison can be made by performing the following hypothesis test: where μ 1 and μ 2 denote the mean of the random outcome from data source 1 and 2.  Note: t-test is a two-sided test; both positive and negative deviations from the hypothesized value (0 in our case) matter.

Statistical Comparison of Outputs 12  In order to make such a comparison, one can utilize the confidence intervals.  Let X and Y represent the random outcome obtained from source 1 and source 2, we define the following random variable:.  We construct a confidence interval for d:  If 0 lies inside this interval, then we cannot claim that the mean differences are significantly different from 0.

Example 13  Suppose we have gathered the following (average total time) statistics from 10 replications of Model 1 and Model 2.  Test whether these two models are significantly different from each other with respect to the average total time performance measure.  Carry out the hypothesis test  Construct the C.I. and observe that the same conclusion is obtained due to the duality of confidence intervals and hypotheses tests.  Check ttest.xlsx for calculations & result. #Model 1Model

Output Analyzer 14  Main uses:  Comparing means / variances (hypothesis tests)  Computing confidence intervals  Plotting histograms, charts, moving averages  Consider Call Center Model.  Compute confidence intervals for  Average number in Process Product Type 1’s Queue  Average time in system  Let the level of significance be 0.05.

Call Center Example 15

Call Center Example 16  Suppose that we are allowed to hire 7 new personnels.  Scenario 1: Hire 3 Sales person, and 4 Tech All  Scenario 2: Hire 1 Sales person, and 6 Tech All  Since “0” is an element of the C.I, there is no statistical evidence that one scenario is better than the other.