Copyright © 2005 Department of Computer Science CPSC 641 Winter 20111 Simulation Validation Plan: –Discuss verification and validation –Define concepts.

Slides:

Advertisements

Similar presentations

Chapter 5 Errors Bjarne Stroustrup

Advertisements

Scientific Method.

Copyright © 2005 Department of Computer Science CPSC 641 Winter Self-Similarity in WAN Traffic A subsequent paper established the presence of network.

COMPUTATIONS ON THE SPHEROID. Learning Objectives Calculate  and of a point given  and s from a point with known  and. Calculate  and s between points.

Scientific Method.

Scientific Method The common steps that biologists and other scientists use to gather information and answer questions.

Chapter 1 Introduction to Modeling DECISION MODELING WITH MICROSOFT EXCEL Copyright 2001 Prentice Hall.

Programming Types of Testing.

Chapter 1 Introduction to Modeling DECISION MODELING WITH MICROSOFT EXCEL Copyright 2001 Prentice Hall Publishers and Ardith E. Baker.

Copyright © 2005 Department of Computer Science CPSC 641 Winter PERFORMANCE EVALUATION Often in Computer Science you need to: – demonstrate that.

Introduction to Research Methodology

ITEC 451 Network Design and Analysis. 2 You will Learn: (1) Specifying performance requirements Evaluating design alternatives Comparing two or more systems.

Paper Title Your Name CMSC 838 Presentation. CMSC 838T – Presentation Motivation u Problem paper is trying to solve  Characteristics of problem  … u.

Copyright © 2005 Department of Computer Science CPSC 641 Winter Simulation Methodology Plan: –Introduce basics of simulation modeling –Define terminology.

1 Simulation Validation H Plan: –Discuss verification and validation –Define concepts and terminology –Example: Weasley Barber Shop simulator u Effect.

1 Validation and Verification of Simulation Models.

Evaluating System Performance in Gigabit Networks King Fahd University of Petroleum and Minerals (KFUPM) INFORMATION AND COMPUTER SCIENCE DEPARTMENT Dr.

Chapter 3 Hypothesis Testing. Curriculum Object Specified the problem based the form of hypothesis Student can arrange for hypothesis step Analyze a problem.

Chapter 14 Simulation. Monte Carlo Process Statistical Analysis of Simulation Results Verification of the Simulation Model Computer Simulation with Excel.

1 PERF EVAL (CONT’D) H There are many other “tools of the trade” used in performance evaluation H Only a few will be mentioned here: –queueing theory –verification.

ENGINEERING YOUR FUTURE

Chapter 13 Auditing Information Technology

Stevenson and Ozgur First Edition Introduction to Management Science with Spreadsheets McGraw-Hill/Irwin Copyright © 2007 by The McGraw-Hill Companies,

General Analysis Procedure and Calculator Policy Calculator Policy.

IE 594 : Research Methodology – Discrete Event Simulation David S. Kim Spring 2009.

Problem Solving Methods. Class Objectives Learn to apply the problem solving process Learn techniques for error-free problem solving.

Genetic Regulatory Network Inference Russell Schwartz Department of Biological Sciences Carnegie Mellon University.

 1  Outline  stages and topics in simulation  generation of random variates.

ANGLES IN A TRIANGLE. Triangles are the simplest polygons with three sides and three angles. The sum of the three angles of a triangle is equal to 180.

Introduction to Research

Writing the Term Paper - 1 DON’T TAKE NOTES! ALL THAT FOLLOWS IS IN THE COURSE MATERIALS! As you write the final term paper, it would be useful to keep.

ENM 503 Lesson 1 – Methods and Models The why’s, how’s, and what’s of mathematical modeling A model is a representation in mathematical terms of some real.

The Nature of Evidence Chapter 3 ©2010 Elsevier, Inc.

Key Stage 3 National Strategy Writing in mathematics © Crown Copyright 2003.

(How do scientists conduct scientific experimentation?)

Debugging Simulation Models CS 780 Spring 2007 Instructor: Peter Kemper Dept of Computer Science, College of William and Mary Prerequisites: A first course.

Loops Wrap Up 10/21/13. Topics *Sentinel Loops *Nested Loops *Random Numbers.

MODES-650 Advanced System Simulation Presented by Olgun Karademirci VERIFICATION AND VALIDATION OF SIMULATION MODELS.

Introduction to Earth Science Section 2 Section 2: Science as a Process Preview Key Ideas Behavior of Natural Systems Scientific Methods Scientific Measurements.

Building Simulation Model In this lecture, we are interested in whether a simulation model is accurate representation of the real system. We are interested.

CPSC 441 Tutorial TA: Fang Wang. Simulation Methodology Plan: Introduce basics of simulation modeling Define terminology and methods used Introduce simulation.

Research Design. Selecting the Appropriate Research Design A research design is basically a plan or strategy for conducting one’s research. It serves.

Scientific Method.

Model validity, testing and analysis. Conceptual and Philosophical Foundations Model Validity and Types of Models –Statistical Forecasting models (black.

Winter 2007SEG2101 Chapter 121 Chapter 12 Verification and Validation.

Validation & Verification Today will look at: The difference between accuracy and validity Explaining sources of errors and how they could be overcome.

ERROR IN MEASUREMENTS P M V Subbarao Professor Mechanical Engineering Department How Truthful are Our Observations?

 Simulation enables the study of complex system.  Simulation is a good approach when analytic study of a system is not possible or very complex.  Informational,

What Is Science?. 1. Science is limited to studying only the natural world. 2. The natural world are those phenomena that can be investigated, discovered,

Dividing Fractions Whole Numbers by Fractions Please start working on your next day.

General Analysis Procedure and Calculator Policy Calculator Policy.

5 September 2002AIAA STC Meeting, Santa Fe, NM1 Verification and Validation for Computational Solid Mechanics Presentation to AIAA Structures Technical.

Simulation. Types of simulation Discrete-event simulation – Used for modeling of a system as it evolves over time by a representation in which the state.

Building Valid, Credible & Appropriately Detailed Simulation Models

1 Software Testing. 2 What is Software Testing ? Testing is a verification and validation activity that is performed by executing program code.

Performance Eval Slides originally from Williamson at Calgary

Scientific Method This Power Point covers only the basics of the scientific method. As you advance further in school, more details will be given.

Simulation Department of Industrial Engineering Anadolu University

CPSC 531: System Modeling and Simulation

Scientific Method.

Simulation Carey Williamson Department of Computer Science

Statistical Methods Carey Williamson Department of Computer Science

Model Validation Carey Williamson Department of Computer Science

Misleading Bioinformatics Mistakes, Biases, Mis-Interpretations and how to avoid them Festival of Genomics 2017.

Tracing What, How, and Why Copyright © Curt Hill,

Carey Williamson Department of Computer Science University of Calgary

Carey Williamson Department of Computer Science University of Calgary

Mathematics Benchmark

Verification & Validation

Akram Bitar and Larry Manevitz Department of Computer Science

Presentation transcript:

Copyright © 2005 Department of Computer Science CPSC 641 Winter Simulation Validation Plan: –Discuss verification and validation –Define concepts and terminology –Example: Barber Shop simulator Effect of arrival rate Effect of service rate(s) Effect of waiting room size Effect of dispatch policy

Copyright © 2005 Department of Computer Science CPSC 641 Winter Verification and Validation Simulation is a widely used (and abused!) technique for systems performance eval It is important to establish trust and confidence in the results generated and reported via simulation Two (slightly different) aspects: –Verification: results are correct –Validation: results match reality

Copyright © 2005 Department of Computer Science CPSC 641 Winter Verification Verification is mostly about establishing correctness (i.e., veracity, truthfulness) Does it do what it is supposed to do? –Meets requirements specifications (if any) Does it produce correct results? –Check results on known cases (input/output) –Handles special “corner” cases –Results are mathematically plausible

Copyright © 2005 Department of Computer Science CPSC 641 Winter Validation Validation is more about the practical applicability of the simulator (usefulness) Does it give answers that make sense? Are the results intuitively plausible? Can it predict results for unknown cases? Can results be corroborated in some way? –Analytical model –Experimental measurement data Relationship to reality is key!

Copyright © 2005 Department of Computer Science CPSC 641 Winter Validation Example Suppose you have a simulator for the Barber Shop problem How to do “simulation validation”? –Manual tracing with printf statements –Try boundary cases (extreme and/or special) –Numerical comparisons to analytical model –Comparison to M/M/1/K analytical model –Comparison to M/M/2 analytical model

Copyright © 2005 Department of Computer Science CPSC 641 Winter Summary Simulation validation is an important step in establishing the correctness of results Rarely discussed in papers themselves Important “behind the scenes” effort Should be part of your science “checklist” If results don’t make sense, then find simplest possible example showing the problem, so you can debug and fix it!