1. Lecture 7 Model Development and Model Verification

2. 2 Model Building  One of the most important and difficult tasks facing a model developer is the verification and validation of the simulation model.  To reduce the degree of skeptic about model’s validity  To increase the model’s credibility  Verification is concerned with building the model right. It is utilized in the comparison of the conceptual model to computer representation that implements that conception.  Is the model implemented correctly in the computer?  Are the input parameters and logical structure of the model correctly represented?  One of the most important and difficult tasks facing a model developer is the verification and validation of the simulation model.  To reduce the degree of skeptic about model’s validity  To increase the model’s credibility  Verification is concerned with building the model right. It is utilized in the comparison of the conceptual model to computer representation that implements that conception.  Is the model implemented correctly in the computer?  Are the input parameters and logical structure of the model correctly represented?

3. 3 Model Building (cont’)  Validation is concerned with building the right model. It is utilized to determine that a model is an accurate representation of the real system.  Validation is usually achieved through the calibration of the model, an iterative process of comparing the model to actual system behavior and using the discrepancies between the two, and the insights gained, to improve the model.  This process is repeated until model accuracy is judged to be acceptable.  Validation is concerned with building the right model. It is utilized to determine that a model is an accurate representation of the real system.  Validation is usually achieved through the calibration of the model, an iterative process of comparing the model to actual system behavior and using the discrepancies between the two, and the insights gained, to improve the model.  This process is repeated until model accuracy is judged to be acceptable.

4. 4 Model Building (cont’)  The first step in model building consists of observing the real system and the interactions among its various components and collecting data on its behavior.  Ask person who are familiar with the system.  New questions may arise.  Model developers will return to this step of learning true system structure and behavior.  The first step in model building consists of observing the real system and the interactions among its various components and collecting data on its behavior.  Ask person who are familiar with the system.  New questions may arise.  Model developers will return to this step of learning true system structure and behavior.

5. 5 Model Building (cont’)  The second step in model building is the construction of a conceptual model.  A collection of assumptions on the components and the structure of the system, plus hypotheses on the values of model input parameters.  The third step is the translation of the operational model into a computer-recognizable form – the computerized model.  The second step in model building is the construction of a conceptual model.  A collection of assumptions on the components and the structure of the system, plus hypotheses on the values of model input parameters.  The third step is the translation of the operational model into a computer-recognizable form – the computerized model.

6. 6 Model-building Process Real system Conceptual model 1.Assumptions on system components 2.Structural assumptions, which define the interactions between system components 3.Input parameters and data assumptions Conceptual model 1.Assumptions on system components 2.Structural assumptions, which define the interactions between system components 3.Input parameters and data assumptions Operational model (Computerized representation) Operational model (Computerized representation) Conceptual validation Model verification Calibration And validation

7. 7Verification  The purpose of model verification is to assure that the conceptual model is reflected accurately in the computerized representation.  The conceptual model quite often involves some degree of abstraction about system operations, or some amount of simplification of actual operations.  Verification asks the question:  Is the conceptual model (assumptions on system components and system structure, parameter values, abstractions and simplifications) accurately represented by the operational model?  The purpose of model verification is to assure that the conceptual model is reflected accurately in the computerized representation.  The conceptual model quite often involves some degree of abstraction about system operations, or some amount of simplification of actual operations.  Verification asks the question:  Is the conceptual model (assumptions on system components and system structure, parameter values, abstractions and simplifications) accurately represented by the operational model?

8. 8 Three Classes of Technique  Common-sense techniques  Thorough documentation  Traces  Common-sense techniques  Thorough documentation  Traces

9. 9 Common-sense Techniques  Checked by someone other than its developer.  Make a flow diagram and follow each event type.  Examine the output for reasonableness under a variety of settings of the input parameters.  Print the input parameters at the end of simulation to ensure that these parameters values have not been changed inadvertently.  Make the model as self-documenting as possible.  Checked by someone other than its developer.  Make a flow diagram and follow each event type.  Examine the output for reasonableness under a variety of settings of the input parameters.  Print the input parameters at the end of simulation to ensure that these parameters values have not been changed inadvertently.  Make the model as self-documenting as possible.

10. 10 Common-sense Techniques (cont’)  If the operational model is animated, verify that what is seen in the animation imitates the actual system.  Use the debugger provided by the simulation software.  Use a variety of graphics to represent different model states.  For example, (reasonableness)  Current contents and total count  Current content refers to the number of items in each component of the system at a given time.  Total count refers to the total number of items that have entered each component of the system by a give time.  If the operational model is animated, verify that what is seen in the animation imitates the actual system.  Use the debugger provided by the simulation software.  Use a variety of graphics to represent different model states.  For example, (reasonableness)  Current contents and total count  Current content refers to the number of items in each component of the system at a given time.  Total count refers to the total number of items that have entered each component of the system by a give time.

11. 11 Oft-neglected Documentation Technique  Documentation is also important as a means of clarifying the logic of a model and verifying its completeness.  If a model builder writes brief comments in the computerized model, plus definitions of all variables and parameters, and descriptions of each major section of the computerized model, it becomes much simpler for someone else, or the model builder at a later date, to verify the model logic.  Documentation is also important as a means of clarifying the logic of a model and verifying its completeness.  If a model builder writes brief comments in the computerized model, plus definitions of all variables and parameters, and descriptions of each major section of the computerized model, it becomes much simpler for someone else, or the model builder at a later date, to verify the model logic.

12. 12 Trace Technique  A more sophisticated technique.  A trace is a detailed computer printout which gives the value of every variable in a computer program, every time that one of these variables changes in value.  The purpose of the trace is to verify the correctness of the computer program by making detailed paper- and-paper calculations.  Some software allows a selective trace.  Whenever the queue before a certain resource reaches five or more, turn on the trace.  A more sophisticated technique.  A trace is a detailed computer printout which gives the value of every variable in a computer program, every time that one of these variables changes in value.  The purpose of the trace is to verify the correctness of the computer program by making detailed paper- and-paper calculations.  Some software allows a selective trace.  Whenever the queue before a certain resource reaches five or more, turn on the trace.

13. 13Recommendations  It is recommended that the first two always be carried out.  Close examination of model output for reasonableness is especially valuable and informative.  A trace can also provide information if it is selective.  The generalized trace can be extremely time consuming.  It is recommended that the first two always be carried out.  Close examination of model output for reasonableness is especially valuable and informative.  A trace can also provide information if it is selective.  The generalized trace can be extremely time consuming.