1. Introduction to Simulation Andy Wang CIS 5930-03 Computer Systems Performance Analysis

2. 2 Simulations Useful when the system is not available Good for exploring a large parameter space However, simulations often fail Need both statistical and programming skills Can take a long time

3. Common Mistakes Inappropriate level of detail –More details  more development time  more bugs  more time to run –More details require more knowledge of parameters, which may not be available E.g., requested disk sector –Better to start with a less detailed model Refine as needed 3 Common Mistakes

4. Improper language –Simulation languages Less time for development and statistical analysis –General-purpose languages More portable Potentially more efficient 4

5. Common Mistakes Invalid models –Need to be confirmed by analytical models, measurements, or intuition Improperly handled initial conditions –Should discard initial conditions –Not representative of the system behavior Too short simulations –Heavily dependent on initial conditions 5

6. Common Mistakes Poor random number generators –Safer to use well-known ones –Even well-known ones have problems Improper selection of seeds –Need to maintain independence among random number streams –Bad idea to initialize all streams with the same seed (e.g., zeros) 6

7. Other Causes of Simulation Analysis Failure Inadequate time estimate –Underestimate the time and effort –Simulation generally takes the longest time compared to modeling and measurement Due to debugging and verification No achievable goal –Needs to be quantifiable 7

8. Other Causes of Simulation Analysis Failure Incomplete mix of essential skills –Project leadership –Modeling and statistics –Programming –Knowledge of modeled system Inadequate level of user participation –Need periodic meetings with end users 8

9. Other Causes of Simulation Analysis Failure Obsolete or nonexistent documentation Inability to manage the development of a large, complex computer program –Needs to keep track of objectives, requirements, data structures, and program estimates Mysterious results –May need more detailed models 9

10. Terminology State variables: the variables whose values define the state of the system –E.g., length of a job queue for a CPU scheduler Event: a change in the system state 10 Terminology

11. Static and Dynamic Models Static model: time is not a variable –E.g., E = mc 2 Dynamic model: system state changes with time –CPU scheduling 11

12. Continuous and Discrete-time Model Continuous-time model System state is defined at all times Discrete-time model System state is defined only at instants in time 12 Time Tuesdays and Thursdays Time spent executing a job Number of students attending this class

13. Continuous and Discrete- state Model Continuous-state model Use continuous state variables Discrete-state model Use discrete state variables 13 Possible to have all four combinations of continuous/discrete time/state models Time spent executing a job Number of jobs

14. Deterministic and Probabilistic Model Deterministic model Output of a model can be predicted with certainty Probabilistic model Gives a different result for the same input parameters 14 input output

15. Linear and Nonlinear Models Linear model Output parameters are linearly correlated with input parameters Nonlinear model Otherwise 15

16. Stable and Unstable Models Stable model Settles down to a steady state Unstable model Otherwise 16

17. Open and Closed Models Open model Input is external to the model and is independent of the model Close model No external input 17

18. Computer System Models Generally –Continuous time –Discrete state –Probabilistic –Dynamic –Nonlinear 18

19. Selecting a Language for Simulation Simulation language General-purpose language Extension of general-purpose language Simulation package 19 Selecting a Language for Simulation

20. Simulation Languages Have built-in facilities –Time advancing –Event scheduling –Entity manipulation –Random-variate generation –Statistical data collection –Report generation Examples: SIMULA, Maisie, ParSEC 20

21. General-purpose Languages C++, Java No need to learn a new language Simulation languages may not be available More portable Can be optimized 21

22. Extensions of General- Purpose Languages Provide routines commonly required in simulation Examples: CSim, NS-2 (OTcl + C++) 22

23. Simulation Packages Provide a library of data structures, routines, algorithms Significant time savings –Can be done in one day However, not flexible for unforeseen scenarios 23

24. Types of Simulations Emulation –Hybrid simulation Monte Carlo simulation Trace-driven simulation Discrete-event simulation 24 Types of Simulations

25. Emulation and Hybrid Simulation Emulation –A simulation using hardware/firmware Hybrid simulation –A simulation that combines simulation and hardware –E.g., a 5-disk RAID One simulated disk Four real disks 25

26. Monte Carlo Simulation A type of static simulation Models probabilistic phenomenon Can be used to evaluate nonprobabilistic expressions –E.g., use the average of estimates to evaluate difficult integrals 26

27. Trace-Driven Simulation Trace: a time-ordered record of events on a real system –Needs to be as independent of the underlying system as possible Storage-level trace may be specific to the cache replacement mechanisms above, the working set, the memory size, etc. 27

28. Advantages of Trace- Driven Simulation Credibility Easy validation –Just compare measured vs. simulated numbers Accurate workload –Preserves the correlation and interferences effects 28

29. Advantages of Trace- Driven Simulation Less randomness –Deterministic input –Less variance –Fewer number of runs to get good confidence Fairer comparison (deterministic input) –For different alternatives Similarity to the actual implementation 29

30. Disadvantages of Trace- Driven Simulation Complexity –More detailed simulation to take realistic trace inputs Representativeness –Trace from one system may not be representative of the workload on another system –Can become obsolete quickly 30

31. Disadvantages of Trace- Driven Simulation Finiteness –A trace of a few minutes may not capture enough activity Single point of validation –Algorithms optimized for one trace may not work for other traces Trade-off –Difficult to change workload characteristics 31

32. Discrete-Event Simulation Uses discrete-state model –May use continuous or discrete time values 32

33. Common Components Event scheduler –E.g., schedule event X at time T Simulation clock A time-advancing mechanism –Unit time: Increments time by small increments –Event-driven: Increments time automatically to the time of the next earliest event 33

34. Common Components System state variables Event routines (handlers) Input routines –E.g., number of repetitions Report generator Initialization routines –Beginning of a simulation, iteration, repetition 34

35. Common Components Trace routines (for debugging) –Should have an on/off feature –Snapshot/continue from a snapshot Dynamic management Main program 35

36. Event-Set Algorithms How to track events –Ordered linked list (< 20 events) –Indexed linked list (20 – 120 events) Calendar queue –Tree structure (> 120 events) E.g., heap 36

37. 37 White Slide