1. © 2003, Carla Ellis Model Vague idea “groping around” experiences Hypothesis Initial observations Experiment Data, analysis, interpretation Results & final Presentation Experimental Lifecycle

2. Models & Methods prototype real sys exec- driven sim trace- driven sim stochastic sim Live workload Benchmark applications Micro- benchmark programs Synthetic benchmark programs Traces Distributions & other statistics monitor analysis generator Synthetic traces “Real” workloads Made-up © 2003, Carla Ellis Data sets monitor

3. closed source implementation monitor Measurement API Instrumented prototype implementation metrics closed source implementation monitor Client driver monitor Live Deployment or Controlled Testbed? Measured Implementations Indirect measurements What info can we get out? What config options do we have?

4. closed source implementation monitor Measurement API Instrumented prototype implementation metrics closed source implementation monitor Client driver monitor Measured Implementations Indirect measurements PlanetLab ECOSystem

5. Execution Driven Simulation / Emulation exec- driven sim Live workload Benchmark applications Micro- benchmark programs Synthetic benchmark programs workloads © 2003, Carla Ellis Data sets System parameters Level of detail still variable – both cycle-accurate timing & functional simulators fit in here.

6. Execution Driven Simulation / Emulation exec- driven sim Live workload Benchmark applications Micro- benchmark programs Synthetic benchmark programs workloads © 2003, Carla Ellis Data sets System parameters SimpleScalar TFSim Simics What can we specify? What level of detail?

7. Trace Driven Simulation exec- driven sim trace- driven sim Traces Distributions & other statistics analysis generator Synthetic traces © 2003, Carla Ellis System parameters What can we specify? What dynamic behavior is modeled?

8. Trace Driven Simulation exec- driven sim trace- driven sim Traces Distributions & other statistics analysis generator Synthetic traces © 2003, Carla Ellis System parameters Ns-2

9. Stochastic Simulations stochastic sim Traces Distributions & other statistics analysis Made-up © 2003, Carla Ellis System parameters What can we specify? What dynamic behavior is modeled?

10. Stochastic Simulations stochastic sim Traces Distributions & other statistics analysis Made-up © 2003, Carla Ellis System parameters Examples?