Realistic CPU Workloads Through Host Load Trace Playback Peter A. Dinda David R. O’Hallaron Carnegie Mellon University

2 Talk in a Nutshell Workloads to evaluate distributed systems Prediction-based systems Shared benchmarks Reconstruct time-varying CPU contention behavior from traces of Unix load average Real (non-parametric) Reproducible Comparable Artifacts ( Playload tool Collection of host load traces

3 Outline Evaluation of distributed systems Adaptive applications Prediction systems Host load traces Synthetic Vs trace-based workloads Host load trace playback Evaluation Conclusion

4 Evaluating Distributed Systems Workloads critical in evaluation Shared benchmarks desperately needed SPEC? Synthetic versus trace-based workloads Parametric versus non-parametric Ideally: Real workloads Reproducible Comparable Sharable

5 Evaluating Prediction Systems Prediction systems model workload RPS, NWS, adaptive applications, etc. Synthetic workloads assume a model Model may be wrong or incomplete Easy to use in simulation or in a testbed Trace-based workloads assume no model However, traces may not be representative Harder to use, especially in a testbed How do we use traces in a testbed? Host load traces

6 Host Load Traces Periodically sampled Unix load average Exponentially averaged run queue length Measure of contention for CPU Complex statistical properties [SciProg99] High variability, strong autocorrelation function, self- similar, epochal behavior, … Difficult to synthesize

7 Host Load and Running Time

8 Available Host Load Traces DEC Unix 5 second exponential average Full bandwidth captured (1 Hz sample rate) Long durations Available!

9 Host Load Measurement Sample1h { Ready Queue Sample2 unknown sample rate f=2 Hz estimated exponential average, tau=5 s f=1 Hz KernelUser Trace File

10 Host Load Trace Playback h -1 Load Generator Trace File Load Measure Sample1h { Sample2 error - applied load measured load target load

11 What are h and h -1 ? run queue length trace file time constant for recorded host applied load (recovered run queue length) h: h -1 : measured load time constant for playback host

12 Load Generator “1.5 load for 1 second” “1.0 load for 1 second”... “0.5 load for 1 second” Master... Worker Processes “0.0 load for 1 second”

13 Load Generator // Split w into n cycles while (!done) { if (uniformrand(1.0) < p) compute for w/n seconds; else sleep for w/n seconds; } “p load for w seconds” done=“w seconds have elapsed” Time-based playback done=“w*p CPU seconds have been used” Work-based playback (simplified)

14 Time-based Playback External continuous 1.0 load External load amplitude modulates applied load

15 Work-based Playback External load amplitude and frequency modulates applied load External continuous 1.0 load

16 Evaluation Traces described earlier Example uses one hour 1997 axp0 trace Characterization of signals and errors Summary stats Distributions Autocorrelation Multiple platforms Digital Unix Solaris Linux FreeBSD

17 Evaluation Summary Stats

18 Evaluation on Alpha/DUX Target MeasuredError ACF Error Histogram

19 Evaluation on Sparc/Solaris Target MeasuredError ACF Error Histogram

20 Evaluation on P2/FreeBSD Target MeasuredError ACF Error Histogram

21 Evaluation on P2/Linux Target MeasuredError ACF Error Histogram

22 Conclusion CPU Workloads from traces of Unix load average Reproduce contention behavior (ignoring priorities) Real, non-parametric workloads Reproducible Artifacts ( Playload tool Collection of host load traces Future Benchmarks Priorities, memory, disk, etc. In-kernel?

23 Feedback? Use error signal to better track the load trace h -1 Load Generator Load Measure error - h -1 + z Trace File x level

24 The Problem With Feedback Feedback would try to make SUM of applied load and external load in system track the load trace External Load h -1 Load Generator Load Measure error - h -1 + z Trace File x level Applied Load Effect of Combined Load

25 Making Feedback Work External Load h -1 Load Generator Load Measure error - h -1 + z Trace File x level Signal Separation Applied Load Effect of Combined Load Estimated Effect of Applied Load Estimated Effect of External Load Load Source Models

26 Why Host Load Traces for Evaluating Distributed Systems? Real Comparable and reproducible Analogous to a SPEC benchmark Usable in simulation and experimentation Non-parametric and non-synthetic Especially important for prediction systems