Methods of Stochastic Performance Analysis Perry Lea Hewlett Packard - Core Technology Laboratory

Why Am I Talking About This Currently I’m an ES working on future architectures and use these techniques with frequency to aid my design decisions Used these techniques to discover good designs Used these techniques to stop bad designs

Who Uses This Stuff System Architects / Engineers / Scientists Transportation systems Industrial and production analysis Behavorialists Communication systems and of course the general engineering community.

Typical Problems Addressed How fast will it print or copy? How do we distribute various tasks to various processors? What is the bus contention? Will we meet the real-time requirements?

What Is Performance Analysis (Jain Book Definition) Specifying performance requirements Comparing 2 or more systems Determining optimal/achievable parameters Finding bottlenecks Capacity planning Workload planning This talk is about: Stochastic Analysis Operational Analysis Discrete Event Simulation

The Stochastic Process Anything that is observed as a random process in time is a stochastic process. A stochastic process can be discrete (good for CS folks) or continuous (good for signals folks) If a future state of your system is not dependent on a previous state, the process can be called a Markov process. This is good because Markov processes are easy to model. Memoryless property If the arrival times of “data” to the different states are independent of each other – AND – the arrival times have the same distribution (hopefully exponentially distributed), then you can describe the process as a Poisson process. IID: Independent and identically distributed

Queue Rules  the interarrival rate, time between 2 successive arrivals (Random Variable)  the arrival rate s the service time  the mean service rate per server, or 1/E[s] n the number of jobs in the system r the response time (includes time waiting in queue and actual cpu time) w the waiting time

Some Math l = arrival rate (going in) m = service rate (going out) r = l / m is the traffic intensity or density The probability that there are n tasks waiting in the queue is:

Birth Death Process The Birth – Death Process Each state represents the queue depth 1 2 j-1 j j+1

Queues Kendall Notation A/B/C/D/E Types of processes A = arrival process B = service process C = number of servers D = maximum number of jobs for which the queue has room. E = maximum number of jobs being simulated Types of processes M = (Memoryless) Poisson arrival process or exponentially distributed service time D = deterministic arrival process (each job has the same service time) Aha! Here’s the deterministic part of all this. G = general distribution arrival process E = Erlang distribution Hyperexponential

Typical Queues M/M/1 M/G/1 Others Jobs arrive according to a Poisson process Service time of each job is exponentially distributed. There is one server There is infinite waiting room and no limit to the number of jobs M/G/1 Service time is generally distributed (no restriction of the form the distribution may take) Others M/D/1 (D denotes each jobs take the same amount of time to be service) M/G/ (Infinite amount of servers available) G/M/1 G/G/m

M/M/1 Model Analysis Traffic Intensity (Utilization): r = l/m Condition of Stability: r < 1 Probability queue empty: p0 = 1 – r Probability queue = n: pn = (1-r)rn * Mean queue depth: E[n]=r/(1 - r) Mean waiting time: E[w]=r ((1/m) / (1-r)) Mean response time (E[n] = /E[r]) * E[r] = (1/m) / (1-r) Other models have similar equations (See Jain Book)

M/M/1 Example A printer processes 125 packets per second and takes 2 ms to process them. When will the printer overflow with 13 buffers? How many buffers are needed to keep packet loss below one in a million? Arrival rate l = 125 pps Service rate m = 1/.002 = 500 pps Utilization r = 0.25 or 25% Probability of n packets in gateway = (1-r)rn = 0.75(0.25)n Mean number of packets in gateway = r/1-r = 0.33 Probability of buffer overflow -> P(more than 13 packets) 13 = 0.2513 = 15 packets in 1 billion To limit the probability loss to 10-6 n <= 10-6 n > log(10-6) / log(0.25) = 9.96 buffers

Operational Laws (Denning and Buzen) (Simple formulas for finite observable time) If you observe a system for a finite amount of time, you can skip the distributions of service and interarrival times. You can use a set of simple (directly measurable) equations to predict performance Arrival Rate : # arrivals / time : Ai/T Throughput : # completions / time : Ci/T Busy Time : Bi Mean Service Time : total time served / # served : Bi / Ci Utilization Law : U = Bi/T (T is the observed time) Forced Flow Law : system throughput : X = C0/T Little’s Law : mean # in device = arrival rate * mean time Qi=i*Ri Bottleneck Identification : The device with the largest Di

Queuing Networks Open Network Closed Network Jackson product-form solution Closed Network Buzen convolution algorithms

Large Queuing Networks Large systems can be modeled as a collection of queues. Bounding algorithms Find upper and lower bounds only. Mean value analysis (MVA) Iterative method to find response times for each device in the system. This will sum to the system response time and overall queue lengths. Convolution algorithms Array convolutions Can answer more detailed questions than MVA, i.e. are 2 IO devices busy at the same time? For large networks, we use software simulators

Discrete Event Simulations Why use a simulator? Hard to build analytic models for: Virtual memory Resource sharing (CPU and IO) Mutual exclusion to resources Bulk packet arrivals (self similarity) Non-exponential service times Can run many variants and perturbations of a system Can build stochastic and deterministic models and intermix them. To automate the mathematical process Allows you to study the problem from many angles with ease compared to analytical models

Modeling Tips Understand the architecture in detail first Decide on how detailed your model should be The more detailed the more time, and possibly the more bugs you will introduce you don’t want to build the final product. Do you need to model the whole system, or just a part? Get the abstraction defined before you start!! Chose the right tool and language Verify and test the models State what the goal of the model is, know when to stop Don’t let mysterious results pass – understand them. Review the model design – like any other piece of code.

Good Simulation Tools http://www.idsia.ch/~andrea/simtools.html C++SIM Good site with links to many modeling tools C++SIM C Libraries Do it yourself standalone, no GUI, good for academics Opnet Modeler MIL3, mainly for network and data flow CSIM Lockheed Martin – good for computer and electronic systems Discrete and continuous models NT and Unix flavors SES Workbench A generic modeling tool GUI, object orientated, NT and Unix

Recommended Reading “The Art of Computer Systems Performance Analysis”, Raj Jain, Wiley & Sons 1991 “Performance Modeling for Computer Architects”, C. Krishna, IEEE Computer Society Press 1996 “Telecommunication Networks”, Mischa Shwartz, Addison Wesley 1987