Aleksandar Kuzmanovic and Edward W. Knightly Rice Networks Group Measuring Service in Multi-Class Networks

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Background l QoS services –SLA guaranteed rate  Ex. Class X serviced at minimum rate R –Relative performance  Ex. Class X has strict priority over class Y –Statistical service  Ex. P(class X pkt. Delay>100ms)<.001 l QoS mechanisms –Priority queues  Rate-based, delay- based... –Policing  Rate limiting... –Over-engineering  Just add more bandwidth... Need:Tools for network clients to assess the networks QoS capabilities

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Inverse QoS Problem l Is a class rate limited? l What is the inter-class relationship? –Fair/weighted fair/strict priority l Is resource borrowing fully allowed or not? l Is the service’s upper bound identical to its lower bound? l What are the service’s parameters?

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Applications - Network Example Providers reluctant to divulge precise QoS policy (if any...) l SLA validation for VPNs –Is the SLA fulfilled? l Capacity planning –What is the relationship among classes? l Edge-based admission control [CK00] and implementation [SSYK01]

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Performance Monitoring and Resource Management l Single WEB server –CPU resource sharing –Listen queue differentiation –Admission control l Distributed WEB server –Load balancing l Internet Data Center –Machine migration Goal:Estimate a class’ net “guaranteed rate”

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 “Off-Line” Solution is Simple l Consider a router with unknown QoS mechanisms

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 “On-Line” Case: Operational Network l Undesirable to disrupt on-going services –High rate probes to detect inter-class relationships would degrade performance l Impossible to force other classes to be idle –… to detect policers

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 System Model and Problem Formulation l Two stage server –Non-work conserving elements –Multi-class scheduler l Observations –Arrival and departure times –Class ID –Packet size

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Determine... l Infer the service discipline –Most likely hypothesis among WFQ, EDF and SP l Detect the existence of non-work conserving elements –Rate limiters (ex. leaky bucket policers) l Estimate the system parameters –WFQ guaranteed rates, EDF deadlines, rate limiter values

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Remaining Outline l Inter-class Resource Sharing Theory l Empirical Arrival and Service Models l MLE of Parameters l EDF/WFQ/SP Hypothesis Testing l Simulation Results and Conclusions

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Theoretical Tool: Statistical Service Envelopes [QK99] l General statistical char. for a (virtual) minimally backlogged flow l Flows receive additional service beyond min rate –Function of other flow demand –Function of scheduler l General characterization of inter-class resource sharing l Framework for admission control for EDF/WFQ/SP

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 l Inter-class theory l Key technique: –Passively monitor arrivals and services at edges –Devise hypothesis tests to jointly:  Detect most likely hypothesis  Estimate unknown parameters Strategy

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Empirical Arrival Model l Envelopes characterize arrivals as a function of interval length –Statistical traffic envelope [QK99] l Empirical envelope - measure first two moments of arrivals over multiple time scales time t + It E *( I ) = 3 Goal: assuming Gaussian distribution for B

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Empirical Service Model l A real-world paradigm for statistical service envelope l Observe: Service can be measured only when packets are backlogged

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Empirical Service Distributions l For each class and time scale –Expected service distributions –Service measures (data) l Empirical service distributions WFQ (400 ms) SP (400 ms)

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Parameter Estimation and Scheduler Inference l GLRT for each time scale l Under MLE parameters for each scheduler l Choose most likely scheduler l Apply majority rule over all time scales

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 EDF/WFQ Testing l Correctness ratio True WFQ  94% True EDF  100% Importance of time scales l Short time scales –Fluid vs. packet model l Long time scales –Ratio of delay shift and time scale decreases as time scale increases (d1=25ms)

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Measurable Regions l What if there is no traffic in particular class? l What traffic load “allows” inferences? l Region where we are able to estimate true value within 5% l Typical utilization should be > 62% for 1.5 Mbps link l Otherwise, active probing required

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Conclusions l Framework for clients of multi-class services to assess a system’s core QoS mechanisms –Scheduler type –Estimate parameters (both w-c and n-w-c) l General multiple time-scale traffic and service model to characterize a broad set of behaviors within a unified framework

Aleksandar Kuzmanovic and Edward W. Knightly Rice Networks Group Measuring Service in Multi-Class Networks

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Ongoing Work l Unknown cross-traffic –Cannot monitor all systems inputs/outputs –Treat cross-traffic statistics as another unknown l Web servers –Evaluation of the framework in a single web server through trace driven simulations –Capacity is statistically characterized

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 WFQ Parameter Estimation l Class 1: flows l Class 2: flows l Large windows improve confidence level –T=2sec: 95% in 11% of true value –T=10sec: 95% in 1.4% of true value  Flow level dynamics & non- stationarities must be considered

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Rate Limited Class State Detection l Can include parameter r in service envelope equations for each class Importance of time scales l Example –Class based fair queuing –C=1.5Mbps, r=1Mbps l Probability decreases with time scale  higher errors when measuring multi-level leaky-buckets

Kuzmanovic & Knightly | Rice Networks Group | INFOCOM 2001 Generalized Likelihood Ratio Test l Detection with unknowns l Note: we do not find a single value of that maximizes likelihood ratio l Under mild conditions (as ), GLRT is Uniformly Most Powerful (maximizes the probability of detection)