1. 1AT&T Labs - Research SNMP Simple Network Measurements Please! Matthew Roughan (+many others)

2. 2AT&T Labs - Research Outline zPart I: SNMP traffic data ySimple Network Management Protocol yWhy? How? What? zPart II: Wavelets yWhat can you do? yWhy not? zPart III: Modeling yPutting time series and traffic modeling together xTraffic modeling deals with stationary processes (typically) xTime series gives us a way of getting a stationary process xBut the analysis requires an understanding of the traffic model

3. 3AT&T Labs - Research Part I: SNMP Traffic Data

4. 4AT&T Labs - Research Data Availability – Traffic Data

5. 5AT&T Labs - Research Data Availability – packet traces Packet traces limited availability special equipment needed (O&M expensive even if box is cheap) lower speed interfaces (only recently OC48 available, no OC192) huge amount of data generated

6. 6AT&T Labs - Research Data Availability – flow level data Flow level data not available everywhere historically poor vendor support (from some vendors) large volume of data (1:100 compared to traffic) feature interaction/performance impact

7. 7AT&T Labs - Research Data Availability – SNMP SNMP traffic data MIB II (including IfInOctets/IfOutOctets) is available almost everywhere manageable volume of data no significant impact on router performance

8. 8AT&T Labs - Research SNMP zAdvantages (MIB-II: IfInOctets/IfOutOctets) ySimple, Easy, available anywhere that supports SNMP yRelatively low volume yIt is used by operations already (lots of historical data) zDisadvantages yData quality xAmbiguous xMissing data xIrregular sampling yOctets counters only tell you link utilizations xHard to get a traffic matrix xCan’t tell what type of traffic xCan’t easily detect DoS, or other unusual events yCoarse time scale (>1 minute typically) xLack of well tested relationship between coarse time-scale averages and performance (hence active perf. measurement)

9. 9AT&T Labs - Research SNMP traffic data SNMP Polls SNMP Octets Counter poller router poll data Like an Odometer 999408 Management system agent

10. 10AT&T Labs - Research Irregularly sampled data zWhy? yMissing data (transport over UDP, often in-band) yDelays in polling (jitter) yPoller sync xMultiple pollers xStaggered polls zWhy care? yTime series analysis yComparisons between links xDid traffic shed from link A go to link B xCalculation of traffic matrices yTotals (e.g. total traffic to Peer X) yCorrelation to other data sources xDid event BGP route change at time T effects links A,B,C,…

11. 11AT&T Labs - Research Applications zCapacity planning yNetwork at the moment is “hand-crafted” yWant to automate processes yProvisioning for failure scenarios requires adding loads zTraffic engineering yEven if done by hand, you need to see results yBGP zEvent detection yOperations are “fire-fighters” yDon’t care about events if they go away yDon’t see patterns zBusiness cases yHelp sales and marketing make cases

12. 12AT&T Labs - Research Part II: Wavelet Analysis zMulti-scale zMulti-resolution

13. 13AT&T Labs - Research Discrete Wavelet Transform zReplace sinusoidal basis functions of FFT with wavelet basis functions zImplementation in pyramidal filter banks HP FIR LP FIR 2 2 HP FIR LP FIR 2 2 HP FIR LP FIR 2 2

14. 14AT&T Labs - Research Dyadic grid zno redundancy, no loss of information zEach frequency/scale examined at a resolution matched to its scale Scale 1 2 3 4 time

15. 15AT&T Labs - Research Dyadic grid: smoothing zZero the fine scale details and reconstruct Scale 1 2 3 4 time

16. 16AT&T Labs - Research Dyadic grid: compression zKeep the coefficients above some threshold Scale 1 2 3 4 time

17. 17AT&T Labs - Research What can you do with wavelets zCompression zSmoothing/interpolation zAnomaly detection/identification yDoS yFlash crowds zMultiple dimensional analysis of data zLRD/self-similarity analysis

18. 18AT&T Labs - Research Example: compression

19. 19AT&T Labs - Research Example: compression (by averaging)

20. 20AT&T Labs - Research Example: compression (Haar)

21. 21AT&T Labs - Research Example: compression (Daubechie’s)

22. 22AT&T Labs - Research Example: interpolation zWavelet based

23. 23AT&T Labs - Research Example: anomaly detection zWavelet based

24. 24AT&T Labs - Research Wavelets, wavelets everywhere and not a … zParameter tuning yHow do know it will work next time? zScale of dyadic grid doesn’t match patterns in data y5 minute measurements y24 hour cycle, 7 day cycle yBut dyadic grid is in powers of 2 yCWT looses many of the advantages of DWT zExample yCompression yLook for parameters/wavelet that don’t loose important data yWhat is the important data? zIf we had a model it could tell us what is important yCompress => estimate model parameters => test difference

25. 25AT&T Labs - Research Part III: Modeling zPutting together theory from yTime series analysis yTraffic theory zTo SNMP data yIn particular for backbone traffic

26. 26AT&T Labs - Research Total traffic into a city for 2 weeks

27. 27AT&T Labs - Research Model zTraffic data has several components yTrend, T t xLong term changes in traffic ySeasonal (periodic) component, S t xDaily and weekly cycles yStationary stochastic component, W t xNormal variation yTransient anomalies, I t xDoS, Flash crowds, Rerouting (BGP, link failures) zmany ways you could combine these components ystandard time series analysis xSum X t = T t + S t + W t + I t xProduct X t = T t S t W t I t xBox-Cox transform

28. 28AT&T Labs - Research A Simple Model (for backbone traffic) zBased on Norros model zNon-stationary mean zStochastic component unspecified (for the moment)

29. 29AT&T Labs - Research Why this model? zBehaves as expected under multiplexing zGood model for backbone traffic yLots of multiplexing zSimple, estimable parameters, flexible, can make predictions, data supports it

30. 30AT&T Labs - Research What does a model get you? zDecomposition yMA for trend (window > period of seasonal component) ySMA for seasonal component (average at same time of day/week) ySeveral methods for segmenting I t zInterpolation yLinear, or wavelet based for short gaps (<3 hours) yModel based for long gaps (>3 hours) zUnderstanding of the effect of multiplexing yShould be understood xPeople still seem to misunderstand yHow smooth is backbone traffic (is it LRD) zCapacity planning

31. 31AT&T Labs - Research Example: decomposition Data => Decomposition trend

32. 32AT&T Labs - Research Example: interpolation zModel based vs linear

33. 33AT&T Labs - Research Conclusion zSNMP is a good data source yAvailable everywhere yYou need to do some work to extract useful data yThere is still more info. to get (packet traces, flow data, …) zWavelets are a flexible tool for extracting info yNot always obvious how to set parameters zTraffic model gives you a little more yA framework for other algorithms yA way to decide what information is important yA way of seeing how smooth traffic really is xEffect of multiplexing zAlgorithms are applicable to other traffic data