1. 1 Network Traffic Measurement and Modeling Carey Williamson Department of Computer Science University of Calgary

2. 2 Network Traffic Measurement u A recent focus of networking research (mid to late 1980’s, early 1990’s) u Collect data or packet traces showing packet activity on the network for different network applications

3. 3 Purpose u Understand the traffic characteristics of existing networks u Develop models of traffic for future networks u Useful for simulations, planning studies

4. 4 Requirements u Network measurement requires hardware or software measurement facilities that attach directly to network u Allows you to observe all packet traffic on the network, or to filter it to collect only the traffic of interest u Assumes broadcast-based network technology, superuser permission

5. 5 Measurement Tools u Can be classified into hardware and software measurement tools u Hardware: specialized equipment u Examples: HP 4972 LAN Analyzer, DataGeneral Network Sniffer, others... u Software: special software tools u Examples: tcpdump, xtr, SNMP, others...

6. 6 Measurement Tools (Cont’d) u Measurement tools can also be classified as intrusive or non-intrusive u Intrusive: the monitoring tool generates traffic of its own during data collection u Non-intrusive: the monitoring tool is passive, observing and recording traffic info, while generating none of its own

7. 7 Measurement Tools (Cont’d) u Measurement tools can also be classified as real-time or non-real-time u Real-time: collects traffic data as it happens, and may even be able to display traffic info as it happens u Non-real-time: collected traffic data may only be a subset (sample) of the total traffic, and is analyzed off-line (later)

8. 8 Potential Uses of Tools u Protocol debugging u Network debugging and troubleshooting u Changing network configuration u Designing, testing new protocols u Designing, testing new applications u Detecting network weirdness: broadcast storms, routing loops, etc.

9. 9 Potential Uses of Tools (Cont’d) u Performance evaluation of protocols and applications u How protocol/application is being used u How well it works u How to design it better

10. 10 Potential Uses of Tools (Cont’d) u Workload characterization u What traffic is generated u Packet size distribution u Packet arrival process u Burstiness u Important in the design of networks, applications, interconnection devices, congestion control algorithms, etc.

11. 11 Potential Uses of Tools (Cont’d) u Workload modeling u Construct synthetic workload models that concisely capture the salient characteristics of actual network traffic u Use as representative, reproducible, flexible, controllable workload models for simulations, capacity planning studies, etc.

12. 12 Measurement Environments u Local Area Networks (LAN’s) u e.g., Ethernet LANs u Wide Area Networks (WAN’s) u e.g., the Internet u ATM Networks

13. 13 Some References u Raj Jain, ‘‘Packet Trains”, 1986 u Cheriton and Williamson, “VMTP”, 1987 u Chiu and Sudama, “DECNET Applications and Protocols”, 1988 u Gusella, “Diskless Workstations”, 1990 u Caceres, Danzig, Jamin, Mitzel, “Wide Area TCP/IP Traffic”, 1991

14. 14 Some References (Cont’d) u Paxson, “Measurements and Models of Wide Area TCP Traffic”, 1991 u Leland et al, “Self-Similarity”, 1993 u Garrett, Willinger, “VBR Video”, 1994 u Paxson, “Failure of Poisson Modeling”, 1994

15. 15 Summary of Measurement Results u The following represents my own synopsis of the “Top 10 Observations” from network measurement and monitoring research in the last 10 years u Not an exhaustive list, but hits most of the highlights u For more detail, see papers (or ask!)

16. 16 Observation #1 u The traffic model that you use is extremely important in the performance evaluation of routing, flow control, and congestion control strategies u Have to consider application-dependent, protocol-dependent, and network- dependent characteristics u The more realistic, the better (GIGO)

17. 17 Observation #2 u Characterizing aggregate network traffic is difficult u Lots of (diverse) applications u Just a snapshot: traffic mix, protocols, applications, network configuration, technology, and users change with time

18. 18 Observation #3 u Packet arrival process is not Poisson u Packets travel in trains u Packets travel in tandems u Packets get clumped together (ack compression) u Interarrival times are not exponential u Interarrival times are not independent

19. 19 Observation #4 u Packet traffic is bursty u Average utilization may be very low u Peak utilization can be very high u Depends on what interval you use!! u Traffic may be self-similar: bursts exist across a wide range of time scales u Defining burstiness (precisely) is difficult

20. 20 Observation #5 u Traffic is non-uniformly distributed amongst the hosts on the network u Example: 10% of the hosts account for 90% of the traffic (or 20-80) u Why? Clients versus servers, geographic reasons, popular ftp sites, web sites, etc.

21. 21 Observation #6 u Network traffic exhibits ‘‘locality’’ effects u Pattern is far from random u Temporal locality u Spatial locality u Persistence and concentration u True at host level, at gateway level, at application level

22. 22 Observation #7 u Well over 80% of the byte and packet traffic on most networks is TCP/IP u By far the most prevalent u Often as high as 95-99% u Most studies focus only on TCP/IP for this reason (as they should!)

23. 23 Observation #8 u Most conversations are short u Example: 90% of bulk data transfers send less than 10 kilobytes of data u Example: 50% of interactive connections last less than 90 seconds u Distributions may be ‘‘heavy tailed’’ (i.e., extreme values may skew the mean and/or the distribution)

24. 24 Observation #9 u Traffic is bidirectional u Data usually flows both ways u Not JUST acks in the reverse direction u Usually asymmetric bandwidth though u Pretty much what you would expect from the TCP/IP traffic for most applications

25. 25 Observation #10 u Packet size distribution is bimodal u Lots of small packets for interactive traffic and acknowledgements u Lots of large packets for bulk data file transfer type applications u Very few in between sizes

26. 26 Summary u There has been lots of interesting network measurement work in the last ten years u We will take a look at some of it soon u LAN, WAN, and Video measurements u Network traffic self-similarity