1 Internet Traffic Measurement and Modeling Carey Williamson Department of Computer Science University of Calgary

2 Network Traffic Measurement u A focus of networking research since the late 1980’s u Collect data or packet traces showing packet activity on the network for different network applications

3 Purpose u Understand the traffic characteristics of existing networks u Develop models of traffic for future networks u Useful for simulations, planning studies

4 Measurement Environments u Local Area Networks (LAN’s) u e.g., Ethernet LANs u Wide Area Networks (WAN’s) u e.g., the Internet u Wireless Local Area Networks (WLANs) u e.g., U of C WLAN

5 Requirements u Network measurement requires hardware or software measurement facilities that attach directly to network u Allows you to observe all packet traffic on the network, or to filter it to collect only the traffic of interest u Assumes broadcast-based network technology, superuser permission

6 Measurement Tools (1 of 3) u Can be classified into hardware and software measurement tools u Hardware: specialized equipment u Examples: HP 4972 LAN Analyzer, DataGeneral Network Sniffer, others... u Software: special software tools u Examples: tcpdump, xtr, SNMP, others...

7 Measurement Tools (2 of 3) u Measurement tools can also be classified as active or passive u Active: the monitoring tool generates traffic of its own during data collection (e.g., ping, traceroute, pchar) u Passive: the monitoring tool observes and records traffic info, while generating none of its own (e.g., tcpdump)

8 Measurement Tools (3 of 3) u Measurement tools can also be classified as real-time or non-real-time u Real-time: collects traffic data as it happens, and may even be able to display traffic info as it happens u Non-real-time: collected traffic data may only be a subset (sample) of the total traffic, and is analyzed off-line (later)

9 Potential Uses (1 of 4) u Protocol debugging u Network debugging and troubleshooting u Changing network configuration u Designing, testing new protocols u Designing, testing new applications u Detecting network weirdness: broadcast storms, routing loops, etc.

10 Potential Uses (2 of 4) u Performance evaluation of protocols and applications u How protocol/application is being used u How well it works u How to design it better

11 Potential Uses (3 of 4) u Workload characterization u What traffic is generated u Packet size distribution u Packet arrival process u Burstiness u Important in the design of networks, applications, interconnection devices, congestion control algorithms, etc.

12 Potential Uses (4 of 4) u Traffic modeling u Construct synthetic workload models that concisely capture the salient characteristics of actual network traffic u Use as representative, reproducible, flexible, controllable workload models for simulations, capacity planning studies, etc.

13 Summary of Key Measurement Results u The following represents my own synopsis of the “Top 10 Observations” from network trafffic measurement and modeling research in the last 20 years u Not an exhaustive list, but hits most of the highlights u For more detail, see papers (or ask!)

14 Observation #1 u The traffic model that you use is extremely important in the performance evaluation of routing, flow control, and congestion control strategies u Have to consider application-dependent, protocol-dependent, and network- dependent characteristics u The more realistic, the better (GIGO)

15 Observation #2 u Characterizing aggregate network traffic is difficult u Lots of (diverse) applications u Just a snapshot: traffic mix, protocols, applications, network configuration, technology, and users change with time

16 Observation #3 u Packet arrival process is not Poisson u Packets travel in trains u Packets travel in tandems u Packets get clumped together (ack compression) u Interarrival times are not exponential u Interarrival times are not independent

17 Observation #4 u Packet traffic is bursty u Average utilization may be very low u Peak utilization can be very high u Depends on what interval you use!! u Traffic may be self-similar: bursts exist across a wide range of time scales u Defining burstiness (precisely) is difficult

18 Observation #5 u Traffic is non-uniformly distributed amongst the hosts on the network u Example: 10% of the hosts account for 90% of the traffic (or 20-80) u Why? Clients versus servers, geographic reasons, popular ftp sites, web sites, etc.

19 Observation #6 u Network traffic exhibits ‘‘locality’’ effects u Pattern is far from random u Temporal locality u Spatial locality u Persistence and concentration u True at host level, at gateway level, at application level

20 Observation #7 u Well over 80% of the byte and packet traffic on most networks is TCP/IP u By far the most prevalent u Often as high as 95-99% u Most studies focus only on TCP/IP for this reason (as they should!)

21 Observation #8 u Most conversations are short u Example: 90% of bulk data transfers send less than 10 kilobytes of data u Example: 50% of interactive connections last less than 90 seconds u Distributions may be ‘‘heavy tailed’’ (i.e., extreme values may skew the mean and/or the distribution)

22 Observation #9 u Traffic is bidirectional u Data usually flows both ways u Not JUST acks in the reverse direction u Usually asymmetric bandwidth though u Pretty much what you would expect from the TCP/IP traffic for most applications

23 Observation #10 u Packet size distribution is bimodal u Lots of small packets for interactive traffic and acknowledgements u Lots of large packets for bulk data file transfer type applications u Very few in between sizes

24 Summary u There has been lots of interesting network measurement work in the last years u LAN, WAN, and Video measurements u Network traffic self-similarity u Web, P2P, and streaming systems