Internet Measurement 5.4 State of the art ECE Department, University of Tehran Fall 2009

Outline Equipment Properties Topology Properties  Static properties of AS graph  Static properties of Router graph  Dynamic aspects of topology  Geographic Location Interaction of Traffic and Network  Packet Delay  Packet Loss  Packet Reordering, Duplication and Jitter  Bandwidth and Throughput 2

Equipment Properties Different types of equipment:  Links and other communication devices that are highly predictable  Routers that are more complex  NATs and Firewalls: delay on the order of 100s of miliseconds 3

Equipment Properties - Routers Normally, routers has very small delay (order of tens of micro seconds). But if heavily loaded, delay is on the order of miliseconds. Additional sources of router delay  Periodic processes within router software  Packets carrying IP options  Packets leaving on different interfaces 4

Equipment Properties - Routers Delay that accounts for traffic production and consumption of routers  LSA processing is on the order of 100 microseconds. Mostly for data copying within the router  OSPF packet processing vary linearly with number of LSAs  Shortest path calculation scales quadratic with the number of nodes in a fully connected topology 5

Outline Equipment Properties Topology Properties  Static properties of AS graph  Static properties of Router graph  Dynamic aspects of topology  Geographic Location Interaction of Traffic and Network  Packet Delay  Packet Loss  Packet Reordering, Duplication and Jitter  Bandwidth and Throughput  6

Static properties of AS graph Highest level of internet topology AS graph is a pair G =(V,E)  V = Autonomous Systems  E = Existence of direct traffic G is highly variable in degree distribution that can often be approximated by power law Most ASes have low degree (<5), but a few ASes have thousands degree. 7

AS graph – Power Distribution Here is a sample created by fusion of different topology measurements (e.g. BGP-based, routing registries…) It was only an approximation in

AS graph – Power Distribution High variability in degree distribution  Some ASes are very highly connected  Different ASes have dramatically different roles in the network  Node degree seems to be highly correlated with AS size Generative models of AS graph  “Rich get richer” model  Newly added nodes connect to existing nodes in a way that tends to simultaneously minimize the physical length of the new connection, as well as the average number of hops to other nodes  New ASes appear at an exponentially increasing rate, and each AS grows exponentially as well 9

AS Graph is a Small World Graph Graphs with high clustering degree and low diameter AS graph taken in Jan 2002 containing 12,709 ASes and 27,384 edges  Average path length is 3.6  Clustering coefficient is 0.46 ( in random graph)  It appears that individual clusters can contain ASes with similar geographic location or business interests  ASes of high degree are likely top-tier ASes. 10

AS Traffic Exchange Policies Four relationships  Customer-provider  Peering Exchange only non-transit traffic  Mutual transit typically between two administrative domains such as small ISPs who are located close to each other Enforced by BGP route advertisement AS Hierarchical structure? 11

Router Graph Router graph is a pair G =(V,E)  V = Routers  E = Existence of direct connection  Impossible to obtain a complete Internet topology. But we can Focus on a single AS subgraph 12

ARPANET Router graph, 1972 IMP: Interface Message Processors, TIP:Terminal IMP The first message ever sent over the ARPANET; it took place at 10:30PM on October 29,

Router-level Topology Abilene Network, Research and educational backbone of USA 14

Static Properties of Router Graph High variability in degree distribution Most nodes have degree less than 5 but some can have degrees greater than 100 Sampling bias (e.g. by traceroute)  Small set of sources with much larger set of destinations  Nodes and links closest to the sources are explored much more thoroughly  Majority of edges are far and so undersampled  Artificially increase the proportion of low- degree nodes in the sampled graph 15

Generative Router-level Topology Based on network robustness & technology constrains Network edge:  High degree nodes  Serve many users with low bandwidth. Network cores:  More likely to be meshes for robustness.  High bandwidth is essential. 16

Static Properties of Router Graph AS vs. Router Graphs  AS graph can be measured passively (Using BGP tables and traffic)  Router graph needs active measurement (Using traceroute that can be slow in practice)  They have different graph structures  In AS graph highly connected central nodes but in router graph they are edges. AS Path properties  Average length around 16, rare paths longer than 30 hops  Path inflation, because of AS-AS policies and interdomain routings 17

Dynamic Aspects of Topology Internet Growth Number of unique AS numbers advertised within the BGP system 18

Dynamic Aspects of Topology Difficult to measure the number of routers  DNS is decentralized  Router-level graph changes rapidly Different ways of measuring number of end systems and routers:  Regional Internet registeries (RIR)  BGP system  Use ping  Query DNS 19

Measuring Number of Routers - RIR Counting number of addresses in RIRs:  Is a serious overestimate as many addresses are not in use. 20

Measuring Number of Routers - RIR Pinging IP addresses is a serious underestimate  Intermittent connection (e.g. Dial up, Wireless)  Network Address Translation (NAT) & Firewalls Query DNS is also an underestimate  All connected hosts do not have DNS name No. of addresses in the BGP system is an overestimate  Prefixes contain addresses not in use 21

Registered Hosts in DNS During the 1990s Internet growing exponentially Slowed down somewhat today Rapid growth means more difficulty for measuring router graph In million 22

Stability of Internet Sources of instability:  Failures, restarts and reconfigurations of network infrastructure changes network topology: Graph structure (nodes, edges) will be altered Routes become unstable  Router misconfiguration and policy changes May need long sequence of BGP updates, packet delay and loss 23

Stability of Internet Router level instability  Some routes exhibit significant fluctuation Consistent with the behavior AS-level paths High variability of route stability  Majority of routes going days or weeks without change High variability of route unavailable duration  Causes of instability of the router graph Failure of links  Majority of link failures are concentrated on a small subset of the links  Marjority of link failures are short-lived (<10min) Router failure 24

Geographic Location Relation of Net. Infrastructure with population, social organization and economic activity Online per interface is a good measure IP addresses 25

Outline Equipment Properties Topology Properties  Static properties of AS graph  Static properties of Router graph  Dynamic aspects of topology  Geographic Location Interaction of Traffic and Network  Packet Delay  Packet Loss  Packet Reordering, Duplication and Jitter  Bandwidth and Throughput  26

Traffic and Network - Delay Packet delay (RTT): high variable distribution  Deterministic delays: Propagation delay & Transmission delay  Stochastic Delays: Forwarding delay (queue) Transmission delay  Only significant on slow access links.  e.g. 1500B takes 1.2 mic.sec on OC192 and 200 milisec. for a 56k modem Propagation delay  Influenced by geographic distance minRTT = Propagation delay + Transmission delay, so it is deterministic  Is strongly affected by topology  can be estimated by Euclidean space 27

Traffic and Network - Delay Forwarding delay and congestion events  Can dominate the deterministic delays Often minRTT<50 mil.sec, maximum 200 mil.sec Stochastic can cause RTT=100s mil.sec or 10s sec.  Often cause high ‘spikes’ in RTTs as a result of Routing changes Temporary queues within routers (Weibull or Pareto distribution)  It is not clear which link is responsible for congested links 28

Traffic and Network - Loss Main sources of packet loss  Congestion within routers  For wireless: packet corruption, radio interference and multipath fading Often occur in ‘bursts’  Because of congestion in routers  Packet loss show correlation over time scales up to 1000 mili.sec Can be modeled using 2,3-state Markov model Generally <0.1% over wired paths but 2% for wireless paths 29

Traffic and Network - Reordering Reordering: inverting order of 2 packets  Parallelism within a router  Load sharing between network paths  Route changes Out of sequence arrival reasons:  True packet reordering  Packet duplication  Often for retransmission by TCP sender Accounts for 1-2% or less of a long-lived TCP connection. 30

Packet Duplication and Jitter Packet duplication rarely happen. sources:  Measurement point is inside a router  Network actually duplicates packets Packet jitter: Non-uniformity of inter-packet gaps  Queuing: can be 100s – 1000s mili.sec. 31

Bandwidth and Throughput Much less known compared to other metrics But declining loss and RTT shows thoughput growth Throughput is a steady metric that changes slowly (e.g. factor of 3 over an hour) 32

Conclusion Internet users in the middle east Internet World Stats: – Nov 2010http:// 33

Any Question ? 34 Internet Infrastructure measurement