1. CS 4700 / CS 5700 Network Fundamentals
Christo Wilson
8/22/2012
CS 4700 / CS 5700 Network Fundamentals
Lecture 16: IXPs
(The Underbelly of the Internet)
Revised 3/23/2015
Defense

2. Outline
Emerging Internet Trends
Internet Exchange Points (IXPs)

3. The Internet as a Natural System
You’ve learned about the TCP/IP Internet
Simple abstraction: Unreliable datagram transmission
Various layers
Ancillary services (DNS)
Extra in-network support
So what does the Internet look like?

4. What does the Internet look like?

5. What does the Internet look like?

6. Characterization challenges
Limited measurements and models can hint at it
Traceroute does not give us a complete view
Gao-Rexford (policy routing) doesn’t capture everything
What is the Internet actually being used for?
Emergent properties impossible to predict from protocols
Requires measuring the network
Constant evolution makes it a moving target

7. How is the Internet used?

8. How is the Internet used?

9. Measuring the capital-I Internet*
Measuring the Internet is hard
Significant previous work on
Router and AS-level topologies
Individual link / ISP traffic studies
Synthetic traffic demands
But limited “ground-truth” on inter-domain traffic
Most commercial arrangements under NDA
Significant lack of uniform instrumentation
*Mainly borrowed stolen from Labovitz 2010

10. Conventional Wisdom (i.e., lies)
Internet is a global scale end-to-end network
Packets transit (mostly) unmolested
Value of network is global addressability /reachability
Broad distribution of traffic sources / sinks
An Internet “core” exists
Dominated by a dozen global transit providers (tier 1)
Interconnecting content, consumer and regional providers

11. Traditional view

12. Does this still hold? Emergence of ‘hyper giant’ services
How much traffic do these services contribute?
Hard to answer!
Reading: Labovitz 2010 tries to look at this.

13. How do we validate/improve this picture?
Measure from
110+ ISPs / content providers
Including 3,000 edge routers and 100,000 interfaces
And an estimated ~25% all inter-domain traffic
Do some other validation
Extrapolate estimates with fit from ground-truth data
Talk with operators

14. Where is traffic going? Increasingly: Google and Comcast
Christo Wilson
8/22/2012
Where is traffic going?
Increasingly: Google and Comcast
Tier 1 still has large fraction, but large portion of it is to Google!
Why?
Consolidation of traffic
Fewer ASes responsible for more of the traffic
Over time Google begins delivering YT’s traffic
As of 2009 Google is 6% of traffic
Defense

15. Why is this happening?

16. Transit is dead! Long live the eyeball!
Commoditization of IP and Hosting / CDN
Drop of price of wholesale transit
Drop of price of video / CDN
Economics and scale drive enterprise to “cloud”
Consolidation
Bigger get bigger (economies of scale)
e.g., Google, Yahoo, MSFT acquisitions
Success of bundling / Higher Value Services – Triple and quad play, etc.
New economic models
Paid content (ESPN 3), paid peering, etc.
Difficult to quantify due to NDA / commercial privacy
Disintermediation
Direct interconnection of content and consumer
Driven by both cost and increasingly performance

17. New applications + ways to access
Fixed vs. Mobile Usage

18. New applications + ways to access
Fixed vs. Mobile Usage

19. The shift from hierarchy to flat$
Verizon
AT&T
Tier 1 ISPs
(settlement free peering)
$$$
Money follows the arrows.
Sprint $ $
Tier 2 ISPs
Regional Access
Provider
Regional Access
Provider
Tier 3 ISPs $ $
Local Access
Provider
Local Access
Provider $ $
Autonomous systems (ASes) connect to each other based on business relationships.
Businesses/consumers

20. The shift from hierarchy to flat
Verizon
AT&T
Tier 1 ISPs
(settlement free peering)
Sprint
Tier 2 ISPs
Regional Access
Provider
Regional Access
Provider
Local Access Provider doesn’t have to pay for consumer access to content!
Content provider no longer needs to pay for transit!
More “eyeballs” less $$
Tier 3 ISPs
Local Access
Provider
Local Access
Provider $ $
IXP $
Businesses/consumers

21. A more accurate model?

22. How do ASes connect? Point of Presence (PoP)
Usually a room or a building (windowless)
One router from one AS is physically connected to the other
Often in big cities
Establishing a new connection at PoPs can be expensive
Internet eXchange Points (IXP)
Facilities dedicated to providing presence and connectivity for large numbers of ASes
Many fewer IXPs than PoPs
Economies of scale

23. IXPs Definition Industry definition (according to Euro-IX)
A physical network infrastructure operated by a single entity with the purpose to facilitate the exchange of Internet traffic between Autonomous Systems
The number of Autonomous Systems connected should be at least three and there must be a clear and open policy for others to join.

24. IXPs worldwide

25. Inside an IXP Connection fabric Also a route server
Can provide illusion of all-to-all connectivity
Lots of routers and cables
Also a route server
Collects and distributes routes from participants

26. IXPs offer connectivity to ASes enable peering
Structure
IXPs offer connectivity to ASes enable peering

27. Inside an IXP

28. IXPs – Publicly available information

29. How much traffic is at IXPs?*
We don’t know for sure!
Seems to be a lot, though.
One estimate: 43% of exchanged bytes are not visible to us
Also 70% of peerings are invisible
*Mainly borrowed stolen from Feldmann 2012

30. Revised model 2012+