1. Co-Founder & Chief Technical Liaison
The Next Wave of
Massive Disruptions
to the Peering Ecosystem
William B. Norton
Co-Founder & Chief Technical Liaison
Equinix, Inc.
Global Peering Forum
March 26, 2007
30-minute-disruption

2. On the Internet
Everyone is a Publisher

3. Internet Operations White Papers
“Interconnection Strategies for ISPs”
“Internet Service Providers and Peering”
“A Business Case for Peering”
“The Art of Peering: The Peering Playbook”
“The Peering Simulation Game”
“Do ATM-based Internet Exchanges Make Sense Anymore?”
“Evolution of the U.S. Peering Ecosystem”
“The Asia Pacific Internet Peering Guidebook”
“The Folly of Peering Traffic Ratios?”
Freely available. See Web site or send to
Internet makes anyone a publisher, similar effect now emerging for video

4. Massive Disruption in U.S. Peering Ecosystem  Short Videos
YouTube – founded 2005
Short video clips – 50 million view per day!
20Gbps of peering traffic Feb 2006
$1M/month in Sept 2006!
Entering Peering Ecosystem
30 Other competitors600Gbps peerable?
DoveTail
Video may dwarf current peered traffic
2010 – 80-90% Internet is Video
Inculcate video guys into peering ecosystem
On the Internet
Everyone is a Broadcaster
Short video clips…Full TV shows…
Source:
Source:

6. Massive Disruption in U.S. Peering Ecosystem  Full Episodes
“Desperate Housewives” – 210MB/hour
For 320x240 H.264 Video iTunes image
10,000,000 households
2,100,000,000 MB = 2.1 peta-Bytes
How long will that take to download?
3 64Gbps non-stop !
Just one show
Try 250M*180 Channels*HDTV
Historical Perspective…review 5yr disruptions…
Source:

7. Research Topic Massive Wave of Internet Traffic
90% of all Internet bits by 2010
How will Video Service Providers distribute this massive amount of Video Traffic over the Internet?
How will Video Service Provider distribute?

8. Modeling the Video Service Provider Distribution Networks
Four Models
Commodity Transit
CDN
Transit/Peering/DIY CDN
Peer2Peer
Four Load Models
A: Small Load
B: Medium Load
C: Large Load
Goal : estimate cost : $/video downloaded

9. Model 1: Commodity Transit
Business Premise:
VSP focuses on core competence
Transit Providers handle traffic better and cheaper
Economies of scale, Aggregation, Expertise, Billing, Peering, etc.

10. Model 1A: Transit Light Load

11. Model 1B: Transit Medium Load

12. Model 1C Upstream ISPs Router4 Router2 10G Router2
8 * 10GE to upstreams each
Server1
GigE
Switch1
Router1 :
Server24 :
10G : :
GigE
Switch14
Server262
Server263
Server264 :
Distribution GigE Switch
48 port GigE for servers
2 10GE for upstream
$10,000
Add another every 24 servers
Routers
Cisco 6509Sup720-3bxl
w/4*4-port 10GE, $150,000
80Gbps from switches, 80Gbps to upstreams

13. Model 2: Content Delivery Networks (CDNs) for the Distribution of Video Content
Business Premise: Single-site transit traffic traverses potentially many network devices, increasing latency and the potential of packet loss:
By spreading web objects closer to the eyeball networks latency is reduced
Fewer network elements are traversed so reliability is improved
Congestion points in the core of the Internet are avoided
CDNs have the expertise, deployed infrastructure, economies of scale from aggregation efficiencies.

14. Assumption CDN Price Points

15. Model 2A: CDN Light Load

16. Model 2A: CDN Light Load

17. Model 2B: CDN Medium Load

18. Model 2C: CDN Large Load

19. Model 3: Transit/Peering/DIY CDN
Business Model Premise: Operation of the Internet distribution is seen as strategic to the VSP:
End-user experience is mission-critical so outsourcing the end user experience to a transit provider or CDN puts the VSP at risk.
The VSP has visibility into what video are being released, which ones are likely to be hot and which ones don’t require special infrastructure adjustments.
Internet Video distribution is so new that the VSP prefers control. This is a strategic focus of the VSP: ensuring reliability, scalability, through the constant monitoring and evolution of the infrastructure to ensure the end user experiences during these early phases of Internet Video Distribution.
The traditional CDN may be ill-suited to distribute very large video object, therefore we have to do it yourself.

20. Model 3: Transit/Peering Light Load

21. Model 3A: Peering/Transit Light Load

22. Model 3B: Transit/Peering Medium Load

23. Model 3C: Transit/Peering Heavy Load

24. Model 4: Peer2Peer
Business Model Premise: The current Internet Service Providers and CDNs at the core can not handle the load across single or even multiple locations:
Backbone, peering interconnects, and the hundreds of thousands of routers deployed can not handle the load of today and tomorrows video.
the leaf nodes (i.e. Grandma’s PCs left on) in aggregate have the cycles and network capacity, if shared, to distribute popular content today.
Popular content can be chopped up into small chunks such that many downloaders become sources, and topologically close downloaders will prefer the topologically close sources. This ‘swarmcasting’ requires only a source ‘seed’, and a lookup mechanism for the first downloaders to find the seed, and then to direct future downloaders to topologically closer sources.

25. Model 4A: P2P Light Load

26. Model 4A: P2P Light Load

27. Model 4B: P2P Medium Load

28. Model 4B: P2P Medium Load

29. Model 4C: P2P Large Load

30. Model 4C: P2P Large Load

31. Summary
Per Video Cost
Of delivery

32. Observation/Implications
Internet Transit Supply ▼
Internet Transit Price ▲
Internet Transit Model  src/dst specific
Bottlenecks
IX Power, Router Capacity, Peer’s Capacity,
Last Mile bottleneck
Do I need to upgrade $$$$ gear to support my competitor (peer)?
Geoff Huston: “P2P has won. Telco/Cable co trying to keep its 1998 biz plan relevant.”
I look forward to discussing this over the next few days.