1. William B. Norton Co-Founder & Chief Technical Liaison Equinix, Inc.
Internet Video: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem
William B. Norton
Co-Founder & Chief Technical Liaison
Equinix, Inc.
V1.5

2. Internet Operations White Papers
“Interconnection Strategies for ISPs”
“Internet Service Providers and Peering”
“A Business Case for Peering”
“The Art of Peering: The Peering Playbook”
“The Peering Simulation Game”
“Do ATM-based Internet Exchanges Make Sense Anymore?”
“Evolution of the U.S. Peering Ecosystem”
“The Asia Pacific Internet Peering Guidebook”
“The Great (Public vs. Private) Debate”
“The Folly of Peering Traffic Ratios?”
“Video Internet: The Next Wave….”
On the Internet,
Everyone if a Publisher
These are 11 of these white papers out there today.
I met a professor at the APRICOT last year in Perth, Australia that was involved with a distance learning class taught in Japan and the US. Simultaneously. The Japanese students read the “Evolution of the U.S. Peering Ecosystem” paper and the U.S. Students read the “Asia Pacific Internet Peering Guidebook” white paper, and then they presented and discussed it over the video link.
One class had the students read a white paper and post on the web 3 points the author made, 3 criticisms of the paper and 3 suggestions for further research, and there was an online bulletin board forum for students discussion. It was insightful as an author to read their reviews and criticisms – talk about having your work under a microscope!
But I walked away from this experience primarily with a sense of awe in how far these papers had traveled, and that on the Internet everyone is a publisher.
Internet makes anyone a publisher, similar effect now emerging for video

3. Massive Disruption in U.S. Peering Ecosystem  Short Videos
YouTube – founded 2005
Short video clips – 50 million view per day!
20Gbps of peering traffic Feb 2006
$1M/month in Sept 2006!
Entering Peering Ecosystem
30 Other competitors600Gbps peerable?
DoveTail
Video may dwarf current peered traffic
2010 – 80-90% Internet is Video
Inculcate video guys into peering ecosystem
Now, On the Internet
Everyone is a Broadcaster
Last year, YouTube was founded as a way for people to share videos with their friends. Remember I said that on the Internet everyone is a publisher. These guys escalated that to “On the Internet everyone is a broadcaster”. They provide a very nice community based web site allowing sharing of video clips with friends or globally, and currently get over 50 million hits per day!
Back in the early days, a good candidate for peering was someone who bought transit for a few hundred megabits per second, and these guys, only a year old are currently pushing over 20Gbps! And now they are entering the Peering Ecosystem. Further, my Venture Capital friends tell me that whenever a startup is launched there are generally at least 30 competitors following the path. This suggests that the U.S. Peering Ecosystem will be inundated by up to 600Gbps of potentially peerable traffic. At NANOG last week I spoke with folks about this and there seemed to be concensus that the peered video traffic would dwarf the currently peered traffic, and by 2010 would represent 80-90% of Internet traffic. This is a massive tidal wave of traffic that will be coming, either over your transit link that you pay for, or over a free peering link at an exchange point. This is a significant disruption to the US Peering Ecosystem and it is happening right now.
All of this information by the way came from the NANOG 37 Peering BOF YouTube Peering Personals presentation as shown by the URL at the bottom of the slide.
Short video clips…Full TV shows…
Source:
Source:

5. Massive Disruption in U.S. Peering Ecosystem  Full Episodes
“Desperate Housewives” – 210MB/hour
For 320x240 H.264 Video iTunes image
10,000,000 households
2,100,000,000 MB = 2.1 peta-Bytes
How long will that take to download?
3 64Gbps non-stop !
Just one show
Try 250M*180 Channels*HDTV
The Point: Massive Wave of Incremental Traffic to document…..
Source:

6. The Research Questions
How to distribute video across the Internet ?
How much does it cost per video?

7. Modeling $ per video? Transit =Metered pipe to the Internet
CDN =Content Distribution
Network
Peering =free & reciprocal
access to each others
Customers
P2P =PeerToPeer
Small =Distribute 10 videos
every 5 minutes on avg.
Medium =Distribute 100 videos
Large =Distribute 1000 videos
Modeling
Varying Sized Loads
$ per video?
Shift from Avg to more typical demand curve...

8. Demand Curve Modeling ρ=6.6xα
95th % ω~ 4 x α
α = average load
ρ = peak load (6.6*α)
ω =95th percentile (4*α)
ρ=6.6xα
α = 10 videos/5 min=400Mbps ω =1.6Gbps ρ = 2.64Gbps
α = 100 videos/5 min=4,000Mbps ω =16Gbps ρ = 26.4Gbps
α = 1000 videos/5 min=40,000Mbps ω =160Gbps ρ = 264Gbps

9. Model 1C – Large Load Commodity Transit
Upstream ISPs
Router4
Router2
10G
Router2
8 * 10GE to upstreams each
Server1
GigE
Switch1
Router1 :
Server24 :
10G :
Serious Operational Issue
Key point – This can’t work work today from any single locations because several of the largest Tier 1 ISPs in the U.S. today are turning away or queuing 10Gbps transit orders.
There are a cascading set of issues:
1) Sufficient IX Colo space/power is unavailable, and then
2) Financial: Gear that hasn’t been paid for yet needs to be replaced/upgraded with gear much more expensive, and then
3) Peers need to all upgrade their gear to handle increased peering load at n*10G, and then
4) Backbone capacity needs to be upgraded to deliver the video traffic to the last mile providers.
This is all made more difficult due to the lack of 40G/100G standards and hardware to easily handle such capacity.
All of these providers need to have an approximately coordinated upgrade approach. :
GigE
Switch14
Server262
Server263
Server264
But this can’t work today. Why? :

10. Model 2C: CDN Large Load

11. Model 3C: Transit/Peering Large Load

12. Model 4C: P2P Large Load

13. Summary
Per Video Cost
Of delivery

14. Observations Internet Transit Supply ▼Price ▲
Internet Transit Model  src/dst specific
Bottlenecks
IX Power, Router Capacity, Peer’s Capacity,
Backbone Capacity, Last Mile bottleneck, 100G NIC?
Do I need to upgrade $$$$ gear to support my competitor (peer)?
Identify Players, Positions, Motivations, Behavior
Geoff Huston: “P2P has won. Telco/Cable co trying to keep its 1998 biz plan relevant.”