Co-Founder & Chief Technical Liaison The Next Wave of Massive Disruptions to the Peering Ecosystem William B. Norton Co-Founder & Chief Technical Liaison Equinix, Inc. Global Peering Forum March 26, 2007 30-minute-disruption

On the Internet Everyone is a Publisher

Internet Operations White Papers “Interconnection Strategies for ISPs” “Internet Service Providers and Peering” “A Business Case for Peering” “The Art of Peering: The Peering Playbook” “The Peering Simulation Game” “Do ATM-based Internet Exchanges Make Sense Anymore?” “Evolution of the U.S. Peering Ecosystem” “The Asia Pacific Internet Peering Guidebook” “The Folly of Peering Traffic Ratios?” Freely available. See Web site or send e-mail to wbn@equinix.com Internet makes anyone a publisher, similar effect now emerging for video

Massive Disruption in U.S. Peering Ecosystem  Short Videos YouTube – founded 2005 Short video clips – 50 million view per day! 20Gbps of peering traffic Feb 2006 $1M/month in Sept 2006! Entering Peering Ecosystem 30 Other competitors600Gbps peerable? DoveTail Video may dwarf current peered traffic 2010 – 80-90% Internet is Video Inculcate video guys into peering ecosystem On the Internet Everyone is a Broadcaster Short video clips…Full TV shows… Source: http://digg.com/tech_news/YouTube_Gets_Bandwidth_Boost_from_Level_3 Source: http://www.nanog.org/mtg-0606/norton.html

Massive Disruption in U.S. Peering Ecosystem  Full Episodes “Desperate Housewives” – 210MB/hour For 320x240 H.264 Video iTunes image 10,000,000 households 2,100,000,000 MB = 2.1 peta-Bytes How long will that take to download? 3 days @ 64Gbps non-stop ! Just one show Try 250M*180 Channels*HDTV Historical Perspective…review 5yr disruptions… Source: http://www.pbs.org/cringely/pulpit/pulpit20060302.html

Research Topic Massive Wave of Internet Traffic 90% of all Internet bits by 2010 How will Video Service Providers distribute this massive amount of Video Traffic over the Internet? How will Video Service Provider distribute?

Modeling the Video Service Provider Distribution Networks Four Models Commodity Transit CDN Transit/Peering/DIY CDN Peer2Peer Four Load Models A: Small Load B: Medium Load C: Large Load Goal : estimate cost : $/video downloaded

Model 1: Commodity Transit Business Premise: VSP focuses on core competence Transit Providers handle traffic better and cheaper Economies of scale, Aggregation, Expertise, Billing, Peering, etc.

Model 1A: Transit Light Load

Model 1B: Transit Medium Load

Model 1C Upstream ISPs Router4 Router2 10G Router2 8 * 10GE to upstreams each Server1 GigE Switch1 Router1 : Server24 : 10G : : GigE Switch14 Server262 Server263 Server264 : Distribution GigE Switch 48 port GigE for servers 2 10GE for upstream $10,000 Add another every 24 servers Routers Cisco 6509Sup720-3bxl w/4*4-port 10GE, $150,000 80Gbps from switches, 80Gbps to upstreams

Model 2: Content Delivery Networks (CDNs) for the Distribution of Video Content Business Premise: Single-site transit traffic traverses potentially many network devices, increasing latency and the potential of packet loss: By spreading web objects closer to the eyeball networks latency is reduced Fewer network elements are traversed so reliability is improved Congestion points in the core of the Internet are avoided CDNs have the expertise, deployed infrastructure, economies of scale from aggregation efficiencies.

Assumption CDN Price Points

Model 2A: CDN Light Load

Model 2A: CDN Light Load

Model 2B: CDN Medium Load

Model 2C: CDN Large Load

Model 3: Transit/Peering/DIY CDN Business Model Premise: Operation of the Internet distribution is seen as strategic to the VSP: End-user experience is mission-critical so outsourcing the end user experience to a transit provider or CDN puts the VSP at risk. The VSP has visibility into what video are being released, which ones are likely to be hot and which ones don’t require special infrastructure adjustments. Internet Video distribution is so new that the VSP prefers control. This is a strategic focus of the VSP: ensuring reliability, scalability, through the constant monitoring and evolution of the infrastructure to ensure the end user experiences during these early phases of Internet Video Distribution. The traditional CDN may be ill-suited to distribute very large video object, therefore we have to do it yourself.

Model 3: Transit/Peering Light Load

Model 3A: Peering/Transit Light Load

Model 3B: Transit/Peering Medium Load

Model 3C: Transit/Peering Heavy Load

Model 4: Peer2Peer Business Model Premise: The current Internet Service Providers and CDNs at the core can not handle the load across single or even multiple locations: Backbone, peering interconnects, and the hundreds of thousands of routers deployed can not handle the load of today and tomorrows video. the leaf nodes (i.e. Grandma’s PCs left on) in aggregate have the cycles and network capacity, if shared, to distribute popular content today. Popular content can be chopped up into small chunks such that many downloaders become sources, and topologically close downloaders will prefer the topologically close sources. This ‘swarmcasting’ requires only a source ‘seed’, and a lookup mechanism for the first downloaders to find the seed, and then to direct future downloaders to topologically closer sources.

Model 4A: P2P Light Load

Model 4A: P2P Light Load

Model 4B: P2P Medium Load

Model 4B: P2P Medium Load

Model 4C: P2P Large Load

Model 4C: P2P Large Load

Summary Per Video Cost Of delivery

Observation/Implications Internet Transit Supply ▼ Internet Transit Price ▲ Internet Transit Model  src/dst specific Bottlenecks IX Power, Router Capacity, Peer’s Capacity, Last Mile bottleneck Do I need to upgrade $$$$ gear to support my competitor (peer)? Geoff Huston: “P2P has won. Telco/Cable co trying to keep its 1998 biz plan relevant.” I look forward to discussing this over the next few days.