A Business Case for Peering William B. Norton Co-Founder and Chief Technical Liaison July 17th, 2001 Gigabit Peering Forum III Dallas, TX

Motivation “Internet Service Providers and Peering” 2 year study of Peering Coordinators Document Process, Mindset, Strategies Documented Peering Processes Learned a lot about Peering Coordinators Peering Coordinators Professionals

Peering is a game of relationships

“Internet Service Providers and Peering” White Paper Documents: The Motivations of Peering Coordinators The 3 Stage Generalized Process of Peering 1) Identification of Peering Candidates 2) Negotiation of Peering 3) Implementation of Peering Exchange Point Selection Criteria Copies available here today

A Business Case for ISP Peering Overcome the CFO Hurdle: “Why should I spend money on these non-revenue generating activities?” “I think you engineers just want more toys to play with.” “Convince me this makes business sense” CFO argument…<comments>…

Pretend you are the CFO… Presentation Outline: Definitions Transit & Peering Terminology and Financial Cost Models Motivations for Peering Save Money and Improve Network Efficiency Introduction to the Peering Breakeven Analysis Specific Examples leading to Generalizations “A Business Case for Peering” Peering Break Even analysis Available as a White Paper…

Definitions Transit Peering Effective Peering Bandwidth Gives us tools (lexicon) to facilitate discussion Market Confusion over misuse of terms: Terminology critical to have meaningful discussion on Peering Transit Peering Effective Peering Bandwidth

Understand that… The Internet is a network of networks Internet Service Providers sell access to the Internet Internet Service Providers must themselves connect to the Internet. How do they connect to the rest of the Internet?

Definition of Transit Upstream Transit Providers Upstream Transit W O K S Upstream Transit Providers Upstream Transit Provider Upstream Transit Provider Upstream Transit Provider Definition of Transit Def: Transit is the business relationship whereby one ISP announces (sells) reachability to the *entire* Internet to a customer. 2) Reachability Announcement ISP A 1) ISP A buys Transit Service 3) Traffic Flows $$$ ? Typically usage-based Pricing ’01: $100-$1200/Mbps Volume based on 95th Percentile measure Transit is Simple, Convenient: Upstream handles the delivery of packets to the Internet by some means

Cost of Transit Traffic Exchange * Source: Wolfgang Tremmel (Via Net.works), UC quote as well Traffic Volume increasing

Traffic Volume Increasing Streaming Media Yahoo! Broadcast: 100,000 concurrent unicast streams 15 million streaming hrs/mo, 1300 Live events/day U P S T R E A M Streaming: Broadcast Telephony Video Multimedia Interactive Gaming SPOT Events IG B M $$$ T Content Heavy ISP V $$$ Access Heavy ISP 56k 384k 1.5m 1 T1 consumer streaming 24/7 cost ~$400/mo in transit Why not just buy transit?

Why Not just buy transit? Motivations for Peering: Financial: Reduce load on expensive Transit service Traffic src/dest Measure vs Intuit Usage-based Billing Engineering: Lower latency 1st Stage of Peering: Top 10 destination ISP list Transit $$$ ISP A Transit ISP Seek transport Interconnection $ x ISP B Transit $$$ Sample Top 10…

Sample Top 10 Destination List Def: Peering

Definition of Peering Routing Tables Def: Peering is the business relationship whereby ISPs reciprocally announce reachability to each others’ transit customers. Peering Peering WestNet USNet EastNet Routing Tables Transit Peering is *not* a transitive relationship Peering *does not* provide access to the entire Internet Usage: “I buy transit from Genuity and Peer with EastNet, WestNet.” Cost of Peering: Def. Transport+Port

Cost Components of Peering Example: Dallas

Dallas Market Prices for Peering 1) Transport into Transit $$$ Exchange ISP A DS3@$1500/mo 2) Rack Space at Exchange Point For Router ½ rack $1000/mo R Transit ISP X 3) Switch Port on R Public Peering Fabric (100M GigE) Transit $$$ Total Cost of Peering ISP B $2500/month Which is more cost effective? Peering or Transit? Price Quotes Averaged from AT&T, Time Warner, Level 3, Cogent, and Telseon, Equinix

Cost of Traffic Exchange Peering vs. solely Transit ~ $400/Mbps Peering $ 1 Mbps ISP A 1 Mbps Cost of Peering: $2500/month R Transit ISP X R Unit Cost of Traffic Exchange In Peering Relationship: $2500=$2500 per Mbps 1 Mbps ISP B

Cost of Traffic Exchange Peering vs. solely Transit ~ $400/Mbps Peering $ 2 Mbps ISP A 2 Mbps Cost of Peering: $2500/month R Transit ISP X R Unit Cost of Traffic Exchange In Peering Relationship: $2500=$1250 per Mbps 2 Mbps ISP B

Cost of Traffic Exchange Peering vs. solely Transit ~ $400/Mbps Peering $ 6 Mbps ISP A 6 Mbps Cost of Peering: $2500/month R Transit ISP X R Unit Cost of Traffic Exchange In Peering Relationship: $2500=$416 per Mbps 6 Mbps ISP B

Traffic Exchange Cost Per Mbps via Peering in Dallas (FIXED) Graph over DS3 BW

Traffic Exchange Cost Per Mbps with Peering Effective Peering BW Min Cost… Note Well: Transit and Peering are *not* perfect substitutes  As traffic volumes increase, the benefits of peering increase.

Effective Peering Bandwidth and Minimum Cost for Traffic Exchange DS-3/100M=$2500=$55/Mbps OC-3/100M=$3850=$38.50/Mbps OC-12/1000M=$10,000=$16/Mbps Min Transit Cost: ~$300/Mbps

Def: “Effective Peering Bandwidth” 1Gbps 1Gbps 1Gbps PORTS 100M Maximum bandwidth available for peering =Min(transportBW, peeringBW) Exchange C I R C U I T S OC-48 (2.4Gbps) DS3 (45Mbps) OC-12 (622Mbps) OC-3 (155Mbps)

Generalization: ISP Peering Breakeven Analysis Graphs $/Mbps Exchanged Breakeven Point (ISPs Indifferent between Peering and Transit traffic exchange) Peering Risk Prefer Peering Cost of Transit Cost of Traffic Exchange in Peering Relationship Number of Mbps exchanged

Top 5 Reasons not to Peer 1) Already get Traffic for “free” (through existing peering relationships) Transit $$$ Yahoo! Transit ISP EXODUS Peering $ AOL

Large Global Network Provider Top 5 Reasons not to Peer Huge investment in Int’s circuits, 100’s of routers and colo sites, Staff installs, peering negotiations, Millions of customers, etc. 2) Not True Peers Traffic inequity Scale inequity Not even investments in infrastructure Form: “I don’t want to haul your traffic around the globe” Large Global Network Provider Small Regional Player

Top 5 Reasons Not to Peer 3) Lack of Technical Competence Troubleshooting network problems takes longer when the other ISP NOC and engineers lack the technical expertise during an outage…

Top 5 Reasons Not to Peer 4) Transit Sales Preferred We rather sell you transit…“Let me introduce you to our sales guys”

Top 5 Reasons Not to Peer 5) BGP is Tough “BGP? No ExpertiseNo measurements No Justification to hire expertsBGP?” Simple Complex Conceptual Hurdle Conceptual Hurdle Primary Backup Transit Primary Backup Transit 6: personality

Top 5 Reasons Not To Peer 5+ Personality Clashes: They don’t understand each other and they didn’t like the interaction

Side Effect: Transit Aggregation C&W Sprint c Genuity AT&T C&W UUNet Sprint Genuity UUNet AT&T c $250/mo Aggregation Effect c c c Local Loops, OC-3@$2500, OC-12@$5000= $37,500/mo Dark Fiber Diverse Path: $12,000

Conclusions ISP Peering makes sense if you can offload X Mbps of traffic to peers… The “Peering Breakeven Analysis” identifies the breakeven point. “Effective Peering Bandwidth”=Min(TransportBW,PortBW) “A Business Case for ISP Peering” available