The Economics of Transit and Peering Interconnections in the Internet Amogh Dhamdhere (CAIDA/UCSD) With Constantine Dovrolis (Georgia Tech) Pierre Francois (Univ. catholique de Louvain) Kc Claffy (CAIDA) 11/27/20151

Outline Motivation ITER: A computational model of interdomain interconnection Modeling the transition from the “old” to the “new” Internet Modeling complex interdomain relationships: value-based peering Ongoing Work 11/27/2015 The Economics of Transit and Peering Interconnections 2 [CoNEXT 2010, to appear] [ITC 2010, NANOG 49] [NSF NETSE grant, ]

The Internet Ecosystem >30,000 autonomous networks independently operated and managed The “Internet Ecosystem” –Different types of networks –Interact with each other and with “environment” Network interactions –Localized, in the form of bilateral contracts –Competitive (customer-provider), symbiotic (peering) Distributed optimizations by each network 11/27/20153 The Economics of Transit and Peering Interconnections

High Level Questions How does the Internet ecosystem evolve? What is the Internet heading towards? –Topology –Economics –Performance Which interconnection strategies of networks optimize their profits, costs and performance? How do these strategies affect the global Internet? 11/27/20154 The Economics of Transit and Peering Interconnections

Economics of the Internet Ecosystem 11/27/20155 The Economics of Transit and Peering Interconnections Source: Cisco Source: Arbor Networks Source: William Norton Source: IAB

Economics of the Internet Ecosystem How do we make sense of all this? 11/27/20156 The Economics of Transit and Peering Interconnections

Economically-principled models Objective: understand the structure and dynamics of the Internet ecosystem from an economic perspective Capture interactions between network business relations, internetwork topology, routing policies, and resulting interdomain traffic flow Create a scientific basis for modeling Internet interconnection and dynamics based on empirical data 11/27/20157 The Economics of Transit and Peering Interconnections

Previous Work “Descriptive” –Match graph properties e.g. degree distribution Homogeneity –Nodes and links all the same Game theoretic, analytical –Restrictive assumptions Little relation to real- world data 11/27/20158 “Bottom-up” –Model the actions of individual networks Heterogeneity –Networks with different incentives, link types Computational –As much realism as possible Parameterize/validate using real data The Economics of Transit and Peering Interconnections

Outline Motivation ITER: A computational model of interdomain interconnection Modeling the transition from the “old” to the “new” Internet Modeling complex interdomain relationships: value-based peering Ongoing Work 11/27/2015 The Economics of Transit and Peering Interconnections 9

The ITER Model Agent-based computational model to answer “what-if” questions about Internet evolution Inputs: According to the best available data… Network types based on business function Peer/provider selection methods Geographical constraints Pricing/cost parameters Interdomain traffic matrix Output: Equilibrium internetwork topology, traffic flow, per-network fitness 1011/27/2015 The Economics of Transit and Peering Interconnections

The ITER approach 11 Interdomain TM Traffic flow Interdomain topology Per-AS fitness Cost/price parameters Routing Provider selection Peer selection Compute equilibrium: no network has the incentive to change its providers/peers Measure topological and economic properties of equilibrium e.g., path lengths, which providers are profitable, who peers with whom 11/27/2015 The Economics of Transit and Peering Interconnections

Why Study Equilibria? The Internet is never at equilibrium, right? –Networks come and go, traffic patterns change, pricing/cost structures change, etc…. Studying equilibria tells us what’s the best that networks could do under certain traffic/economic conditions, and what that means for the Internet as a whole If those conditions change, we need to re- compute equilibria 11/27/ The Economics of Transit and Peering Interconnections

ITER: Network Types Enterprise Customers (EC) –Stub networks at the edge, e.g. Georgia Tech Transit Providers –Provide Internet transit –Regional in scope (STP), e.g. Comcast –“Tier-1” or global (LTP), e.g., AT&T Content Providers (CP) –Major sources of content, e.g. Google 11/27/ The Economics of Transit and Peering Interconnections

ITER: Provider and Peer Selection Provider selection –Choose providers based on customer cone size –Good measure of the “size” of a provider –Used by commercial products, e.g., Renesys Peer selection strategies –Peer based on total traffic handled –Approximates the “equality” of two ISPs 11/27/ The Economics of Transit and Peering Interconnections

ITER: Economics, Routing and Traffic Matrix Realistic transit, peering and operational costs –Transit prices based on data from NANOG BGP-like routing policies –No-valley, prefer customer, prefer peer routing policy Traffic matrix –Heavy-tailed content popularity and consumption by sinks –client-server traffic: from CPs to ECs –peer-to-peer traffic: between ECs 11/27/ The Economics of Transit and Peering Interconnections

Computing Equilibrium Situation where no network has the incentive to change its connectivity Too complex to find analytically: Solve computationally Computation –Proceeds iteratively, networks “play” in sequence –Compute routing, traffic flow, AS fitness –Repeat until no player has incentive to move 11/27/ The Economics of Transit and Peering Interconnections

Properties of the equilibrium Is equilibrium always reached? –Yes, in most cases Is the equilibrium unique? –No, can depend on playing sequence Multiple runs with different playing sequence –Per-network properties vary widely across runs –Macroscopic properties show low variability 11/27/ The Economics of Transit and Peering Interconnections

Outline Motivation ITER: A computational model of interdomain interconnection Modeling the transition from the “old” to the “new” Internet Modeling complex interdomain relationships: value-based peering Ongoing Work 11/27/2015 The Economics of Transit and Peering Interconnections 18 [CoNEXT 2010, to appear]

Recent Trends: Arbor Networks Study The Old Internet (late 90s – 2007) Content providers generated small fraction of total traffic Content providers were mostly local Peering was restrictive The New Internet (2007 onwards) Content providers generate large fraction of total traffic Content providers are present everywhere Peering is more open “Internet Interdomain Traffic”, Labovitz et al., Sigcomm /27/ The Economics of Transit and Peering Interconnections

Plugging into ITER Simulate two instances of ITER: “Old” and “New” Internet Change three parameters –Fraction of traffic sourced by CPs –Geographical spread of CPs –Peering openness Compute equilibria for these two instances –Compare topological, economic properties 11/27/2015 The Economics of Transit and Peering Interconnections 20

ITER Sims: End-to-end Paths End-to-end paths weighted by traffic are shorter in the “new Internet” 11/27/2015 The Economics of Transit and Peering Interconnections 21

ITER Sims: Traffic Transiting Transit Providers Traffic bypasses transit providers More traffic flows directly on peering links Implication: Transit providers lose money! Content providers get richer 11/27/2015 The Economics of Transit and Peering Interconnections 22

ITER Sims: Peering in the New Internet Transit providers need to peer strategically in the “new” Internet Peering with large Content Providers benefits transit providers in the new Internet Try to attract traffic by peering with CPs 11/27/2015 The Economics of Transit and Peering Interconnections 23

Outline Motivation ITER: A computational model of interdomain interconnection Modeling the transition from the “old” to the “new” Internet Modeling complex interdomain relationships: value-based peering Ongoing Work 11/27/2015 The Economics of Transit and Peering Interconnections 24 [ITC 2010, NANOG 49]

Complex Interdomain Relationships So far, we considered customer-provider and settlement-free peering links Customer-provider: customer pays provider Settlement-free peering: No payments A whole spectrum of relationships somewhere in between – paid peering “To peer or not to peer?”  “To peer or not to peer, at what price?” 11/27/ The Economics of Transit and Peering Interconnections

To Peer or Not to Peer, at What Price? 26 A A B B Should I peer with B? make Should it be settlement-free or paid-peering? What price would B be willing to offer (or accept) ? ??? 11/27/2015 The Economics of Transit and Peering Interconnections

Value Based Peering Price based on the “value” of the link For a network, define the notion of “fitness” f = revenue – interconnect costs – backhaul cost Value of the link is the difference in fitness with and without the link V = f with - f without Revenue and costs could change on peering/depeering 2711/27/2015 The Economics of Transit and Peering Interconnections

What Affects Peering Value? Interconnect cost changes: Avoid a transit provider Backhaul cost changes: Peering link changes how traffic is routed in a network Revenue changes: Attract/lose traffic due to new peering link 28 A A B B T T $$$ 11/27/2015 The Economics of Transit and Peering Interconnections

The Fair Peering Price An oracle knows V A and V B Oracle must decide the price for peering Fair price is (V A -V B )/2 29 The fair price equalizes the benefit that A and B see from the link A A B B VAVA VBVB (V A -V B )/2 11/27/2015 The Economics of Transit and Peering Interconnections

Why Peer at the Fair Price? Peering with the fair price is optimal Both networks see better fitness by peering at the fair price Peering with the fair price is stable No network has the incentive to unilaterally depeer the other network Unique Nash Equilibrium Optimal and stable as long as V A + V B > 0 Either V A or V B can be negative, as long as total is positive For cost-benefit peering, both V A and V B must be positive 3011/27/2015 The Economics of Transit and Peering Interconnections

Some Hard Questions.. Value-based peering is fair, optimal and stable. But is there an incentive to be fair? How do networks estimate peering value? What if networks lie about peering value? What happens if everyone uses value-based peering? 3111/27/2015 The Economics of Transit and Peering Interconnections

ITER Results: Value-based Peering Higher density of peering links with value- based peering  Shorter end-to-end paths Links that are not allowed with traffic-ratio or cost- benefit peering are possible with value-based peering Payment direction: Content providers end up paying large transit providers, get paid by smaller transit providers This could be happening already! 3211/27/2015 The Economics of Transit and Peering Interconnections

Outline Motivation ITER: A computational model of interdomain interconnection Modeling the transition from the “old” to the “new” Internet Modeling complex interdomain relationships: value-based peering Ongoing Work 11/27/2015 The Economics of Transit and Peering Interconnections 33 [NSF NETSE grant, Aug ]

Ongoing Work Strategy selection by ISPs –Model how networks dynamically change their provider/peer selection strategies –What is the best possible strategy for different network types? Managing the cost of network traffic –Model traffic related-costs for ISPs –Support peering decisions, capacity upgrades, differential pricing models 11/27/2015 The Economics of Transit and Peering Interconnections 34 [A. Lodhi, A. Dhamdhere, C. Dovrolis] [M. Motiwala, A. Dhamdhere, N. Feamster, A. Lakhina]

Avoiding “garbage-in, garbage-out” Models are only as good as the data you provide as input How do we get the best possible data to parameterize ITER-like models? What data do we need? –Interdomain traffic patterns –Peering policies –Geographical presence of networks –Cost/pricing structures 11/27/ The Economics of Transit and Peering Interconnections

Measuring Interdomain Traffic We don’t really know how much traffic each pair of networks exchanges! Measure qualitative properties of the interdomain TM from different vantage points 11/27/ The Economics of Transit and Peering Interconnections GT A A B B Internet Netflow

Measuring Interdomain Traffic We don’t really know how much traffic each pair of networks exchanges! Measure qualitative properties of the interdomain TM from different vantage points 11/27/ The Economics of Transit and Peering Interconnections C C A A B B ISP Netflow

Measuring Interdomain Traffic We need data from as many vantage points as possible! Currently working with GEANT, SWITCH, Georgia Tech Let us know if you can help! 11/27/ The Economics of Transit and Peering Interconnections C C A A B B ISP Netflow

Summary We need realistic, economically-principled models to make sense of the Internet ecosystem We developed ITER, a computational model of interdomain interconnection We are working on better parameterization and validation using real-world measurements We need your help and feedback! 11/27/2015 The Economics of Transit and Peering Interconnections 39

Thanks! 11/27/ The Economics of Transit and Peering Interconnections

References “The Internet is Flat: Modeling the Transition from a Transit Hierarchy to a Peering Mesh” –A. Dhamdhere, C. Dovrolis [CoNEXT 2010] “A Value-based Framework for Internet Peering Agreements” –A. Dhamdhere, C. Dovrolis, P. Francois [ITC 2010] “The Economics of Transit and Peering Interconnections in the Internet” –C. Dovrolis, K. Claffy, A. Dhamdhere [NSF NETSE ] 11/27/ The Economics of Transit and Peering Interconnections

Backup slides 11/27/2015 The Economics of Transit and Peering Interconnections 42

Network actions Networks perform their actions sequentially Can observe the actions of previous networks –And the effects of those actions on traffic flow and economics Network actions in each move –Pick set of preferred providers –Evaluate each existing peering link –Try to create new peering links 11/27/ The Economics of Transit and Peering Interconnections

Three Factors Fraction of traffic sourced by CPs Geographical presence of CPs Peering openness All three factors need to change to see the differences between the “old” and “new” Internet 11/27/2015 The Economics of Transit and Peering Interconnections 44

Peering Requirements Laundry list of conditions that networks specify as requirements for (settlement-free) peering Traffic ratios, minimum traffic, backbone capacity, geographical spread … Heuristics to find networks for which it makes sense to exchange traffic for “free” But when it comes to paid peering.. What is the right price? Who should pay whom? Are these heuristics always applicable? Mutually beneficial peering links may not be formed 4511/27/2015 The Economics of Transit and Peering Interconnections

Peering Uncertainty – Current Peers 46 A A B B Why is B still a settlement-free peer? Does B benefit more than me? make Should I demand payment? Should I depeer? 11/27/2015 The Economics of Transit and Peering Interconnections

Negative Peering Value 47 A A B B f A : $50k$60kf B : $100k$95k V B =-$5k V A =$10k $7.5k $52.5k $102.5k 11/27/2015 The Economics of Transit and Peering Interconnections

Modeling the Internet Ecosystem Networks select providers and peers to optimize an objective function E.g., Profit, performance… What are the effects of provider and peer selection strategies on the involved networks? What are the global, long-term effects of these strategies on the whole Internet? Topology, traffic flow, economics, performance (path lengths) E.g., Can we predict what would happen if (fair) paid- peering becomes the common case? 4811/27/2015 The Economics of Transit and Peering Interconnections

Measuring Peering Value How do A and B measure V A and V B ? With Peering trials: Collect: netflow, routing data Know: topology, costs, transit providers With peering trials, A and B can measure their own value for the peering link (V A and V B ) reasonably well Hard for A to accurately measure V B (and vice versa) 4911/27/2015 The Economics of Transit and Peering Interconnections

Hiding peering value Assume true V A + V B > 0 and V B > V A A should get paid (V B - V A )/2 If A estimates V B correctly, and claims its peering value is V L, where V L << V A B is willing to pay more: (V B - V L )/2 If A doesn’t estimate V B correctly, and V L + V B < 0, the peering link is not feasible! A loses out on any payment  Does the risk of losing out on payment create an incentive to disclose the true peering value? 5011/27/2015 The Economics of Transit and Peering Interconnections

Peering Policies What peering policies do networks use? How does this depend on network type? Do they peer at IXPs? How many IXPs are they present at? PeeringDB: Public database where ISPs volunteer information about business type, traffic volumes, peering policies Collecting peeringDB snapshots periodically Goal is to study how peering policies evolve 11/27/ The Economics of Transit and Peering Interconnections

peeringDB 11/27/2015 The Economics of Transit and Peering Interconnections 52