Peering Exchange Architectures Jeff Bartig University of Wisconsin WiscNet Engineering

Overview OmniPOP - new CIC R&E exchange in Chicago Way too many hours of conference calls about hardware and architecture of OmniPOP I returned Paul Schopis’ phone call.

Layer 2 Shared Exchange Layer 2 switch Each peer connects to switch Single broadcast domain Generally a single subnet allocated - each peer gets an address out of the block Used at many commercial exchanges

Layer 2 shared - Pros/Cons Cons Single broadcast domain Peer abuse possible Broadcast traffic Multicast difficulties Per-peer statistics more difficult to collect (per- MAC address stats necessary) Single MTU size Pros Easy/Simple No exchange provider involvement needed to establish a new peer Many peers on a single port - lower cost

Layer 2 VLAN Exchange Each peer has a trunk interface on layer 2 exchange switch Each peer may have a VLAN to every other peer

Layer 2 VLAN Exchange Each peer has a trunk interface on layer 2 exchange switch Each peer may have a VLAN to every other peer (n * (n-1)/2 VLANs for a full mesh

Layer 2 VLAN - Pros/Cons Pros Separation MTU per VLAN Multicast easier Many peers on a single port - lower cost Logical interface stats Cons Many VLANs VLAN conflicts possible Exchange provider involvement if VLANs are not preallocated

Layer 1 No central exchange hardware Each peer establishes cross connects to parties they want to peer with

Layer 1 Pros/Cons Requires an interface for each peer No exchange provider hardware needed - may be a cost savings Capacity/protocol flexibility

Layer 3 Exchange Each peer gets a port on the exchange router Each peer establishes a BGP session with exchange router

Layer 3 - Pros/Cons Cons Less control Routing policy determined by exchange provider Extra AS hop in path, possible impact on routing decisions Pros Simpler Single BGP session No need for a peering coordinator - outsourcing to exchange provider

Layer 3 Exchange - AS-path Length Concern Example Peer to WN AS Paths Layer 2 Exchange WN direct peer shorter Layer 3 Exchange Equal paths. Which one will be chosen by peer?

AS-Path Length Experiment WiscNet peers with 31 networks at Equinix- Chicago What difference would it make if we instead did this peering via a layer 3 exchange? Depends upon the routing policies of the peers (local pref wins over AS-path length) Prepended extra AS hops into the advertisements to see what would happen.

AS-Path Length Experiment Results 13:50 - prepended extra hop 450 to 350Mb/s drop 22% loss 14:26 - prepended 2nd extra hop 410 to 350Mb/s drop additional 15% loss 14:45 - removed all prepending 310 to 430Mb/s increase 38% increase

End