Peer Policy Policing with NETFLOW NANOG 25 June 9, 2002

Matthew Meyer Traffic Engineering NANOG 25 June 9, 2002

» On Net Cities » 27 On Net Countries » Nearly 100,000 route miles » 17 Metro Networks The Global Crossing Network

Peer Policy Policing With Netflow »Discovering and engaging the wayward packet flows that stumble onto your network »Giving default free networking a fighting chance »Get off my lawn »Bottom line: Just detecting a peer defaulting traffic us

Peer Policy Policing with Netflow Defining the problem »Telecom & Internet-space companies going into Ch11 »Punctuated mass customer moves due to Ch7 backbone liquidations »Peering less flexible »Some will resort to uncouth methods to mitigate the congestion and sidestep potential costs

Peer Policy Policing with Netflow Defining the problem »Fewer players, larger peerings »Peering inherits more flux and less flexibility to deal with it »Some more liberal peering channels may dry up or become heavily utilized

Peer Policy Policing with Netflow »Time to think like a bean counter »Is peering being abused? »Effect: Lower capex due to longer upgrade cycles »End goal: Knowing that we run a tight ship and being alerted when uninvited traffic enters the network Addressing the Problem

Peer Policy Policing with Netflow »Not rocket science »1:100 Netflow sampling »Sampling points: All traffic arriving on our border routers »Currently set to do peer-as type flow export Measurement

Peer Policy Policing with Netflow »One centrally located collector »Collector handling approximately 20 selected routers »Collector iBGP peers with border routers »Records route table changes every 5 minutes »Dual Pentium III, 1G memory, multiple Ultra-160 SCSI drives, directly connected to backbone Measurement

Peer Policy Policing with Netflow DEFAULTING PEER REPORT: Rec'd Peer Bytes percentage of total router interface destined for peer Bytes for interface br2.HUB1.gblx.net_so-2/1/ M <-Peer A br2.HUB1.gblx.net_so-2/1/ M <-Peer B br2.HUB1.gblx.net_so-3/1/ M <-Peer C br2.HUB1.gblx.net_so-2/1/ M <-Peer D br2.HUB1.gblx.net_at-2/2/ M <-Peer E br2.HUB1.gblx.net_so-1/2/ M <-Peer F br2.HUB1.gblx.net_so-3/1/ M <-Peer G br2.HUB1.gblx.net_so-0/0/ M <-uplink br2.HUB1.gblx.net_so-1/0/ M <-uplink Measurement

Peer Policy Policing with Netflow EXAMPLE OF FLOWDATA /Ixia/SeeFlow/bin/rseeas2as -S ' :00' br2.w00t1.gblx.net Facets: TimeInterval : 06/04/ :50: /04/ :31: UTC RouterIpv4Addr : InputIfIndex : 67 InputIfIpv4Addr : InputIfName : so-1/2/3.0 RouterName : br2.w00t1.gblx.net Src AS Dst AS Packets Pkts/sec Bytes Bits/sec K M K K M K K M K K M K K M K [~300 more lines clipped] Measurement

Peer Policy Policing with Netflow »Extracted with Ixia tools »24 hour cumulative byte count per interface + dest-as key pair »Created a peer-as list »Ignored incorrectly reported Netflow data according to routing policy Manipulating the Data

Peer Policy Policing with Netflow »Our design is hierarchical »Peers tend to be on dedicated peering routers »Our peering in consistent and rich »Collecting closer to the core would not catch this behavior universally Where to Look

Peer Policy Policing with Netflow »BGP import policy gets in the way of trusting source AS »Trace levels of false peer to peer traffic associated with most peering interfaces »In initial beta, no peers have been found blatantly defaulting to us Analysis

Peer Policy Policing with Netflow »For the moment peer defaulting does not seem to be a problem »We can move forward and easily complete a detection system »Feeling more confident about possible tighter peering ahead So Far So Good

Peer Policy Policing with Netflow »Change flow export style from peer-as to origin- as »Putting the discovery ‘on cron’ »Long term: »Distribute collection »Build some visualization »Integrate with RRDtool What’s Next

Peer Policy Policing with Netflow »Good exercise in ‘Netflow 101’ »Sampling capability excellent »Data quality excellent »Restored confidence in Netflow reliability Retrospect

GLOBAL REACH. SEAMLESS NETWORK.

THANK YOU