1. Peer Policy Policing with NETFLOW NANOG 25 June 9, 2002

2. Matthew Meyer Traffic Engineering NANOG 25 June 9, 2002

3. » 200 + On Net Cities » 27 On Net Countries » Nearly 100,000 route miles » 17 Metro Networks The Global Crossing Network

4. Peer Policy Policing With Netflow »Discovering and engaging the wayward packet flows that stumble onto your network »Giving default free networking a fighting chance »Get off my lawn »Bottom line: Just detecting a peer defaulting traffic us

5. Peer Policy Policing with Netflow Defining the problem »Telecom & Internet-space companies going into Ch11 »Punctuated mass customer moves due to Ch7 backbone liquidations »Peering less flexible »Some will resort to uncouth methods to mitigate the congestion and sidestep potential costs

6. Peer Policy Policing with Netflow Defining the problem »Fewer players, larger peerings »Peering inherits more flux and less flexibility to deal with it »Some more liberal peering channels may dry up or become heavily utilized

7. Peer Policy Policing with Netflow »Time to think like a bean counter »Is peering being abused? »Effect: Lower capex due to longer upgrade cycles »End goal: Knowing that we run a tight ship and being alerted when uninvited traffic enters the network Addressing the Problem

8. Peer Policy Policing with Netflow »Not rocket science »1:100 Netflow sampling »Sampling points: All traffic arriving on our border routers »Currently set to do peer-as type flow export Measurement

9. Peer Policy Policing with Netflow »One centrally located collector »Collector handling approximately 20 selected routers »Collector iBGP peers with border routers »Records route table changes every 5 minutes »Dual Pentium III, 1G memory, multiple Ultra-160 SCSI drives, directly connected to backbone Measurement

10. Peer Policy Policing with Netflow DEFAULTING PEER REPORT: Rec'd Peer Bytes percentage of total router interface destined for peer Bytes for interface br2.HUB1.gblx.net_so-2/1/3.0 0.011M 0.006 <-Peer A br2.HUB1.gblx.net_so-2/1/0.0 0.026M 0.008 <-Peer B br2.HUB1.gblx.net_so-3/1/0.0 0.087M 0.008 <-Peer C br2.HUB1.gblx.net_so-2/1/2.0 0.145M 0.011 <-Peer D br2.HUB1.gblx.net_at-2/2/0.0 0.167M 0.024 <-Peer E br2.HUB1.gblx.net_so-1/2/3.0 0.339M 0.017 <-Peer F br2.HUB1.gblx.net_so-3/1/2.0 2.464M 0.246 <-Peer G br2.HUB1.gblx.net_so-0/0/0.0 3319.615M 56.722 <-uplink br2.HUB1.gblx.net_so-1/0/0.0 3381.523M 61.515 <-uplink Measurement

11. Peer Policy Policing with Netflow EXAMPLE OF FLOWDATA /Ixia/SeeFlow/bin/rseeas2as -S '20020603 00:00' br2.w00t1.gblx.net Facets: TimeInterval : 06/04/2002 16:50:49.217018 - 06/04/2002 19:31:52.879363 UTC RouterIpv4Addr : 10.10.10.10 InputIfIndex : 67 InputIfIpv4Addr : 10.0.0.1 InputIfName : so-1/2/3.0 RouterName : br2.w00t1.gblx.net Src AS Dst AS Packets Pkts/sec Bytes Bits/sec ------- ------- ------------- ------------- ------------- ------------- 1111 2222 654.061K 67.683 321.386M 266.058K 1111 3333 177.794K 18.398 130.125M 107.723K 99 44444 139.861K 14.473 91.889M 76.070K 1111 3549 257.006K 26.595 78.603M 65.071K 1111 5555 72.634K 7.516 65.807M 54.478K [~300 more lines clipped] Measurement

12. Peer Policy Policing with Netflow »Extracted with Ixia tools »24 hour cumulative byte count per interface + dest-as key pair »Created a peer-as list »Ignored incorrectly reported Netflow data according to routing policy Manipulating the Data

14. Peer Policy Policing with Netflow »Our design is hierarchical »Peers tend to be on dedicated peering routers »Our peering in consistent and rich »Collecting closer to the core would not catch this behavior universally Where to Look

15. Peer Policy Policing with Netflow »BGP import policy gets in the way of trusting source AS »Trace levels of false peer to peer traffic associated with most peering interfaces »In initial beta, no peers have been found blatantly defaulting to us Analysis

16. Peer Policy Policing with Netflow »For the moment peer defaulting does not seem to be a problem »We can move forward and easily complete a detection system »Feeling more confident about possible tighter peering ahead So Far So Good

17. Peer Policy Policing with Netflow »Change flow export style from peer-as to origin- as »Putting the discovery ‘on cron’ »Long term: »Distribute collection »Build some visualization »Integrate with RRDtool What’s Next

18. Peer Policy Policing with Netflow »Good exercise in ‘Netflow 101’ »Sampling capability excellent »Data quality excellent »Restored confidence in Netflow reliability Retrospect

19. GLOBAL REACH. SEAMLESS NETWORK.

20. THANK YOU