Yaping Zhu with: Jennifer Rexford (Princeton University) Aman Shaikh and Subhabrata Sen (ATT Research) Route Oracle: Where Have All the Packets Gone?

Yaping Zhu, Princeton University 2 Route Oracle: Network Management Tool Useful for many different applications Src: A Dest: B IP Packet

Case 1: Historical Traceroute Yaping Zhu, Princeton University 3 Network troubleshooting –Diagnose reachability or performance problems Traceroute –Lists of hops along the path to the destination –Popular active measurement tool –Limitation: sending IP packets in real time Wouldn’t it be nice to run traceroute in the past?

Case 1: Historical Traceroute Historical Traceroute by Route Oracle Requirement: answer queries at any time Yaping Zhu, Princeton University 4 Src: A Dest: B

Case 2: Analysis of Routing Disruptions React quickly to large network disruptions –E.g. Mediterranean Cable Break What happened? Yaping Zhu, Princeton University 5

Case 2: Analysis of Routing Disruptions Yaping Zhu, Princeton University 6 Characterization by Route Oracle –Percent of IPs unreachable? –How was the traffic rerouted? Help network operators –Respond to customer complaints –Redirect traffic Requirement: answer queries at scale

Page 7 AT&T Sprint Internet AT&T CDN Server Atlanta users Router in Atlanta Egress in Atlanta Egress in Washington DC Case 3: Service Quality Management CDN Throughput Drop caused by Egress Change Case Study from AT&T Intelligent Content Distribution Service

Application-layer monitoring is not enough –Correlate with network-layer changes: routing changes, network congestion –Joint analysis with Route Oracle Requirement: answer queries at scale and in real time Yaping Zhu, Princeton University 8 Case 3: Service Quality Management

Yaping Zhu, Princeton University 9 Route Oracle: System Interfaces Route Oracle Query Input: vantage point router, query time period, list of IPs or prefixes Output: route changes for each query (prefix match, egress router, AS path) Input Data Feeds: periodic dumps of routing table continuous feed of updates

Challenges Changes in the longest prefix match (LPM) Scalability Real time Yaping Zhu, Princeton University 10

Yaping Zhu, Princeton University 11 BGP Background Src: A Dest: B IP Packet Ingress Router Egress Router AS Path AS (Autonomous System)

BGP Background BGP updates are based on prefixes An IP address can be covered by multiple prefixes Longest prefix-match (LPM) determines forwarding LPM may change over time Yaping Zhu, Princeton University 12 Routing Table: / / 24 IP Packet Longest Prefix Match

Yaping Zhu, Princeton University 13 Track LPM Changes: Example Track LPM changes for IPs from to Suppose in the routing table, we have prefix: – /8 – / /8 / /8 /16 LPM IPs

Yaping Zhu, Princeton University 14 Track LPM Changes: Example BGP route announcement: /24 /24 More specific unchanged /8 / /8 /16 LPM IPs unchanged

Yaping Zhu, Princeton University 15 Track LPM Changes: Example BGP route withdrawal: /16 /24 unchanged Less specific /8 / /8 LPM IPs unchanged

Yaping Zhu, Princeton University 16 Track LPM Changes: at Scale Definitions –Prefix set: all matching prefixes for a given IP address –Address range: contiguous addresses that have the same prefix set / / /8 /16 LPM IPs /16

Yaping Zhu, Princeton University 17 Track LPM Changes: in Real Time Definitions –Prefix set –Address range Real-time algorithm –For each update: Update address ranges Update prefix set for affected address ranges

Yaping Zhu, Princeton University 18 Route Oracle: System Components Precomputation Query Input Input Data Feeds Query Processing Snapshot of routes for address ranges Output Incremental route updates for address ranges

Query Processing: Optimizations Amortize the cost of reading address range records across al queried IP addresses Parallelize the processing of the address range files on multi-core machines Yaping Zhu, Princeton University 19

Performance Evaluation Goals –Pre-processing module: handle updates stream in real time? –Query module: query processing time Experiment on SMP server –two quad-core Xeon X5460 Processors –Each CPU: 3.16 GHz and 6 MB cache –16 GB of RAM Yaping Zhu, Princeton University 20

Evaluation: Pre-processing Time Experiment design –BGP updates received over fixed time-intervals –Compute the pre-processing time for each batch of updates Results –99% of the updates received in 5 mins interval could be processed within 2 seconds –The maximum time to process the BGP updates received in 20 mins intervals is about 5 seconds Yaping Zhu, Princeton University 21

Evaluation: Query Processing Time 22

Evaluation: Query Parallelization 23

Prototype Deployment Deployed since Sep 2009 (AT&T) –Process BGP updates from 25 vantage points at PoPs Systems interfaces –Both GUI and command-line interfaces Services –Network troubleshooting –ICDS (Intelligent Content Distribution Services) Performance optimizations based on user feedback Yaping Zhu, Princeton University 24

Conclusion Route Oracle provides simple functionality –Crucial for many network management tasks Definition of address ranges –Track route changes at scale and in real time Design, implementation, evaluation, deployment –Systems optimizations to improve performance Yaping Zhu, Princeton University 25

Thanks! Questions & Comments? Yaping Zhu, Princeton University 26

Evaluation: Pre-processing Time 27