1. 1 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Experience in Black-box OSPF Measurement Aman Shaikh, UCSC Albert Greenberg, AT&T Labs-Research Sigcomm IMW – November 2001

2. 2 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Why Measure OSPF? OSPF behavior in large ISPs not well understood, yet –any meaningful performance assurance depends on routing stability –an internal network change (OSPF event) can have major impact on flows and customers, during which intra-domain routing reconverges inter-domain routing reconverges (BGP uses OSPF metrics) Internal OSPF processing delays matter! –message processing, routing calculation, table update –add up to impact convergence, instabilities OSPF measurements also needed for –guidance in tuning configurable parameters –head to head vendor comparisons

3. 3 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 How to Measure OSPF? Problem: Instrumenting routing code for measuring delays is challenging –commercial implementations are proprietary –may involve grappling with numerous code versions, hardware platforms, and developers Solution: black-box measurements –measure the timing delays using external observations Contribution: black-box measurements for internal OSPF delays –applied to Cisco and GateD OSPF implementations Key prior work: –IS-IS measurements by Packet Design [draft- alaettinoglu-ISIS-convergence-00]

4. 4 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Black-box Techniques are Effective Works across wide range of timing delays –100 sec for packet processing –10s of msec for routing calculation Works even for the purely CPU bound tasks –packet processing subtasks, Dijkstras shortest path calculation Captures scaling –O(n 2 ) time for shortest path calculation, for full n n mesh topologies

5. 5 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 OSPF Background Link-state routing protocol –all routers in the domain come to a consistent view of the topology by exchange of Link State Advertisements (LSAs) set of LSAs (self-originated + received) at a router = topology SPF Calculation –each router calculates a single source shortest path tree Forwarding Information Base (FIB) –each router uses the tree to build its FIB, which governs packet forwarding

6. 6 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Link-state Advertisement (LSA) LSA propagation: each router –describes local connectivity in an LSA –floods LSA to other routers in the domain –acknowledges LSA in an LS Ack packet Duplicate LSAs: each router –can receive multiple copies of a given LSA first copy received is termed new copies received later are termed duplicate Duplicate LSAs MUST be acknowledged immediately (RFC2328) –allows us to build a timestamp

7. 7 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Router Model Route Processor (CPU) FIB Interface card Forwarding Switching Fabric Data packet Topology View SPF Calculation OSPF Process LSA LS Ack LSA Forwarding LSA Processing LSA Flooding SPF Calculation FIB Update

8. 8 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Methodology TopTracker Target router Emulated topology Load emulated topology on target router Initiate task of interest Measure the time for task Testbed LSA

9. 9 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Measuring Task Time top bracket event bottom bracket event task start time task finish time time 1.Use a black-box method to bracket task start and finish times 2.Subtract out intervals that precede and exceed these times X B C X = A - (B+C) A

10. 10 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Methodology for SPF Calculation Ack for duplicate LSA arrives Initiator LSA arrives SPF calculation ends SPF calculation starts time Target Router TopTracker Send initiator LSA Send duplicate LSA Load desired topology Send ack for duplicate LSA X = A – (B + C + D + E) Estimate the overhead = B + C + D + E A X C D B E

11. 11 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Estimating the Overhead Remove SPF calculation from bracket –spf_delay = 60 seconds Ack for duplicate LSA arrives Initiator LSA arrives Initiator LSA processing done Duplicate LSA arrives time Target Router TopTracker Send initiator LSA Send duplicate LSA Duplicate LSA processing done; send ack SPF calculation starts overhead = B + C + D + E B E C D Overhead

12. 12 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Results Results for Cisco GSR, 7513 and GateD –for GateD, comparison of black-box results with those obtained using instrumentation (white-box) –route processors Cisco: 200 MHz R5000 processor GateD: 500 MHz AMD-K6 processor Topology used is a full n n mesh with random OSPF edge weights –vary n in the range 10, 20, …, 100

13. 13 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Results for Cisco Routers Similar results for two models SPF calculation time is O(n 2 )

14. 14 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Results for GateD Black-box over-estimates white-box measurement Black-box captures the characteristics very well

15. 15 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 OSPF Task Delays (Cisco) LSA Processing –100-800 microseconds LSA flooding –30-40 milliseconds –pacing timer is the determining factor SPF calculation –1-40 milliseconds –O(n 2 ) behavior for full n x n mesh FIB update time –100-300 milliseconds –no dependence on the size of the topology

16. 16 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Toolkit Use of topology emulator –loads topologies –generates specific patterns of LSAs Use of protocol dynamics mandated by standards –duplicate LSA mechanism: OSPF is required to ack a duplicate LSA immediately –useful for estimating end-point of tasks like SPF calculation Use of vendor-specific parameters: –spf_delay –spf_holdtime –Pacing timer

17. 17 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Conclusions Black-box methods for estimating OSPF processing delays: –LSA processing and flooding –SPF calculation and FIB Update Applied techniques to Cisco GSR and 7513 routers as well as GateD Black-box methods worked Future work –develop techniques for other protocols, in particular BGP

18. 18 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Backup

19. 19 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 OSPF Overview : Example A B D C E F I G H J 1 11 1 121 3 21 1 11 OSPF Domain (single area) A B D C E F I G H J 1 1 11 21 1 1 SPT at G 1

20. 20 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 LSA Processing Receive an LSA New/duplicate? new Update topology view Schedule SPF calc. if reqd. duplicate Acknowledge LSA immed. Send LS Ack packet back Flood the LSA out LSA Processing over SPF Calculation paced by hold-down timer (spf_delay)

21. 21 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 SPF Calculation LSA Processing over SPF calculation ends FIB is updated SPF calculation starts

22. 22 A. Sshaikh, A. Greenberg; Nov 01 UCSC Sigcomm IMW - 2001 Internal OSPF Tasks to Measure Processing Link State Advertisements (LSAs) Flooding LSAs Performing SPF calculation –described in this talk Updating the Forwarding Information Base (FIB)