1 Cross-layer Visibility as a Service Ramana Rao Kompella Albert Greenberg, Jennifer Rexford Alex C. Snoeren, Jennifer Yates

2 Layering in the current Internet IP Optics Fiber MPLS OVERLAYS Ethernet Fiber-spans

3 Layering is a mixed blessing Layering allows us to contain complexity Each layer evolves independently without affecting any other layer Allows us to focus on one layer at a time There are associated challenges too… Routine operational tasks need associations across layers Example: mapping an IP link to optical circuit, overlay link to an IP path Lack of accurate cross-layer associations can affect the reliability of the network

4 Intended planned maintenance Los Angeles San Francisco Seattle Denver Dallas St Louis Chicago Boston New York Orlando Planned maintenance on optics

5 Intended planned maintenance Optical component is on circuit id A Lookup database to map circuit id A to IP link Due to mis-association, incorrectly maps it to LA to Dallas Increase OSPF weight on LA to Dallas link Disconnect component Causes failure Los Angeles San Francisco Denver Traffic from LA to Dallas is rerouted via Denver Dallas X LA to San Francisco link is congested High OSPF weight Planned maintenance can induce faults if accurate associations are not maintained

6 Customer Fault Tolerance Customer in NJ New York Philadelphia Internet Shared optical element INTRA-CARRIER DIVERSITY Customer in NJ New York Internet Sprint Level 3 INTER-CARRIER DIVERSITY Going through same conduit or Holland tunnel ? Customer diversity information requires accurate cross-layer associations, sometimes across domains

7 Fault diagnosis Los Angeles San Francisco Seattle Denver Dallas St Louis Chicago Boston New York Orlando What happened ?!! X Because of a bug, IP forwarding path changed, but MPLS did not ! MPLS circuit between LA and New York Diagnosing faults requires accurate cross-layer associations

8 Why is it hard ? Cant the operators maintain associations in a centralized fashion ? Maintain database as links are provisioned Update as and when interfaces are re-homed Hard due to flux in topology Churn because of dynamic topology changes Human errors during re-homing interfaces Operational realities – separation of concerns

9 How it is done today ? A combination of non-standard databases Human-generated inventory data Measurement data obtained from probes Configuration state from network elements Policies implemented in network elements Higher complexity and overhead No compatibility across ASes Difficult to evolve a network Difficult to integrate two networks after acquisition Difficult to incorporate third-party tools

10 Why not concentrate on restoration? Advantages of lower-layer restoration Hides lower-layer failures from impacting upper layers Obviates to some extent need for cross-layer visibility Cross-layer visibility still important Lower-layer restoration more expensive than IP restoration Subtle performance changes (e.g., RTT) need diagnosis

11 Why not fatten the interfaces ? Fattening interfaces to make layers aware of the entire topologies above and below Layers discover and propagate mappings automatically Management system can query the network to obtain mappings Fattening results in high complexity Interoperability is a big challenge – long design and test cycles Wider interfaces impact security

12 Cross-layer Policy Server Architecture for cross-layer visibility Ping Trace-route Backbone planning Customer diversity Backbone maintenance Fault diagnosis DB MANAGEMENT APPLICATIONS BOW-TIE IP Optics Fiber MPLS OVERLAYS IP HOUR-GLASS Ethernet Fiberspans

13 IP Optics FIBER, FIBERSPAN MPLS OVERLAYS Standardize what goes in ! IP Optics FIBER, FIBERSPAN MPLS OVERLAYS Standardize what goes in (e.g. IP topologies) AS1 Facilitates interaction between ISP policy servers AS2

14 Advantages of the bow-tie Topology, routing information and other associations can be queried for maintenance, diversity, and fault diagnosis Cooperation across ASes to present better visibility across domains Policies easily enforced through the server Lower overhead on network elements Caching of common queries possible Historical questions can be answered

15 Evolution path to improve accuracy A lot of room for improvement Architecture accommodates evolution so that accuracy can be improved over time Evolution path for individual layers Fiber & Fiber-spans Optical components IP links MPLS and overlay paths

16 Fiber & Fiberspans Automated mechanisms [sebos02] Inject labels through fibers or use RFID GPS to determine the location of fibers Transmit this information to the DB More coverage results in better accuracy but expensive FIBER DB GPS OPTICAL TAPS / RFID FIBER

17 Optical components Manual mechanisms Basic consistency checks Automatic correlation mechanisms such as [kompella05nsdi] to output errors Automatic mechanisms Neighbor discovery for active optical devices Configuration state from intelligent optical networks (that support dynamic restoration)

18 Optical components Neighbor discovery through periodic broadcasts at optical layer DB Configuration state during restoration Intelligent Optical Network ROUTER A ROUTER B

19 Other layers IP layer Periodically obtain configuration information to construct topology Automatically collect up/down messages to provide up-to-date view MPLS and overlay paths Static paths obtained from configuration Dynamic paths obtained by monitoring signaling messages

20 Summary Accurate associations critical to many operational tasks A bow-tie architecture for cross-layer visibility Provides the cross-layer associations as a service to various applications Allows better cooperation among ASes through standardizing what goes into the database Policy controlled export of these associations Lower overhead on network elements Allows for innovation while containing complexity

21 Future research directions Design automated mechanisms at each layer to improve cross-layer visibility What frequency should information be obtained? How do we resolve conflicts (minimal edits) in the database? Identify higher-level models that we need to standardize Devise incentives for cooperation among ASes Define a language to specify policies

22 Questions ?