1. SDN Controllers in the WAN
Julian Lucek
@julianlucek

2. Agenda
Introduction to SDN Controller for WAN Towards Self-Driving with Closed Loop Automation Automated Congestion Avoidance Dynamic Minimum Latency Path Self-Driving Egress Peering Engineering Automated Network Self-Healing P2MP Traffic Engineering

3. WAN Controller Concept
Real-time topology (BGP-LS)
Streaming Telemetry
LSP status (PCEP)
TE LSP creation/modification (PCEP, BGP SR-TE)
Requests via Northbound REST API
Controller
MPLS Network
Requests via GUI
Creation of LSPs according to service requirements.
Placement of LSP paths according to traffic demands.
Automation of network operations.

4. Topology discovery using BGP-LSF H
BW=100
Adj-SID = 78
Admin-group: green A K E
SRLG 200 C G J
Red-dashed zones denote IGP areas
At least one BGP-LS session into each IGP area
In practice, 2+ RR’s for resilience RR
BGP-LS session
Controller

5. Path computation element protocol (PCEP)
PCE: Path Computation Element
Computes the path
PCC: Path Computation Client
Receives the path. Sets up LSP using RSVP or Segment Routing.
PCEP: PCE Protocol (RFC 5440)
For PCE/PCC communication
PCEP
PCE
PCEP
PCC
PCEP
PCC
PCC

6. CLOSED-LOOP Automation
Disturbances (Outages, Congestion etc)
WAN Controller
The Network + X
User intent
Network state -
Measured output
Streaming Telemetry
Traffic on each link
Traffic on each LSP
Link latency data
Packet loss stats
Streaming Telemetry allows much more frequent updates than SNMP. Push paradigm, rather than request/response. Stats collected on linecard are sent directly from there without passing through control processor.

7. Automated actions, based on link utilisation (1)
Controller knows via Streaming Telemetry that red link is currently experiencing high utilization

8. Automated actions, based on link utilisation (2)
Controller also knows via Streaming Telemetry how much traffic is travelling on each LSP
So it automatically moves away some LSPs from the congested link

9. Programmable cost function
Cost function = lowest IGP metric path that meets the required path constraints (BW etc) 15 15 15 15 10 X 10 Y 10
Lowest IGP metric path
Blue numbers show IGP metric
Node is tagged as available again after maintenance window finishes. Optimum LSP paths are restored again through PCEP update and LSP re-signaling

10. Minimum latency path X Y Lowest latency path10 10 10 10 11 X 11 Y 10
Lowest latency path
Red numbers show latency of each link
Node is tagged as available again after maintenance window finishes. Optimum LSP paths are restored again through PCEP update and LSP re-signaling

11. Minimum latency path X Y
Latency has increased
Controller automatically moves minimum-latency LSP onto the lowest latency path 15 10 10 10 11 X 11 Y 10
Lowest latency path
Red numbers show latency of each link
Node is tagged as available again after maintenance window finishes. Optimum LSP paths are restored again through PCEP update and LSP re-signaling

12. Automated self-healing
3. REST API call:
Maintenance interval on Link L1
NorthStar WAN Controller
4. WAN Controller recomputes paths of all TE LSPs that pass through Link L1, and sends PCEP messages to corresponding ingress routers to change the paths accordingly
AppFormix
2. AppFormix determines that Link L1 is unhealthy
1. Streaming Telemetry from all network nodes L1

13. Automated self-healing
3. REST API call:
Maintenance interval on Link L1
NorthStar WAN Controller
4. WAN Controller recomputes paths of all TE LSPs that pass through Link L1, and sends PCEP messages to corresponding ingress routers to change the paths accordingly
AppFormix
2. AppFormix determines that Link L1 is unhealthy
1. Streaming Telemetry from all network nodes L1

14. Self-Driving Egress Peering Engineering

15. Self-Driving egress peering engineering (EPE)
prefix X
Each PE sends flow stats to Controller - how much traffic the PE is sending to each destination prefix.
Controller decides which {egress link, egress ASBR} each PE should use to send traffic to prefix X.
Cost function based on:
cost to reach ASBR
cost of using each peering link
current utilisation of each peering link
user-defined preferences
ASBR G
ASBR F
ASBR E
ASBR1
ASBR2
PE1
PE4
PE2
PE3

16. Self-driving egress peering engineering (EPE)
prefix X
Peering link on ASBR1 is approaching congestion outbound. Controller moves traffic from PE2 to prefix X onto one of the peering links on ASBR2.
ASBR G
ASBR F
ASBR E
ASBR1
ASBR2
PE1
PE4
PE2
PE3

17. EPE ingredients: PEER-SID
SID values manually configured on ASBR A
Advertised via BGP-LS to NorthStar
SID label steers packet towards desired peering link
ASBR A pops label, so plain IP packet is sent on the peering link
ASBR B
PeerNode SID = 100 IP IP
ASBR C
PeerNode SID = 200
ASBR A
PeerAdj SID
= 300
PeerAdj SID
= 400
ASBR D

18. Self-Driving egress peering engineering (EPE)
prefix X
ASBR G
ASBR F
ASBR E
Traffic to Prefix X from PE2 uses the SR-TE tunnel shown in red, which has been computed and instantiated by the Controller. This tunnel steers the traffic across the core to ASBR2, and then onto the peering link to ASBR F
300
ASBR1
ASBR2 26 58 17
PE1
PE4
PE2
PE3
SR-TE tunnel: Push 300, Push 26, Push 58, Push 17 (top), NH=ASBR2

19. P2MP LSPs with WAN Controller

20. P2MP LSP Path Diversity
Controller performs diverse path computation and then instantiates the P2MP LSPs using PCEP
Applications:
Broadcast TV
Financial Market Data
Air Traffic Radar
Multicast Source
Diversely routed P2MP LSPs
Receiver at Site A
Receiver at Site B

21. P2MP LSP: Physical layout

22. P2MP LSP: LOGICAL layout

23. NorthStar P2MP LSP deployment in SKY
Explain the various screenshots and explain how SKY O&M is using NS today
Source: IBC 2019 Showcase Theatre,

24. NorthStar P2MP LSP deployment in SKY
Explain the various screenshots and explain how SKY O&M is using NS today
Source: IBC 2019 Showcase Theatre,

25. NorthStar P2MP LSP deployment in SKY
Explain the various screenshots and explain how SKY O&M is using NS today
Source: IBC 2019 Showcase Theatre,

26. Useful References NorthStar Controller documentation
Blogs at
Nanog presentation
PCEP and BGP-LS deep-dive webinar

27. ThaNK YOU!