1. BGP

2. BGP Configuration Create Fabric ASN Enable BGP on a given Tenant & VRF Create BGP Neighbor and associated config eBGP Vs iBGP Route Maps BGP over OSPF BGP Policy templates Inheriting BGP Policies Restrictions and Caveats

3. Create Fabric ASN Before deploying BGP on a given tenant+VRF, BGP’s pod-level configurations need to be deployed. This consists of creating the Fabric Autonomous System Number (ASN) for the given pod and setting up spine switches to act as the route-reflectors for the MP-BGP sessions within the fabric. The configuration can be done as follows: apic1# configure apic1(config)# pod 1 apic1(config-pod)# bgp fabric apic1(config-pod-bgp)# asn 100 apic1(config-pod-bgp)# route-reflector spine 101,102 apic1(config-pod-bgp)# exit

4. Enable BGP On Tenant & VRF Before configuring BGP Neighbors for a Tenant, BGP needs to be enabled on a given tenant+VRF. The configuration can be done as follows: apic1# configure apic1(config)# leaf 101 apic1(config-leaf)# vrf context tenant bgp_Nubecentro vrf vrf1 apic1(config-leaf-vrf)# exit apic1(config-leaf)# router bgp 100 apic1(config-bgp)# vrf member tenant bgp_Nubecentro vrf vrf1

5. Create BGP Neighbor Once BGP is enabled on a given tenant & VRF, the BGP Neighbor can be created. Syntax to create BGP Neighbor is as follows: −neighbor −The neighbor’s address can either be IPv4 or IPv6 BGP Neighbor’s configuration can be done as follows: apic1(config-leaf-bgp-vrf)# neighbor 10.10.20.1 apic1(config-leaf-bgp-vrf-neighbor)# allow-self-as apic1(config-leaf-bgp-vrf-neighbor)# route-map map0 in apic1(config-leaf-bgp-vrf-neighbor)# route-map map1 out apic1(config-leaf-bgp-vrf-neighbor)# allowed-self-as-count 4 apic1(config-leaf-bgp-vrf-neighbor)# default-originate apic1(config-leaf-bgp-vrf-neighbor)# disable-connected-check apic1(config-leaf-bgp-vrf-neighbor)# disable-peer-as-check apic1(config-leaf-bgp-vrf-neighbor)# ebgp-multihop 10 apic1(config-leaf-bgp-vrf-neighbor)# local-as 1000 apic1(config-leaf-bgp-vrf-neighbor)# next-hop-self apic1(config-leaf-bgp-vrf-neighbor)# remote-as 300 apic1(config-leaf-bgp-vrf-neighbor)# send-community apic1(config-leaf-bgp-vrf-neighbor)# send-community extended apic1(config-leaf-bgp-vrf-neighbor)# address-family ipv4 unicast apic1(config-leaf-bgp-vrf-neighbor-af)# maximum-prefix 10 threshold 10 action restart restart-time 10

6. BGP Source Interfaces For a given BGP Neighbor, the source IP address can be fetched from one of the following interface types: −Ethernet (Physical) −Vlan (SVI) −Loopback Syntax to update source interface is as follows: −update-source { ethernet | vlan | loopback } Source interface can be changed as follows: apic1(config-leaf-bgp-vrf)# neighbor 10.10.20.1 apic1(config-leaf-bgp-vrf-neighbor)# update-source Ethernet 1/10 apic1(config-leaf-bgp-vrf-neighbor)# update-source loopback 10.1.1.1 apic1(config-leaf-bgp-vrf-neighbor)# update-source vlan 100 When the user does not provide any source interface, the switch will automatically choose the interface through which the peer route resolves.

7. SVI As BGP Source Interface BGP Sessions can have SVI as their source interface The Vlan ID alone is enough to make this association. The only restriction is that the SVI interface MUST have L2 interface association before making it as BGP source.

8. BGP Peer Reachability BGP Peer reachability can be achieved using static routes or OSPF When EBGP or iBGP sessions use loopback (v4, v6 or both) as source and OSPF is used to advertise the loopback, OSPF will be used only to provide connectivity to BGP. Any route-map associated for OSPF will be deleted. BGP will be the primary routing protocol. From BGP: apic1(config-leaf-bgp-vrf)# neighbor 10.10.20.1 apic1(config-leaf-bgp-vrf-neighbor)# update-source loopback 10.1.1.1 From OSPF: apic1(config)# leaf 101 apic1(config-leaf)# router ospf default apic1(config-leaf-ospf)# vrf member tenant bgp_Nubecentro vrf vrf1 apic1(config-leaf-ospf-vrf)# area 0.0.0.1 loopback 10.1.1.1 apic1(config-leaf-ospf-vrf)#end NOTE: Loopback interface is created from either one of the above configuration, which ever one is configured first.

9. Route Maps For a given BGP neighbor, we can apply route-maps in both in-bound (import) or out-bound (export) directions. Syntax: −route-map { in | out } By default, no route-maps are applied. Route-Maps can be changed as follows: apic1(config-leaf-bgp-vrf)# neighbor 10.10.20.1 apic1(config-leaf-bgp-vrf-neighbor)# route-map map0 in apic1(config-leaf-bgp-vrf-neighbor)# route-map map1 out −When no route-maps are associated, the system allows all routes in the ingress direction, but no routes are exported out. −The source of the routes exported can be other protocols like OSPF, EIGRP, Static Route or BGP from the same/different Leaf in the ACI fabric. System internally sets up the redistribute route-maps into BGP. Hence, there is no additional route-map <> redistribute commands required from the user.

10. BGP Policy Templates We can define 2 different BGP Policies as templates: −BGP Timer Policy Template −BGP Address Policy Template The templates can be defined under leaf config, but is globally visible in all other leaf switches where the tenant has VRF configured. Syntax: −template bgp { address-family | timers } tenant Sample configuration is as follows: apic1(config-leaf)# template bgp timers timer1 tenant bgp_Nubecentro This template will be available on all leaves where tenant bgp_Nubecentro has a VRF deployment apic1(config-bgp-timers)# timers bgp 10 20 apic1(config-bgp-timers)# graceful-restart stalepath-time 3600 apic1(config-bgp-timers)# exit apic1(config-leaf)# template bgp address-family bgpAF1 tenant bgp_Nubecentro This template will be available on all leaves where tenant bgp_Nubecentro has a VRF deployment apic1(config-bgp-af)# distance 250 240 230 apic1(config-bgp-af)# exit

11. Inherit BGP Policies The BGP Policy Templates defined can be inherited by any VRF under the given tenant Syntax: −inherit bgp { timers | address-family } The inherited template will apply on all leaf Switches where the VRF has been deployed. Sample configuration is as follows: apic1(config-leaf)# router bgp 100 apic1(config-bgp)# vrf member tenant bgp_Nubecentro vrf vrf1 apic1(config-leaf-bgp-vrf)# inherit bgp timer timer1 This template will be inherited on all leaves where VRF vrf1 has been deployed apic1(config-leaf-bgp-vrf)# address-family ipv4 unicast apic1(config-leaf-bgp-vrf-af)# inherit bgp address-family bgpAf1 This template will be inherited on all leaves where VRF vrf1 has been deployed apic1(config-leaf-bgp-vrf-af)# exit

12. Restrictions And Caveats Loopback IP as source interface: ACI model has a restriction that the same Loopback IP cannot be in 2 different L3Outs Due to this restriction, all BGP Neighbors using the same loopback IP will be placed in the same L3Out. This will cause any L3Out-wide operation (such as changing route-map) to affect ALL neighbors in the L3Out. BGP Session Password: There can be scenarios where BGP Neighbor object (bgpPeerP) needs to be moved from one L3Out to another based on configuration change. For example, changing loop-back for the Neighbor will force bgpPeerP MO into another L3Out. In such cases where the move happens, the user will have to re-enter the session password (if it was enabled), since there is no way for us to read and copy the password from back-end For example: apic1(config-leaf-bgp-vrf-neighbor)# update-source loopback 3010::4 Warning: BGP Configuration changed. Please re-configure BGP Password if it was enabled

13. Restrictions And Caveats (cont.) Route-Maps: When multiple BGP Neighbors have the same loopback interface as source interface, they MUST share the same route-map Changing route-map for one such BGP neighbor will affect all neighbors using the same loopback IP as the source. This is because they are share the same L3Out (due to loopback restriction discussed in the previous slide). When more than one BGP Peer use the same routemap and if the routemap on one of the peeris is changed, it may result in BGP session flap (since we may need to move the bgpPeerP MO into another L3Out corresponding to the new route-map). However, If the BGP Peer uses a unique route-map, session will not flap. (in this case, bgpPeerP MO will stay in the same L3Out). BGP Policy Templates: Even though BGP Timer and Address Policy Templates are available under leaf config, they are applicable under the entire tenant as defined by the model. When inherited under a leaf & VRF, these templates will end up getting inherited under ALL leaves where the corresponding VRF has been deployed

14. OSPF

15. Overview This section covers the CLI for OSPF protocol configuration The OSPF configuration is maintained entirely under the “leaf” mode The configuration is identical to Nexus OS CLI

16. Overview Create OSPF Area on a given Tenant & VRF Enable OSPF on Interfaces OSPF Policy Templates Inherit OSPF Policy Templates Inline Policy Configurations OSPF Default Route Leak Configuration Show Commands Restrictions and Caveats

17. Create Area Commands to create an area on a given Tenant & VRF −[no] area stub −[no] area loopback −[no] area default-cost −[no] area nssa [no-redistribute] [default-information-originate] −[no] area route-map out The configuration is under Leaf mode Area configuration example with area type “nssa” : apic1(config)# leaf 101 apic1(config-leaf)# router ospf default apic1(config-leaf-ospf)# vrf member tenant Nubecentro vrf vrf1 apic1(config-leaf-ospf-vrf)# area 1.2.3.4 nssa NOTE: There is only one OSPF and one OSPFv3 process running on the leaf and these are internally created (named “default”) by the APIC on deploying OSPF area on the leaf on IPv4 / IPv6 interfaces.

18. Create Area (cont.) Area configuration example with loopback IP: apic1(config)# leaf 101 apic1(config-leaf)# router ospf default apic1(config-leaf-ospf)# vrf member tenant Nubecentro vrf vrf1 apic1(config-leaf-ospf-vrf)# area 1.2.3.4 loopback 1.1.1.1 NOTE: Loopback interface with IP 1.1.1.1 is created and advertised out of area 1.2.3.4. One loopback address is supported per address family per area. Area configuration example with area default cost: apic1(config)# leaf 101 apic1(config-leaf)# router ospf default apic1(config-leaf-ospf)# vrf member tenant Nubecentro vrf vrf1 apic1(config-leaf-ospf-vrf)# area 1.2.3.4 default-cost 15

19. Create Area (cont.) Area configuration example with Route Map: apic1(config)# leaf 101 apic1(config-leaf)# router ospf default apic1(config-leaf-ospf)# vrf member tenant Nubecentro vrf vrf1 apic1(config-leaf-ospf-vrf)# area 1.2.3.4 route-map routemap1 out NOTES: 1)OSPF supports route control only in the export direction. The source of the routes in the route-map can be from other routing protocols, another OSPF Area, or Static / Connected routes. 2)When OSPF advertises a loopback and a BGP session uses the loopback as the source, the routemap configuration is automatically removed from OSPF. In this scenario, OSPF will only be providing connectivity to BGP. 3)Route Map related configurations will be discussed in the Route Map slides in detail.

20. Enable OSPF On Interfaces Commands for enabling OSPF on interfaces: −ip router ospf defaultarea −ipv6 router ospf defaultarea Configuration example for IPv4: apic1# configure apic1(config)# leaf 101 apic1(config-leaf)# interface ethernet 1/15 apic1(config-leaf-if)# vlan-domain member dom1 apic1(config-leaf-if)# no switchport apic1(config-leaf-if)# vrf member tenant Nubecentro vrf vrf1 apic1(config-leaf-if)# ip address 21.1.1.1/24 apic1(config-leaf-if)# ipv6 address 2111::1/64 preferred apic1(config-leaf-if)# ip router ospf default area 1.2.3.4 apic1(config-leaf-if)# ipv6 router ospf default area 1.2.3.4 NOTE: OSPF area can be enabled on the interface before OSPF VRF configuration

21. OSPF Default Route Leak Default Route Leak is supported per NSSA Area and per VRF. In Per VRF mode, default route leak is injected on all area types except “nssa”. Default Route Leak configuration per VRF example with “always” option: apic1(config)# leaf 101 apic1(config-leaf)# router ospf default apic1(config-leaf-ospf)# vrf member tenant Nubecentro vrf vrf1 apic1(config-leaf-ospf-vrf)# default-information originate always Default Route Leak configuration example for “nssa” area type: apic1(config)# leaf 101 apic1(config-leaf)# router ospf default apic1(config-leaf-ospf)# vrf member tenant Nubecentro vrf vrf1 apic1(config-leaf-ospf-vrf)# area 1.3.4.5 nssa apic1(config-leaf-ospf-vrf)# area 1.3.4.5 nssa default-information-originate

22. OSPF Policy Templates There are two kinds of OSPF Policy templates: 1)OSPF Interface Policy Template 2)OSPF VRF Policy Template The templates are defined under Leaf mode, but are common for all leaf switches in that Tenant. Syntax: −template ospf {interface-policy | vrf-policy} tenant

23. OSPF Policy Templates (cont.) OSPF Interface Policy Configuration example: apic1(config)# leaf 101 apic1(config-leaf)# template ospf interface-policy intPol1 tenant Nubecentro This template will be available on all leaves where tenant Nubecentro has a VRF deployment apic1(config-interface-policy)# ? advertise-subnetSet OSPF Interface Policy Controls costSet OSPF cost for the interface dead-intervalSet the interval hello-intervalSet interval between hello packets mtu-ignoreSet OSPF Interface Policy Controls networkSet OSPF Interface Policy Network Type passive-interfaceSet OSPF Interface Policy Controls prioritySet OSPF Interface Priority retransmit-intervalSet OSPF Policy Graceful Restart Timers transmit-delaySet the delay time needed to send an LSA

24. OSPF Policy Templates (cont.) OSPF VRF Policy Configuration example: apic1(config)# leaf 101 apic1(config-leaf)# template ospf vrf-policy vrfPol1 tenant Nubecentro This template will be available on all leaves where tenant Nubecentro has a VRF deployment apic1(config-vrf-policy)# ? auto-costSet OSPF Policy Bandwidth Reference distanceSet OSPF Policy Preferred Administrative Distance graceful-restartDisable OSPF Policy Graceful Restart Helper maximum-pathsSet the maximum ECMP for the OSPF protocol timersConfigure OSPF timers apic1(config-vrf-policy)# timers ? lsa-arrivalSet the minimum interval between the arrival of LSA lsa-group-pacingSet the interval in which LSAs are grouped throttleConfigure OSPF Timers Policy

25. Inherit OSPF Interface Policy Templates Command Syntax: −{ip | ipv6} ospf inherit interface-policy tenant Inherit interface policy configuration example: apic1# configure apic1(config)# leaf 101 apic1(config-leaf)# interface Ethernet 1/15 apic1(config-leaf-if)# vlan-domain member dom1 apic1(config-leaf-if)# no switchport apic1(config-leaf-if)# vrf member tenant Nubecentro vrf vrf1 apic1(config-leaf-if)# ip address 21.1.1.1/24 apic1(config-leaf-if)# ip router ospf default area 1.2.3.4 apic1(config-leaf-if)# ip ospf inherit interface-policy intPol1 apic1(config-leaf-if)# ipv6 ospf inherit interface-policy intPol2 Use “inherit” to apply the template policy on a VRF, interface. Templates are optional. When no template is inherited, “default” template from tenant common is inherited. Policies inherited are address family specific.

26. Inherit OSPF Interface Policy Templates (cont.) Command Syntax: −inherit {ip | ipv6} ospf vrf-policy Inherit VRF policy configuration example for IPv4: apic1(config)# leaf 101 apic1(config-leaf)# router ospf default apic1(config-leaf-ospf)# vrf member tenant Nubecentro vrf vrf1 apic1(config-leaf-ospf-vrf)# inherit ip ospf vrf-policy vrfPol1 apic1(config-leaf-ospf-vrf)# inherit ipv6 ospf vrf-policy vrfPol1

27. OSPF Policy Templates (cont.) OSPF interface policy can be configured in inline mode, where the policy parameters are configured specific to that interface. Inline OSPF Interface Policy Configuration example: apic1(config)# leaf 101 apic1(config-leaf)# interface ethernet 1/15 apic1(config-leaf-if)# ip ospf ? advertise-subnetSet OSPF Interface Policy Controls authenticationSet the OSPF authentication type authentication-keySet OSPF authentication key costSet OSPF cost for the interface dead-intervalSet the interval dead interval hello-intervalSet interval between hello packets mtu-ignoreSet OSPF Interface Policy Controls networkSet OSPF Interface Policy Network Type passive-interfaceSet OSPF Interface Policy Controls prioritySet OSPF Interface Priority retransmit-intervalSet OSPF Policy Graceful Restart Timers transmit-delaySet the delay time needed to send an LSA

28. OSPF Show Commands Show command example: apic1# show running-config leaf 101 leaf 101 template ospf interface-policy nss tenant Nubecentro ospf hello-interval 10 ospf retransmit-interval 5 ospf transmit-delay 1 exit template ospf vrf-policy default tenant Nubecentro exit interface ethernet 1/15 vlan-domain member dom1 no switchport vrf member tenant Nubecentro vrf vrf1 ip address 21.1.1.1/24 exit router ospf default vrf member tenant Nubecentro vrf vrf1 area 0.0.0.3 loopback 222:3:1::101 area 0.0.0.3 loopback 222.3.1.101 area 0.0.0.3 nssa area 0.0.0.3 route-map t3:ctx1_101_rm1 out exit

29. Restrictions And Caveats OSPF can not be enabled on an interface if EIGRP is already enabled Area properties need to be same for both IPv4 and IPv6 for one area ID. When BGP uses OSPF for connecitivity through loopback Source IP, OSPF is not used to advertise other routes except for the loopback. On a OSPF Area, only one Loopback IP per address family can be advertised. The above restrictions are imposed in the policy model implementation and are not CLI specific.