Improving Availability in Multilayer Switched Networks

Multilayer Network Design Access Distribution Backbone Core Cisco Introduced the multilayer design model 3 years ago. The design model is easy to scale, understand and troubleshoot. We take a layered approach in building this model, Access, Distribution and Core. Each layer serves its own purpose. Access layer, Server Farm, WAN, Internet and PSTN are all modules which plug in as building blocks in this model. This is a very common implementation of the multilayer design model. In this model we utilize Layer 2 switching in the wiring closet (access layer) and utilize layer 3 and higher protocols as we go up to distribution and core. A good design is a design which incorporates a balance of both layer 2 and layer 3 without adding complexity of either by running a pure layer 2 or layer 3 based network. Again, we’re going to focus on the access block. Building Block Additions Server Farm WAN Internet PSTN

Multi-VLAN Load Balancing Methods VLAN A and B VLAN Trunk A&B Fwd VLAN B Block VLAN A Fwd VLAN A Block VLAN B Layer-2 Mode Load Balancing VLAN A and B VLAN Trunk A&B Forward VLAN B Forward VLAN A Layer-3 Mode Load Balancing HSRP 1A HSRP 2s HSRP 1s HSRP 2A Let’s summarize the two load balancing or load sharing methods at the access layer. Layer 2 load balancing: access switch supports two vlans, trunks to distribution layer, layer 2 trunk between switches. Spanning tree provides loop free network, root is at active layer 3 switch. Layer 3 load balancing: access switch supports two vlans, trunks to distribution layer, layer 3 link between switches. Both trunks are forwarding. Two HSRP groups, one active on one switch, one active on the other switch. Half of the devices use one virtual IP as the default gateway, half use the other. Layer 3 load balancing is preferred because it reduces dependency on spanning tree. But it does require more planning and definition. Or does it? What if we had a way….

First Hop Redundancy Schemes Hot Standby Router Protocol (HSRP) Cisco informational RFC 2281 ( March 1998) Virtual Router Redundancy Protocol (VRRP) IETF Standard RFC 2338 (April 1998) Gateway Load Balancing Protocol (GLBP) Cisco designed, load sharing, patent pending

HSRP A group of routers function as one virtual router by sharing ONE virtual IP address and ONE virtual MAC address One (Active) router performs packet forwarding for local hosts The rest of the routers provide “hot standby” in case the active router fails Standby routers stay idle as far as packet forwarding from the client side is concerned In an HSRP or VRRP group, one router is elected to handle all requests sent to the virtual IP address. With HSRP, this is the active router. An HSRP group has one active router, at least one standby router, and perhaps many listening routers. A VRRP group has one master router and one or more backup routers. MHSRP provides 2 different standby interfaces under one interface. But some DHCP server like Cisco Network Registrar has to take responsibility of assigning the default gateway to one of the virtual IP addresses in a round robin fashion.

First Hop Redundancy with HSRP R1- Active, forwarding traffic; R2, R3 - hot standby, idle HSRP ACTIVE HSRP STANDBY HSRP LISTEN IP: 10.0.0.254 MAC: 0000.0c12.3456 vIP: 10.0.0.10 vMAC: 0000.0c07ac00 IP: 10.0.0.253 MAC: 0000.0C78.9abc vIP: vMAC: IP: 10.0.0.252 MAC: 0000.0cde.f123 vIP: vMAC: Gateway routers R1 R2 R3 HSRP provides a redundant default gateway service for a common subnet Two or more routers in an HSRP group share one virtual MAC address and one virtual IP address Clients on the subnet set their default gateway to the virtual IP address of the HSRP group During normal operation, only the primary router (and associated uplink) provides gateway service to all clients on the subnet Non-primary routers (and associated uplinks) remain idle - wasting uplink bandwidth with no load balancing Multi-group HSRP (mHSRP) can be used for load balancing between the HSRP groups. However, this requires that clients on a common subnet be divided and configured with different default gateways - introducing administrative overhead, making plug-and-play client configuration impossible. CL1 CL2 CL3 Clients IP: 10.0.0.1 MAC: aaaa.aaaa.aa01 GW: 10.0.0.10 ARP: 0000.0c07.ac00 IP: 10.0.0.2 MAC: aaaa.aaaa.aa02 GW: 10.0.0.10 ARP: 0000.0c07.ac00 IP: 10.0.0.3 MAC: aaaa.aaaa.aa03 GW: 10.0.0.10 ARP: 0000.0c07.ac00

VRRP Very similar to HSRP A group of routers function as one virtual router by sharing ONE virtual IP address and ONE virtual MAC address One (master) router performs packet forwarding for local hosts The rest of the routers act as “back up” in case the master router fails Backup routers stay idle as far as packet forwarding from the client side is concerned In HSRP, both the active and standby routers send periodic messages (known as hello messages). In VRRP, only the master sends periodic messages (known as advertisements). Same problem with load balancing requirements. Cisco developed HSRP in response to emerging customer requirements. The company continues to enhance its capability based on customer feedback and market direction. Widely deployed by many Cisco customers, HSRP is a time-proven feature of Cisco IOS software. It has some great benefits like HSRP tracking and preempt feature which is not available in standardized VRRP. However we strive to be standards compliant and therefore we are in the process of supporting VRRP. VRRP is also something which will be useful interoperating with 3rd party switches for gateway redundancy.

First Hop Redundancy with VRRP R1- Master, forwarding traffic; R2, R3 - backup VRRP ACTIVE VRRP BACKUP VRRP BACKUP IP: 10.0.0.254 MAC: 0000.0c12.3456 vIP: 10.0.0.10 vMAC: 0000.5e00.0100 IP: 10.0.0.253 MAC: 0000.0C78.9abc vIP: vMAC: IP: 10.0.0.252 MAC: 0000.0cde.f123 vIP: vMAC: Gateway routers R1 R2 R3 Same problem with load balancing requirements…only 1 router is active forwarding traffic from the client subnet to outside. CL1 CL2 CL3 Clients IP: 10.0.0.1 MAC: aaaa.aaaa.aa01 GW: 10.0.0.10 ARP: 0000.5e00.0100 IP: 10.0.0.2 MAC: aaaa.aaaa.aa02 GW: 10.0.0.10 ARP: 0000.5e00.0100 IP: 10.0.0.3 MAC: aaaa.aaaa.aa03 GW: 10.0.0.10 ARP: 0000.5e00.0100

GLBP Defined A group of routers function as one virtual router by sharing ONE virtual IP address but using Multiple virtual MAC addresses for traffic forwarding Provides uplink load-balancing as well as first hop fail-over IP Leadership feature

GLBP Requirements Allow traffic from a single common subnet to go through multiple redundant gateways using a single virtual IP address Provide upstream load-balancing by utilizing the redundant up-links simultaneously Eliminate the need to create multiple vLANs or manually divide clients for multiple gateway IP address assignment Preserve the same level of first-hop failure recovery capability as provided by HSRP

First Hop Redundancy with GLBP R1- AVG; R1, R2, R3 all forward traffic GLBP AVG/AVF,SVF GLBP AVF,SVF GLBP AVF,SVF IP: 10.0.0.254 MAC: 0000.0c12.3456 vIP: 10.0.0.10 vMAC: 0007.b400.0101 IP: 10.0.0.253 MAC: 0000.0C78.9abc vIP: 10.0.0.10 vMAC: 0007.b400.0102 IP: 10.0.0.252 MAC: 0000.0cde.f123 vIP: 10.0.0.10 vMAC: 0007.b400.0103 Gateway routers R1 R2 R3 A redundancy group will consist of one virtual IP address and multiple virtual MAC addresses Three main functions: Active Virtual Gateway responds to all ARP requests with the designated virtual MAC address according to the load balancing algorithm. Each member of the group monitors state of other member gateways. In the event of failure, a secondary virtual forwarder takes over for traffic destined to a virtual MAC impacted by the failure. Default load balancing algorithm AVG uses to assign virtual mac to clients is round-robin. Others are host-dependent and Weighted. Benefits: Simplified configuration, less administration, increased throughput in non-failure conditions. CL1 CL2 CL3 Clients IP: 10.0.0.1 MAC: aaaa.aaaa.aa01 GW: 10.0.0.10 ARP: 0007.B400.0101 IP: 10.0.0.2 MAC: aaaa.aaaa.aa02 GW: 10.0.0.10 ARP: 0007.B400.0102 IP: 10.0.0.3 MAC: aaaa.aaaa.aa03 GW: 10.0.0.10 ARP: 0007.B400.0103

Campus Access Layer Design GLBP balances traffic across both layer-3 switches Better utilization of resources and uplinks Campus Network Layer-3 switches at distribution layer 10.88.49.10 10.88.50.10 vIP address vMAC A vMAC C vMAC B vMAC D Layer-2 switches at access layer These next few slides show some typical design where GLBP can be used. Here you see a diagram of a typical campus with access and distribution layer. You could use GLBP in the layer 3 switches to load balance traffic from end stations on a common IP subnet. All devices within a subnet could point to a common default gateway, 10.88.49.10. Traffic would be handled by each layer-3 switch on a per-host basis. Tip: Set the layer-2 CAM entry aging time in distribution layer switches to the same duration as the ARP timeout. This will minimize any flooding of IP unicast traffic upon expiration for traffic that is never received for a given MAC. set cam agingtime 1-1000 14400 Will set the time to 4 hours, same as default ARP cache timeout for MSFC A A D B C A D B C GW= 10.88.49.10 GW= 10.88.50.10

Service Provider Edge High Availability for Remote Office GLBP balances traffic across both routers Better utilization of resources and uplinks SP Network Redundant CPE routers 10.88.49.10 10.88.50.10 vIP address vMAC A vMAC C vMAC B vMAC D Here you see a diagram of a typical remote office with redundant CPE routers and links to a service provider network. You could use GLBP in the routers to load balance traffic from end stations on a common IP subnet. You could use one IP subnet per wiring closet switch. All devices on a switch could point to a common default gateway. Traffic would be handled by each router on a per-host basis. The benefit is that you can now use both links concurrently with simplified configuration. Layer-2 switches at access layer A D B C A D B C GW= 10.88.49.10 GW= 10.88.50.10

Some application but SLB more appropriate Server Farm Example L2 Dual-homed servers for port and switch redundancy Layer-2 switches at access layer Layer-3 switches at distribution layer GLBP balances traffic across both layer-3 switches Some application but SLB more appropriate 10.88.49.10 vIP address Better utilization of resources and uplinks Here is an example of a typical server farm connected at layer-2 and then to distribution layer-3 switches. Designs exist for using HSRP in this environment. This design has yet to be verified with GLBP. Campus Network

SLB – Server Load Balancing SLB Presents a Virtual Address and Load Balances the Traffic Across Multiple Servers Virtual Server: Represents an instance of a server farm Real Server: An individual server within the farm Virtual IP 192.168.1.200 192.168.1.1 80 192.168.1.2 80

SLB Benefits High performance is achieved by distributing client requests across a cluster of servers. Administration of server applications is easier Clients know only about virtual servers No administration is required for real server changes Maintenance with continuous availability is achieved by allowing physical (real) servers to be transparently placed in or out of service Security of the real server is provided because its address is never announced to the external network Users are familiar only with the virtual IP address Filtering of unwanted traffic can be based on both IP address and IP port numbers

MSFC2 High Availability Features Provides multilayer switching and routing services between switched VLANs Dependent on Supervisor Supervisor reset or failure will reset the MSFC2 Operates in Dual Router Mode (DRM) or Single Router Mode (SRM)

Dual Router Mode (DRM) Both MSFCs online Each MSFC independently builds an accurate picture of the Layer 3 network The failover mechanism between MSFCs in DRM is the HSRP MSFCs maintain nearly identical configurations First online is ‘designated router’, second is ‘non-designated router’ Designated router programs the Layer 3 entries in the PFC2s Cisco Express Forwarding (CEF) table

MSFC Config Sync Startup and running configurations between the designated (primary) and nondesignated (secondary) MSFCs are synchronized The following commands enable MSFC config-sync: Configuration of the nondesignated MSFC is accomplished through the use of the alt keyword MSFC-Sup-15 (config)# redundancy MSFC-Sup-15 (config-r)# high-availability MSFC-Sup-15 (config-r-ha)# config-sync MSFC-Sup-15 (config-if)# ip address a.b.c.1 x.x.x.0 alt ip address a.b.c.2 x.x.x.0 MSFC-Sup-15 (config-if)# standby 10 priority 100 alt standby 10 priority 50

Sample DRM Configuration hostname DRM ! redundancy high-availability config-sync interface Vlan20 ip address 10.20.1.3 255.255.255.0 alt ip address 10.20.1.2 255.255.255.0 standby ip 10.30.1.4 standby priority 100 alt standby priority 50 no ip redirects interface Vlan30 ip address 10.30.1.3 255.255.255.0 alt ip address 10.30.1.2 255.255.255.0 standby ip 10.30.1.4 standby priority 100 alt standby priority 50 end

DRM Challenges Each MSFC must have a unique IP address for each VLAN interface At least one router (the other MSFC) on each VLAN receives non-RPF traffic when multicast is used Requirement for exact configuration parameters on both MSFCs complicates matters

SRM – Single Router Mode Single Router Mode (SRM) addresses the drawbacks of the previous HSRP based redundancy scheme Only the designated router (MSFC) is visible to the network at any given time Non-designated router is booted up completely and participates in configuration synchronization, which is automatically enabled when entering SRM Non-designated router interfaces are kept in a "line down" state and are not visible to the network

SRM Requirements Both MSFCs must run the same IOS image High availability needs to be configured on the SUP Routing protocol processes are also created on the non-designated router, but dormant MSFC-Sup-15 (config)# redundancy MSFC-Sup-15 (config-r)# high-availability MSFC-Sup-15 (config-r-ha)# single-router-mode

Sample SRM Configuration hostname SRM ! redundancy high-availability single-router-mode interface Vlan20 ip address 10.20.1.3 255.255.255.0 no ip redirects interface Vlan30 ip address 10.30.1.3 255.255.255.0 end

Verify SRM Configuration SRM# show redundancy Designated Router: 1 Non-designated Router: 2 Redundancy Status: designated Config Sync AdminStatus : enabled Config Sync RuntimeStatus: enabled Single Router Mode AdminStatus : enabled Single Router Mode RuntimeStatus: enabled Single Router Mode transition timer : 120 seconds sh redundancy command can be used to verify that SRM is enabled: Transition timer is used to ensure routing protocol convergence prior to PFC updates

Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 26 26 26