© 2006 Cisco Systems, Inc. All rights reserved.Presentation_ID 1 Configuring Cisco Nexus 5000 Switches in NPV Mode Module 9

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 2 Objectives NPV Overview Configuration of switch in NPV mode Show commands and troubleshooting Future NPV features

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 3 N-Port Virtualization (NPV): An Overview N5K FCOE NPV Device Uses the same domain(s) as the NPV-core switch(es) F-port NPV-Core Switch (MDS or 3 rd party switch with NPIV support) NP-port FC VSAN Solves the domain-id explosion problem FC VSAN (no FL ports) VSAN 15 FC server port (F) Target Initiator Can have multiple uplinks, on different VSANs (port channel and trunking in a later release)

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 4 NPV: Supported Hardware & Mgmt.  NPV Switch Mode Supported on: All MDS blade switches MDS switches (9124, 9134, etc.) Nexus 5020, 5010 UCS 6100  NPV-Core Switch MDS 9513, MDS 9509, 3 rd party switch with NPIV support (Support Matrix)  Management NPV device has it’s own IP address and management port – for mgmt. & debugging All relevant configs. are supported via SNMP and CLI FM support for discovering and configuring NPV switches (e.g. NP port mode) No change in image upgrade and installation procedure

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 5 NPV Features Continued  Different uplinks from an NPV device can be connected in different VSANs ( Trunking and Port Channeling of F ports will be supported in a later release)  If one of the uplink fails then the corresponding servers would be logged in via other uplinks (if any) automatically  Zoning, DPVM…, are enforced on the NPV-core switch only  Domain manager, FSPF, Zone server, Fabric login sever, Name server do not run on the NPV device  Local switching is disabled – all traffic routed via the NPV-core switch  No license required for NPV functionality

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 6 NPV: Internal Logins (FLOGIs)  When an NP port comes up NPV itself first FLOGIs into the core npv-core# show fcns database detail VSAN:1 FCID:0x port-wwn (vendor) :20:02:00:0d:ec:2f:c1:40 (Cisco) node-wwn :20:01:00:0d:ec:2f:c1:41 class :2,3 node-ip-addr : ipa :ff ff ff ff ff ff ff ff fc4-types:fc4_features :npv symbolic-port-name :para-3:fc1/2 symbolic-node-name :para-3 port-type :N port-ip-addr : fabric-port-wwn :20:02:00:0d:ec:04:99:40 hard-addr :0x permanent-port-wwn (vendor) :20:02:00:0d:ec:2f:c1:40 (Cisco) NPV NPV-Core Switch NP fc1/2 F P1 P2 fwwn of Port P1 fwwn of Port P2 VSAN WWN of NPV switch name of NPV & Interface name of P2 IP Address of NPV

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 7 NPV: Logins from End Devices (FDISCs)  An end port logs into the npv-core as follows: VSAN:1 FCID:0x331e port-wwn (vendor) :2f:ff:00:06:2b:10:c4:4c (LSI) node-wwn :2f:ff:00:06:2b:10:c4:4c class :3 node-ip-addr : ipa :ff ff ff ff ff ff ff ff fc4-types:fc4_features :scsi-fcp:init symbolic-port-name :LSI7404XP-LC BR A A FW: Port 0 symbolic-node-name : port-type :N port-ip-addr : fabric-port-wwn :20:02:00:0d:ec:04:99:40 hard-addr :0x permanent-port-wwn (vendor) :20:02:00:0d:ec:2f:c1:40 (Cisco) NPV NPV-Core Switch NP fc1/2 F P1 P2 fc1/1 N F P3 P4 LSI FCID 0X331e00 fwwn of Port P1 fwwn of Port P2 NPV converts FLOGI to FDISC

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 8 NPV: Distribution of end Device Logins NPV NPV-Core Switch NP Ports P1 P2 P3 NP Port No. of mapped F ports P1 5 (fc1/1, 1/5...) P2 2 (fc1/7, fc1/21) P3 9 (fc1/2, fc1/8,...) Next F port on NPV would be assigned to NP Port P2 FC F An example of current mapping of ports (NP port with minimum number of mapped F ports)

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 9 NPV: What happens when an NP port goes down?  All F ports mapped to that NP port are re-initialized (shut  no shut)  These N ports would attempt re-login  If another NP port is available N ports would be logged in via the available NP port(s) Logins would be distributed as per the previous slide  If no NP port is available F ports would remain in down state waiting for an NP port  When the failed NP port comes back up The logins are NOT re-distributed (to avoid disruption) NPV NP Ports P1 P2 P3 F F F

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 10 NPV: What happens when an F port goes down?  NPV sends a LOGO to NPV-core on behalf of the N port If there were multiple FCIDs assigned to that N port (NPIV-enabled host) then a LOGO is sent to the NPV-core for each FCID  The F port is not allowed to come back up until the LOGO(s) are completed  Conflict in Port-security, DPVM or FC-SP configuration on NPV-code switch can logout an FCID or prevent it from coming up NPV NP Ports P1 P2 P3 FC F

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 11 NPV Feature  NPV is enabled with the following commands: switch(config)# feature fcoe 2008 Aug 6 19:02:21 switch %$ VDC-1 %$ %PFMA-2- FC_LICENSE_DESIRED: FCoE/FC feature will be enabled after the configuration is saved followed by a reboot switch(config)# npv enable Verify that boot variables are set and the changes are saved. Changing to npv mode erases the current configuration and reboots the switch in npv mode. Do you want to continue? (y/n): This does a write-erase and reloads the switch in NPV mode (same image) “no npv enable” to go back to non-npv mode (also involves write-erase and reload)  Only the following port modes allowed: NP (a new port mode), F and SD  If there are multiple uplinks then the server logins are equally distributed amongst them  Uplinks from an NPV device can be connected to multiple aggregation switches

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 12 NPV: Configuration NPV NP Port P1 (fc3/4) FC F NPV-Core Switch 1.Enable NPIV on NPV-Core switch npv-core(config)# npiv enable 2.Set port mode of P2 to F npv-core(config)# interface fc8/14 npv-core(config)# switchport mode F 3.Set the Port VSAN of this F port npv-core(config)# vsan database npv-core(config-vsan-db)# vsan 5 interface fc8/14 4.Enable NPV on Nexus 5020 npv(config)# npv enable npv switch would reboot 5.Set Port mode of P1 to NP npv(config)# interface fc3/4 npv(config-if)# switchport mode NP 6.Set Port mode of P3 to F npv(config)# interface vfc19/1 npv(config-if)# switchport mode F 7.Set the Port VSAN of NP and VFC port to be same as the F port on NPV-core switch npv(config)# vsan database npv(config-vsan-db)# vsan 5 interface fc3/4 npv(config-vsan-db)# vsan 5 interface vfc23/1 8.No shut ports P1, P2, and P3 P2 (fc8/14) P3 (vfc23/1)

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 13 NPV: Default Port Mode Configuration switch# show interface brief Interface Vsan Admin Admin Status SFP Oper Oper Port Mode Trunk Mode Speed Channel Mode (Gbps) fc3/1 1 NP -- down swl fc3/2 1 NP -- down swl … By default the FC ports come up as NP They can be modified: switch(config)# int fc3/1 switch(config-if)# switchport mode f switch# show interface brief Interface Vsan Admin Admin Status SFP Oper Oper Port Mode Trunk Mode Speed Channel Mode (Gbps) fc3/1 1 F -- down swl fc3/2 1 NP -- down swl …

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 14 NPV Related Show Commands on NPV Switch The following show commands can be used on the NPV switch to display info. show npv flogi-table show npv status show tech-support npv show npv internal event-history { errors | events | ext-if-fsm [ ] | flogi-fsm [ interface | pwwn ] } show npv internal event-history msgs show npv internal event-history svr-if-fsm [ interface ] show npv internal info show npv internal info external-interface { all | } show npv internal info global show npv internal info interface { all | } show npv internal info server-interface { all | } show npv internal mem-stats [ detail ] show npv internal pending-queue interface { all | debug npv { all | erros | events |... } show debug npv

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 15 NPV Related Show Commands on NPV Switch switch# sh npv flogi-table SERVER EXTERNAL INTERFACE VSAN FCID PORT NAME NODE NAME INTERFACE vfc23/1 5 0xa :00:00:00:c9:3c:f4:82 20:00:00:00:c9:3c:f4:82 fc3/1 Total number of flogi = 1. switch# show npv status npiv is disabled External Interfaces: ==================== Interface: fc3/1, VSAN: 5, FCID: 0xa40000, State: Up Number of External Interfaces: 1 Server Interfaces: ================== Interface: vfc19/1, State: Initializing Interface: vfc23/1, VSAN: 5, State: Up Number of Server Interfaces: 2

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 16 NPV Related Show Commands on NPV Switch switch# show npv internal info NPV Globals: ============ NPV mode: ENABLED Switch-Name: switch Switch Mgmt IP Address: proxy logo Retries: 1 Internal FLOGI max timeout Retries: -1 NS Registration max timeout Retries: 5 External Interface Info: ======================== Interface Information: ifindex: fc3/1, VSAN: 5, internal FLOGI fcid: 0xa40000 FSM current state: NPIVP_EXT_IF_ST_UP fabric mgmt addr: fabric pwwn: 20:4d:00:0d:ec:3f:a5:c0, fabric nwwn: 20:05:00:0d:ec:3f:a5:c1 my pwwn: 20:81:00:0d:ec:6b:cd:40, my nwwn: 20:05:00:0d:ec:6b:cd:41 Port Parameters: Rx B2B Credits: 16, Multiplier: 0, Buff Size: 2112 Tx B2B Credits: 16, Multiplier: 0, Buff Size: 2112, bbscn: 0 bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0 N5K NPV Fabric Login Timer & Retry Information: Busy Timer (1), id: 10, active: FALSE time remaining: 0 Fail Retry Timer (7), id: 00, active: FALSE time remaining: 0 FDISC Response Timer (2), id: 01, active: FALSE time remaining: 0 Error Clear Timer (6), id: 01, active: TRUE time remaining: 159 Statistics: flogi retry count : 1 ns registration retry count : 0 number of LS_RJTs rcvd : 0 LS_RJT timestamp : Wed Aug 6 20:58: pending queue size : 0 FLOGIs on this interface: pwwn : 10:00:00:00:c9:3c:f4:82, fcid : 0xa40001

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 17 NPV Related Show Commands on NPV Switch VFC (FCOE) interface waiting for FLOGI from host VFC (FCOE) interface up and working Interface Information: ifindex: vfc23/1, VSAN: 5, NPIV requested: TRUE, lcp init done: TRUE Selected External Interface: fc3/1 FSM current state: NPIVP_SVR_IF_ST_INTERFACE_UP Port Parameters: rxbbcredit: 0 rxbufsize: 2112 txbbcredit: 64 txbufsize: 2048 txbbscn: 0 bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0 Statistics: number of LS_RJTs sent : 0 LS_RJT timestamp : Wed Aug 6 20:58: number of FLOGIs: 1 pwwn : 10:00:00:00:c9:3c:f4:82, fcid : 0xa40001, requested fcid : 0x000000, ex t ifindex: fc3/1 class: 3 logo retries done: 0, flogi accept sent: TRUE Unknown Timer (0), id: 00, active: FALSE time remaining: 0 FSM current state: NPIVP_FLOGI_ST_STEADY_STATE switch# show npv internal info (output continued) Server Interface Info: ====================== Interface Information: ifindex: vfc19/1, VSAN: 5, NPIV requested: FALSE, lcp init done: TRUE Selected External Interface: fc3/1 FSM current state: NPIVP_SVR_IF_ST_WAITING_FOR_FLOGI Port Parameters: rxbbcredit: 0 rxbufsize: 2112 txbbcredit: 0 txbufsize: 0 txbbscn: 0 bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0 Statistics: number of LS_RJTs sent : 0 LS_RJT timestamp : Thu Jan 1 00:00: number of FLOGIs: 0

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 18 NPV Related Show Commands on NPV-Core Switch The following show commands can be used in the NPV-core switch to display information on the NPV devices. show fcns database npv [ detail [ vsan ]] show fcns database npv [ node_wwn ] [ vsan ] Example Outputs switch# show fcns database npv VSAN 5: NPV NODE-NAME NPV IP_ADDR NPV IF CORE SWITCH WWN CORE IF :05:00:0d:ec:6b:cd: fc3/1 20:00:00:0d:ec:3f:a5:c0 fc2/13 Total number of entries = 1

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 19 NPV Related Commands on NPV-Core Switch switch# show fcns database npv detail VSAN:5 NPV Node Name: 20:05:00:0d:ec:6b:cd: NPV Fabric Port-WWN :20:81:00:0d:ec:6b:cd:40 class :2,3 NPV IP Address : ipa :ff ff ff ff ff ff ff ff fc4-types:fc4_features :npv NPV Switch Name:Interface :switch:fc3/1 port-type :NP Core Switch fabric-port-wwn :20:4d:00:0d:ec:3f:a5:c0 permanent-port-wwn (vendor) :20:81:00:0d:ec:6b:cd:40 (Cisco) switch# show fcns data npv node_wwn 20:05:00:0d:ec:6b:cd:41 vsan 5 VSAN 5: FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE xa40001 N 10:00:00:00:c9:3c:f4:82 (Emulex) ipfc scsi-fcp:init Total number of entries = 1

© 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 20