1. ACI switch New Hardware For Brazos

2. New Hardware Switch hardware for ACI FCS
TORS: N9K-C93120TX also known as [Squaw Valley]
TORS: N9K-C9372PX-E, also known as [Oslo Plus]
GEM: N9K-M6PQ-E also known as [Coors Plus]
Troubleshooting
Show commands
Debug commands
Faults/Events
Diagnostics

3. N9K-C93120TX – System Interface 1/97 Interface 1/102 Fixed Chassis
96- Front Ports 10GT copper
6-Port QSFP+ Uplink
Front-to-Back & Back-to-Front Airflow
Wire-Speed L2/L3 Forwarding

4. N9K-C93120TX – PSU and Fan FAN #1
Require 1200W AC power supply or 930w DC power supply
PSUs are Hot Swappable
There are two PSUs 1+1
Following new PSU modules will be supported on C93120TX:
NXA-PAC-1200w-PE
NXA-PAC-1200w-PI
UCS-PSU-6332-DC
There are three Fans 2+1
Air flow for PSU and Fan has to be matched
FAN #3
PS 2
PS 1
1200W (hot air out) or
1200W (cold air in)
N9K-C9300-FAN2 (hot air out) Or
N9K-C9300-FAN2-B (cold air in)

5. N9K-C93120TX – Features
93120TX has the following modular components which can be field upgradable or replaced in case of failure to minimize the system downtime.
Fan Tray
Power Supply
The C93120TX also has an OBFL (On-Board Failure Logging) for failure analysis and trend identifying in the field.
Interface 1/1-96 are 10GB copper ports.
Interface 1/97-1/102 are 40 GB ports.
Management interface eth0
SSD 64GB
2.5GHz Ivy bridge Gladden 2 Core CPU.

6. N9K-C93120TX – Block Diagram

7. N9K-C9372PX-E – Oslo Plus
New N9K-C9372PX-E has new Cisco ASIC – Donner rev C
There is no change in both hardware and software functionality.
Interface 1/49-54
Interface 1/1-48

8. N9K-M6PQ-E – Coors Plus
New N9K-M6PQ-E has new Cisco ASIC – Donner rev C
There is no change in both hardware and software functionality.

9. Troubleshooting (same as previous hardware
Troubleshooting (same as previous hardware. No changes from previous hardware)

10. MAC Port Status
To view the MAC port status, login to the Leaf and run the show command on the line card.
apic1#
apic1#attach Leaf101
Leaf101#vsh_lc
module-1#show platform internal ns mac

11. MAC Port Counters
To view the MAC port counters, login to the Leaf and run the show command on the line card.
apic1#
apic1#attach Leaf101
Leaf101#vsh_lc
module-1#show platform internal ns counters mac

12. NS/Alpine ASIC Block Counters
To view the MAC port counters, login to the Leaf and run the show command on the line card.
apic1#
apic1#attach Leaf101
Leaf101#vsh_lc
module-1#show platform internal ns counters mac

13. NS/Alpine Error Interrupts
To view the MAC port counters, login to the Leaf and run the show command on the line card.
apic1#
apic1#attach Leaf101
Leaf101#vsh_lc
module-1#show platform internal ns interrupts

14. Overview
All aspects of platform are monitored by diagnostics [device tests, run time tests, LC/FC link issues etc]. A separate presentation is dedicated just for that.
Statistics presentation captures in detail, how to look for and dump LC/FC connections/stats and debug.
This presentation will capture other failures and debug commands [mostly eobc/epc].

15. Show commands
The show commands are best run from inside Bash. VSH has similar commands but are not as accurate. Show space double escape will show you all the show possibilities.
If you need to go to the objects they can be found under /mit/sys/ch

16. Debug commands:
Issue: The switch continuously reboots ( Non DME processes do not support restarts. Any process crash will result in the switch rebooting.)
Solution:
While the switch is rebooting,
Break into the loader (press CTRL-C when the CISCO BIOS message appears as the switch is rebooting)
From the loader prompt:
cmdline clear_config no_hap_reset
boot <image-name>
Issue: Module boot failures
Solution:
Leaf101#show module interval event-history module (#)
(If there are timeouts or errors, then the module is probably the bad module)

17. Debug commands: (EOBC/EPC Part 1)
Debug platform internal emon options:
Heartbeat interval get:
Leaf101# debug platform internal emon heartbeat interval get
Heartbeat interval is 300
EOBC mon statistics:
Leaf101# debug platform internal emon stats get 1
FSM ID is 1
Number of Heartbeat packets sent: 15358
Number of total packets sent: 4865
Number of total packets received: 20223
Number of total packet tx errors: 0
Disable eobc/epc heartbeats:
Enable eobc/epc heartbeats:

18. Debug commands: (EOBC/EPC Part 2)
EOBC mon kfsm state for FSM 0:
EOBC mon kfsm state for FSM 1:
Emon vsh_lc commands on line cards (FC, LC and SC) are below:
EOBC/EPC switch status (switch 0: EOBC, switch 1: EPC)
Switch type: Marvell 98DXN port switch
Port Descr Enable Status Aneg Speed Mode InByte OutByte InPkts OutPkts
           
0 SC1-EOBC-switch Yes UP No
1 SC2-EOBC-switch No DOWN No
2 Local-CPU-EOBC Yes UP No
3 SC1-EPC-switch Yes UP No
4 SC2-EPC-switch No DOWN No
5 Local-CPU-EPC Yes UP No
6 Not-Connected No DOWN No
7 Not-Connected No DOWN No
8 Not-Connected No DOWN No
9 Not-Connected No DOWN No

19. Debug commands: (EOBC/EPC Part 3)
SC card broadcom switch status:
EOBC - BCM56321 switch status
=============================
Port number BCM number Type Enbld Speed Aneg Dplx Link Encap
                  
0 2 GE Yes 1000 No 1 Down 0
1 3 GE Yes 1000 No 1 Down 0
2   4 GE Yes 1000 No 1 Down 0
3 5 GE Yes 1000 No 1 Down 0
4 6 GE Yes 1000 No 1 Down 0
5 7 GE Yes 1000 No 1 Down 0
6 8 GE Yes 1000 No 1 Down 0
7 9 GE Yes 1000 No 1 Down 0
8 10 GE Yes 1000 No 1 Up 0
9 11 GE Yes 1000 No 1 Up 0
10 12 GE Yes 1000 No 1 Up 0
GE Yes 1000 No 1 Down 0
GE Yes 1000 No 1 Down 0
13 15 GE Yes 1000 No 1 Down 0
GE Yes 1000 No 1 Down 0
15 17 GE Yes 1000 No 1 Down 0
GE Yes 1000 No 1 Up 0
17 19 GE Yes 1000 No 1 Up 0
GE Yes 1000 No 1 Up 0
19 21 GE Yes 1000 No 1 Down 0
GE Yes 1000 No 1 Up 0
GE Yes 1000 No 1 Up 0
GE Yes 1000 No 1 Up 0
23 25 GE Yes 1000 No 1 Down 0
24 26 XE Yes 1000 No 1 Down 0
25 27 XE Yes 1000 No 1 Up 0
26 28 XE Yes 1000 No 1 Up 0
27 29 XE Yes 1000 No 1 Up 0

20. Debug commands: (EOBC/EPC Part 4)
EOBC - BCM56321 port statistics =============================== Port Description InPkts InBytes OutPkts OutBytes ==== =========================== ========= ========= ========== ========== 0 LC LC LC LC LC LC LC LC LC LC LC LC LC LC LC LC FM FM FM FM FM FM SC-SC MTF Secondary-SUP Primary-SUP Local-SC-4-port-sw Other-SC 4-port-sw