Jim Mallory, Supervisor of Network Operations Saginaw Intermediate School District Troubleshooting Switches, Firewalls, and Wireless

You will be able to use some of the tools built-in to your network infrastructure gear to troubleshoot common problems How you benefit: Increase your ability to solve day-to-day issues on your network or, if the need arises to engage vendor support, you will have a base level of knowledge to help them with the issue I have been troubleshooting Enterprise level networks for 25 years. (Ethernet, Token-Ring, IP, SNA, AppleTalk, IPX, X.25, Frame-Relay, and ATM, HDLC/SDLC). The majority of the time troubleshooting very low level protocol and hardware issues with specialized hardware. Introduction (what I hope to accomplish)

Network Switches ▫ Identifying what port a device is on if you know the IP address  Finding the MAC address  Identifying the device ▫ Finding a wireless MAC address ▫ Troubleshooting fiber connections ▫ Troubleshooting cable issues Firewall Troubleshooting ▫ Setting up a packet capture on a Cisco ASA with ASDM and Wireshark ▫ Graphing CPU, Memory, and Sessions Wireless Troubleshooting ▫ How to check signal strength ▫ Spectrum Analysis ▫ Packet Capture with Wireshark Presentation Outline

We know the IP address ▫ We need the MAC address. This is what the switch tracks ▫ We can get the MAC address two ways  The switches ARP table, this sometimes (rarely) works  HP ProVision: show ip arp  DHCP Server Records: This always works ▫ We now know the MAC address  If I know the building, I will start at the building level core (MDF) switch  If I don’t, I start at the district core  HP ProVision: show mac-address aabbcc-ddeeff  Follow-up with a show lldp info remote-device or show cdp neighbor to determine if the device on the far end is a switch, AP, or the device itself. Some devices don’t support lldp/cdp so YMMV.  If a switch, telnet (ssh) into that switch; if an AP log into the controller / AP  Repeat these steps until you get to an AP or the device itself Network Switches – How to find what port a device is on

Finding the Wireless MAC Address

Current fiber optic drivers have built-in DOM (Diagnostics On Module) capability that can be used to do some basic troubleshooting HP ProVision Command: show interface transceiver or detail Interested in Rx Power ▫ 0mw, 0dBm is bad – not receiving light from far side ▫ Intermittent Connections: Could be that your Rx power is marginal but you will need to know what the minimum amount of power the module needs. ▫ Should be able to find it on your fiber optic vendors support site. ▫ Example: 10Gbase-LR requires -14 dBm Troubleshooting Fiber Links

Some new “Enterprise” class switches have built in Time Domain Reflectometers (TDRs) that can be used for cable diagnostics This is usually disruptive as it breaks Ethernet connectivity while the switch is performing the test Ubquity EdgeSwich OS: cablestatus slot/port Cable Testing

Packet Capture with Cisco ASDM and Wireshark CPU, Memory, Sessions graphing with ASDM Firewall problems sometimes aren’t strictly about “bandwidth”, could be other issues such as the number of active sessions or the number of new sessions being setup per second. Firewall Troubleshooting

Open Source packet capture and decode tool (started life as Ethereal) You really need to understand the low level protocols to get the most out of this tool Runs on Windows, Mac OS X, and other *nixes (Linux) Similar functionality to commercial packet analysis tools at a much, much, lower cost. Also can do wireless sniffing with the right adapter (AirPCAP, next session) Available at (along with some training materials) A Little Bit on “Wireshark”

Signal Strength ▫ How to determine via the controller Spectrum Analysis ▫ Via the Controller or AP (this is usually disruptive) ▫ Dedicated Spectrum Analysis Tools  Metageek Wireless Troubleshooting

Two ways to measure RSSI (Relative Signal Strength Indication) ▫ At least -67 dBm ▫ One Ruckus Engineer stated that with high density 1:1 environments, high 50s may now be the new “ideal” SNR (Signal to Noise Ratio) ▫ Calculated from the difference between the RSSI of the wireless device as measured by the AP/Controller and the noise floor again as measured by the AP/Controller ▫ HP considers the minimum SNR for what it considers a “low” quality signal is 16 dB Signal Strength

Some Wireless systems allow you to put an AP into spectrum analysis mode for troubleshooting This is almost always disruptive as the AP in this mode will no longer service wireless clients Dedicated software / hardware based analysis tools are available ▫ If you manage any kind of substantial wireless install (greater than a dozen APs) or multiple installs (ISD). You need this tool or at least access to someone who does ▫ NOT A SITE SURVEY TOOL ▫ Does require special USB spectrum analysis cards along with your built-in WiFi card to capture BSSID information Spectrum Analysis

Hardware ▫ Microsoft Surface Pro 2 (Windows 8.1 Update 1) ▫ i5 processor ▫ 4GB of RAM ▫ 128GB of SSD ▫ Built-in wireless card (used to collect BSSID info) ▫ Two USB spectrum analyzers  WiSpy DBx 2.4Ghz / 5Ghz combo spectrum analyzer  WiSpy DB2.4x 2.4Ghz dedicated spectrum analyzer ▫ AirPCAP – Packet capture USB radio Software ▫ Metageek Channelyzer Pro (Spectrum Analyzer software) ▫ Metageek EyePA (Basic Packet Capture software) ▫ Wireshark (Advanced Packet Capture software) My Wireless Toolkit

Channelyzer Demo

You may be able to setup your AP to capture packets and send them to Wireshark for further decoding Will not capture radio information (just higher level protocols) To capture packets at the radio level. You will need the AirPCAP software and compatible card. I use the Riverbed card. Wireless Packet Capture and Analysis with Wireshark

Any questions about any of this? You can always me at Wrap-up

– Wireshark download, Wiki, Videos, etc. – Excellent resource for everything – Blog detailing an design methodology. (wouldn’t have to do so much troubleshooting if the things were designed properly) ubleshooting.pdfhttp://forums.juniper.net/jnet/attachments/jnet/Day1Books/5/8/Junos%20Monitoring%20and%20Tro ubleshooting.pdf – Juniper “Day One” book on monitoring and troubleshooting. Requires J-Net Membership (Free) – Illustrates with JunOS commands, but basic concepts are the same. +Version+2-Paperback-8409 Excellent resource to convert the ProVision commands I used to “Ciscospeak”. Also a great resource if you’re an old Cisco guy like me and need to learn HP. The PDF is FREE! Recommended Websites and Downloads