CIT 384: Network Administration Troubleshooting Switches CIT 384: Network Administration

CIT 384: Network Administration Topics Troubleshooting Physical Layer Troubleshooting Cisco Discovery Protocol (CDP) Ethernet Troubleshooting CIT 384: Network Administration

CIT 384: Network Administration Troubleshooting Novice Change something (don’t document.) If it’s not fixed, go back to step 1. Serial substitution Replace each component in system with known good one until system works. Methodical Understand the problem. Form hypotheses about possible causes. Focus on most probable cause first. CIT 384: Network Administration

Cisco Troubleshooting Method Start Define the Problem Gather Facts Finished Consider Possibilities Document Results Create an Action Plan Yes Implement the Action Plan Do Problem Symptoms Stop? Observe Results CIT 384: Network Administration No

CIT 384: Network Administration Define the Problem Write description of problem and symptoms. Writing forces you to clarify problem. How does current situation differ from normal? CIT 384: Network Administration

CIT 384: Network Administration Gather Facts Facts include Data from users + admins. Data from protocol analyzer, IOS diagnostics. Answer these questions How often does problem occur? When did problem first occur? What changes were made right before problem started happening? Is the problem reproducible? CIT 384: Network Administration

Consider Possibilities Isolate the problem based on facts Which devices are having problems? At which network layer is the problem? Which protocols are showing problems? Determine possibilities Have you seen this problem before? Have you seen a similar problem before? Use your TCP/IP knowledge and facts to determine what might fail. CIT 384: Network Administration

CIT 384: Network Administration Create an Action Plan Develop plan to test likely causes. Change only one variable at a time. Otherwise you don’t know what fixed it. Divide and conquer Partition problem domain into discrete areas that are physically or logically isolated. Testing outward Does local NIC work? Can you communicate with PC on same subnet? Can you communicate with router? Can you communicate with next hop? ... CIT 384: Network Administration

Implement and Observe Results Follow action plan steps Document which step you’re trying. Document results. Test all fixes you make. Be sure there are no side-effects. Observing results Verify that users see that problem is fixed. CIT 384: Network Administration

CIT 384: Network Administration Document Results Record which plan worked and why. Ensures that you can fix the problem again. If your fix causes new problems later, you know what you did and how to undo it. CIT 384: Network Administration

Troubleshooting Lower Layers Electrical problems. Cable problems. Interface problems. NIC configuration errors. Switch config errors. Application Presentation Session Transport Network Data Link Physical CIT 384: Network Administration

Physical Layer Troubleshooting Check the link lights NICs have transmit, receive, collision LEDs. Switches/routers have many more LEDs. Lights blink time is much longer than actual event (at 10Mbps, 1 byte transferred per us.) Use a cable tester. Check for interface configuration errors. Swap NIC for a known good NIC. CIT 384: Network Administration

CIT 384: Network Administration Cable Testers Wide variety of testers exist. Specialized for different media types (Ethernet, fiber, etc.) More capabilities mean higher prices, starting around $100 to many $1000s. Fluke NetTool Series II image from http://www.bsd-bulgaria.com/i/d0050931_2.jpg CIT 384: Network Administration

CIT 384: Network Administration Cable Tests Continuity Tries to pass a current down the cable. If the current doesn’t flow, cable is bad (short, etc.) Attenuation How much signal is lost over cable length. High values indicate wrong cable type, bad connector, excessive length. Length By timing return of signal (signal on UTP at 0.59c), it determines the length of the cable. Wire map Checks if pins on each end are correctly paired. Near End Cross-Talk (NEXT) Measure how much signal on one wire interferes with other wires. High values can indicate improper termination or wrong cable type. CIT 384: Network Administration

Cisco Discovery Protocol (CDP) Discovers info about neighboring devices Device identifier (hostname) Address list Local interface Port identifier Capabilities list Platform (hardware + software versions) Routers and switches advertise info by multicasting CDP messages. CIT 384: Network Administration

CIT 384: Network Administration CDP Commands Command Description show cdp neighbors List one summary line of info about each neighbor. show cdp neighbors detail List one set of info about each neighbor. show cdp entry name List same info as above command but only for the named neighbor. CIT 384: Network Administration

CIT 384: Network Administration CDP Demo Local switch Lab switch CIT 384: Network Administration

CIT 384: Network Administration Interface Status show interfaces description Lists line and protocol status (up/down) Switch will only forward frames in up/up state. show interfaces status One-line summary of each interface’s status. Status (connected or notconnect) Duplex(auto, a-full, a-half, full, half) Speed(10, 100, 1000, a-) Type(10/100BaseTX, etc.) CIT 384: Network Administration

Interface Status Codes Line Status Protocol Status Interface Status Typical Cause Administratively down Down disabled Interface configured with shutdown command. notconnect No cable; bad or wrong cable; other end is down. Up Not expected. down (err-disabled) err-disabled Port security has disabled interface. connect Interface working CIT 384: Network Administration

CIT 384: Network Administration Interface Status show interfaces name Hardware (MAC address) Speed and duplex settings Flow control ARP Statistics Input rate: bits/sec, packets/sec Output rate: bits/sec, packets/sec Total packets, bytes, broadcasts, collisions Various error types CIT 384: Network Administration

CIT 384: Network Administration Interface Counters FastEthernet0/1 is up, line protocol is up (connected) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 109212347 packets input, 70838129251 bytes, 0 no buffer Received 308656 broadcasts (0 multicasts) 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 watchdog, 305530 multicast, 0 pause input 0 input packets with dribble condition detected 104860540 packets output, 64589349605 bytes, 0 underruns 0 output errors, 0 collisions, 1 interface resets 0 babbles, 0 late collision, 0 deferred CIT 384: Network Administration

Interface Counter Errors Problem Counter Values Common Causes Excessive noise Many input errors, few collisions. Wrong cable category; damaged cables; EMI. Collisions More than 0.1% of frames are collisions. Duplex mismatch (seen on half-duplex side; jabber (NIC ignores Ethernet rules); DoS Late collisions Increasing late collisions Collision domain or cable too long; duplex mismatch. CIT 384: Network Administration

CIT 384: Network Administration Interface Demo Local switch Lab switch CIT 384: Network Administration

CIT 384: Network Administration References James Boney, Cisco IOS in a Nutshell, 2nd edition, O’Reilly, 2005. Cisco, Cisco Connection Documentation, http://www.cisco.com/univercd/home/home.htm Cisco, Internetwork Troubleshooting Handbook, http://www.cisco.com/univercd/cc/td/doc/cisintwk/itg_v1/index.htm, 2006. Wendell Odom, CCNA Official Exam Certification Library, 3rd edition, Cisco Press, 2007. Priscilla Oppenheimer and Joseph Bardwell, Troubleshooting Campus Networks, Addison-Wesley, 2002. W. Richard Stevens, TCP/IP Illustrated, Addison-Wesley, 1994. CIT 384: Network Administration