1. Version 4.1 Chp. 9: Troubleshooting CCNA Discovery 2

2. Contents 9.1: Troubleshooting approaches9.1: 9.2: Troubleshooting Layer 1 and 29.2 9.3: Troubleshooting Layer 3 IP Addressing9.3: 9.4: Troubleshooting Layer 3 Routing9.4: 9.5: Troubleshooting Layer 4 & Upper Layers9.5:

3. Troubleshooting by Layers One of the most important abilities for a network professional to develop is the ability to efficiently troubleshoot network problems Knowing and Understanding the OSI Layers can help you Troubleshoot Networks ▫Logical networking models separate network functionality into modular layers. ▫Each layer of the OSI or TCP/IP model has specific functions and protocols.

4. Layered Models Network Applications File format, compression and encryption Manages communication sessions between applications Manages end-to-end message delivery, TCP provides reliability IP addressing, Routing of packets between networks Frames, MAC Addressing, Media Access Control Transmission of Bits over Media, Signaling, Encoding

5. Upper Layer Errors The upper layers (5-7) of the OSI model deal with specific application functionality and are generally implemented only in software. Problems in the Upper Layers (5 to 7) are frequently be caused by software configuration errors on clients and servers

6. Layer 3 and 4 Errors The lower layers (1-4) of the OSI model handle data-transport issues. Layers 3 and 4 are generally implemented only in software. Problems in layers 3 and 4 are frequently caused by: 1.Software configuration errors on Clients 2.Router configuration errors 3.Firewall configuration errors Layer 3 Problems are commonly caused by: 1.IP addressing and routing errors

7. Layer 1 and 2 Errors Layer 1 and Layer 2 are implemented in both hardware and software. The Physical Layer is closest to the physical network medium, such as the network cabling, and is responsible for actually placing information on the medium. Most problems in Layer 1 and Layer 2 are caused by: 1.Hardware problems 2.Hardware incompatibilities

8. Technologies by Layer It is important to understand where different Networking Technologies fit into the OSI Layers so you will know where to begin Troubleshooting Layer 1 Media Hubs Signals NICs Layer 2 NICs MAC Address Frames Switches Layer 3 Routers IP Addresses Packets Layer 4 TCP UDP Segments Port Numbers Sockets Upper Layers Applications Telnet FTP HTTP DNS DHCP SMTP SNMP

9. Troubleshooting Approaches There are 3 main troubleshooting approaches when using network models: 1.Top-down 2.Bottom-up 3.Divide-and-conquer Using one of these troubleshooting methods, a troubleshooter can verify all functionality at each layer until the problem is located and isolated.

10. Top-down The Top-down approach starts with the Application Layer and works down. It looks at the problem from the point of view of the user and the application. Is it just one application that is not functioning, or do all applications fail? For example: ▫Can the user access various web pages on the Internet, but not email? ▫Do other workstations have similar issues? Suitable for simpler problems or those where an application error is suspected

11. Bottom-up The Bottom-up approach starts with the Physical Layer and works up. The Physical Layer is concerned with hardware and wire connections. ▫Are cables securely connected? ▫If the equipment has indicator lights, are those lights on or off? Suitable for more complex problems Slow, but solid method

12. Divide-and-Conquer The Divide-and-Conquer begins at one of the middle layers and works up or down from there. For example: a)The troubleshooter begins at the Network Layer by verifying IP configuration information b)Next, they may work up or down the layers:  The functionality of the NIC is checked OR  Port Number Assignments are checked Suitable for more complex problems and when the problem has less precise symptoms

13. Troubleshooting Approaches Problem: A customer reports that they are unable to view web pages from a web server located at the ISP A.Bottom-Up Approach 1.The technician checks the cable connections between the web server and switch 2.The technician checks the lights on the NIC in the web server B.Top-Down Approach: 1.The technician calls the customer to find out which apps are affected 2.The technician verifies that the DNS server has the correct DNS entry and is resolving the name C.Divide-and-Conquer Approach: 1.The technician pings the server and then pings the customer’s switch 2.The technician checks the firewall configuration

14. Troubleshooting Tools It is very difficult to troubleshoot any type of network connectivity issue without a network diagram that depicts the IP addresses, IP routes, and devices, such as firewalls and switches. Logical and physical topologies are extremely useful in troubleshooting.

15. Physical Network Topology A physical network topology shows the physical layout of the devices connected to the network. ▫Knowing how devices are physically connected is necessary for troubleshooting problems at the Physical Layer, such as cabling or hardware problems. Physical network topologies typically include: ▫Device types ▫Models and manufacturers of devices ▫Locations ▫Operating system versions ▫Cable types and identifiers ▫Cabling endpoints

16. Physical Topology Diagram

17. Logical Network Topology A logical network topology shows how data is transferred on the network. ▫Symbols are used to represent network elements such as routers, servers, hubs, hosts, and security devices. Logical network topologies typically include: ▫Device identifiers ▫IP addresses and subnet masks ▫Interface identifiers ▫Routing protocols ▫Static and default routes ▫Data-link protocols ▫WAN technologies

18. Logical Topology Diagram

19. Troubleshooting tools In addition to network diagrams, other tools may be needed to effectively troubleshoot network performance issues and failures. ▫Documentation and baseline tools  To create network diagrams and document hardware ▫Management system tools  To monitor network performance ▫Knowledge bases  Network device vendor knowledge bases are an indispensable sources of information. ▫Protocol analyzers  Decodes the protocol layers in a recorded frame and presents the information in an easy-to-use format.

20. Troubleshooting Hardware Tools Sometimes failures in the lower layers of the OSI model must be identified with hardware troubleshooting tools: ▫Cable testers  Handheld devices designed for testing data communication cabling for broken wires, crossed-over wiring, shorted connections, and improperly paired connections ▫Digital multimeters  Test instruments that measure electrical values of voltage, current, and resistance. ▫Portable network analyzers  For testing network traffic utilization

21. 9.2: Layer 1 and 2 Issues The Physical and the Data Link Layers encompass both hardware and software functions. All network communications rely on the technologies at Layers 1 and 2 in order to function A network technician must be able to quickly isolate and correct problems occurring at these layers To effectively troubleshoot Layer 1 and Layer 2 problems, technicians must be familiar with: ▫Cabling standards ▫Encapsulation ▫Framing

22. Layer 1 Issues Layer 1 problems can result in loss of connectivity or degraded network performance Symptoms of Layer 1 Issues: 1.Loss of connectivity 2.Network congestion 3.High collision count 4.Console error messages Causes of Layer 1 Issues : 1.Bad, Improperly terminated or too long cables 2.Wrong cable type 3.Electrical or wireless Interference See Troubleshooting chart

23. Troubleshoot Layer 1 Issues Most layer 1 issues are caused by Hardware problems and incompatibilities To Troubleshoot Layer 1 Issues: 1.Visually Inspect Cables 2.Check Hardware Configurations 3.Use Show Interfaces to check Interface statistics 4.Monitor Network Performance

24. Layer 2 Issues Symptoms of Layer 2 Issues: 1.Excessive Broadcasts 2.Intermittent Loss of connectivity 3.Excessive Collisions on an Interface 4.Console Error Messages Causes of Layer 2 Issues: 1.Misconfigured Network cards 2.Faulty NIC drivers 3.Encapsulation errors on WAN links 4.Overly large Broadcast Domains

25. Troubleshoot Layer 2 Issues Network analyzers can diagnose Layer 2 issues To Troubleshoot Layer 2 Issues: 1.Check NIC drivers and configuration 2.Use Show Interfaces to check encapsulation 3.Monitor Network Performance with Network analzyers

26. Router Bootup To troubleshoot device hardware failures and boot errors, you must understand the Router Boot Process: 1.Perform the POST and load the bootstrap program. ▫tests the router hardware ▫After POST, the bootstrap program is loaded ▫The Bootstrap program is loaded from ROm 2.Locate and load the Cisco IOS software ▫IOS is located from: flash memory (default), a TFTP server, or router Boots into ROMmon Mode and loaded into RAM ▫The IOS locates the hardware and software components and lists the results on the console terminal 3.Locate and load the startup configuration file or enter setup mode ▫Config file is loaded from: NVRAM, a TFTP server, or the router enters into setup mode ▫This starts routing processes and supplies addresses to interfaces

27. Router Bootup Process

28. Layer 1 & 2 Bootup Errors 1.POST failure ▫If POST fails, turn off the power, unplug the device, remove all interface modules., then reboot 2.Corrupt flash image ▫Some devices can load a boothelper image from ROM memory with limited capabilities. If no boothelper, the device will boot into ROMmon mode 3.Corrupt or missing configuration file ▫Use either a TFTP server to restore a backup file or manually re-configure the device 4.Configuration-Register Setting error ▫An incorrect configuration register setting tells the router the wrong location to look for the Cisco IOS image ▫From ROMmon mode, you can change the config-register setting back 5.Memory error ▫If there is not enough memory to load the image, you can usually boot into ROMmon mode with Ctrl-Break and load an older one 6.Module error ▫Faulty or improperly seated interface modules may not be recognized during the POST and Cisco IOS load. Remove them and reboot

29. Configuration Register The configuration register setting on a router tells it where to look for the IOS file to boot from The normal, default config-register setting is: 0x2102 ▫This indicates a normal Flash boot and tells the router to use the default boot sequence: Flash, TFTP Server, ROMmon mode ▫If the config-register is set to something other than 0x2102, this will cause the router to look somewhere other than FLASH memory for the IOS file, and it will usually not boot properly

30. Viewing the Config-Register Once a router is booted, the ONLY command that can be used to view the config-register is show version ▫Router# show version

31. ROMmon Mode ROMmonIf a router fails to boot a valid IOS file, it will boot up in ROM monitor (ROMmon) mode ROMmon software is a simple command set stored in read only memory (ROM) It can be used to troubleshoot boot errors and recover the router when the IOS is not present Troubleshooting in ROMmon mode: 1.Look in flash memory for a valid IOS image  rommon 1> dir flash: 2.Once an IOS image is found, boot the image manually  rommon 1> boot flash:

32. Troubleshooting Boot Errors In order to troubleshoot Router bootup errors: 1.Observe console messages during the boot sequence 2.Use the show commands to verify operational status When boot problems cause a network outage, use substitution to replace the device with a known good device to restore services to end users

33. Troubleshoot Boot Errors Show version ▫displays the version of the operating system and whether all interface hardware is recognized. Show flash ▫displays the contents of the Flash memory, including the Cisco IOS image file. It also displays the amount of Flash memory currently being used and the amount of memory available. Show ip interfaces brief ▫shows the operational status of the device interfaces and IP addresses assigned. Show running-configuration and show startup- configuration ▫ verify whether all the configuration commands were recognized during boot

34. Layer 1 & 2 Router Interface Errors Router Interface errors are often the first symptom of Layer 1 and Layer 2 cabling or connectivity errors Causes of Layer 1 Interface errors: 1.Cable Errors 2.Physical Interface Errors 3.Hardware Errors Causes of Layer 2 Interface Errors: 1.Encapsulation Error 2.Clock Rate Error 3.Misconfigured IP Address 4.Interface not Enabled

35. Interface Issues The interface cannot detect any signal on the line – due to a bad cable, no cable, interface error, or failed CSU/DSU on a serial interface There is an encapsulation error, a missing clock rate, misconfigured IP Address or the interface is not receiving keepalives The Interface is disabled due to hardware errors The Interface has not been enabled with the no shutdown command Layer 1 Issues Layer 2 Issues

36. Layer 1 & 2 Interface Media Errors 4 different types of Layer 1 and 2 Media Errors that can exist on a router interface ▫Excessive noise ▫Excessive collisions ▫Excessive runts ▫Late collisions

37. Excessive Noise The Interface shows many CRC errors but not many collisions Usually indicates a media or cable error Common causes are electrical interference, loose or damaged connections, or incorrect cabling type

38. Excessive Collisions Can only occur on half-duplex or shared-media Ethernet connections (when hubs in use) Can be caused by Damaged cables

39. Excessive Runts Run Frames are usually caused by Malfunctioning NICs Can also be caused by Damaged Cables

40. Late Collisions A properly designed and configured network should never have late collisions. The most common cause is excessive cable lengths Can also be caused by Duplex mismatches

41. Troubleshoot Interface Errors To troubleshoot Interface Errors: 1.Use show interfaces to look at the interface statistics 2.Use show ip interface brief to check the status of interfaces

42. Show ip interfaces brief

43. Layer 2: LAN Connectivity Errors LAN troubleshooting usually centers around switches, because the majority of LAN users connect to the network via switch ports. Causes of LAN Connectivity Errors: 1.Port hardware failures 2.Cable errors 3.Ports not Enabled 4.Duplex and/or Speed Mismatch

44. LAN Connectivity Troubleshooting Switch Troubleshooting: 1.Observe port LEDs to verify port functionality ▫Red or orange indicates an error 2.Verify cable connections 3.Verify configuration to ensure ports are enabled ▫ Switch# show run ▫ Switch# show interfaces 4.Verify duplex settings on ports ▫ Switch# show interface port status

45. Layer 2: WAN Connectivity Errors Typically, WAN connectivity relies on equipment and media that is owned and managed by a telecommunications service provider (TSP) 2 Most Common WAN Connectivity Issues on Routers: 1.Clock rate errors 2.Mismatched Encapsulation or Timing Troubleshooting WAN connectivity issues: 1.Verify functionality of WAN Interfaces with show interfaces serial 1.Router# show interfaces serial You should also know the type of modem or CSU/DSU being used

46. 9.3: Layer 3 Issues Layer 3 Issues are largely due to poorly designed and configured IP addressing schemes ▫At Layer 3, each packet must be identified with the source and destination addresses of the two end systems. ▫A device must be configured with an IP address to exchange messages using TCP/IP To troubleshoot Layer 3 problems, you must be able to: ▫Determine the range of host addresses that belong to each individual IP network or subnet

47. Subnetting Example The Network 192.168.1.0 /24 needs to be subnetted to create 6 usable Subnets ▫Class: C ▫Default Subnet Mask: 255.255.255.0 ▫# of Bits to borrow : 3  Total Subnets Created: 8  2 X (x = bits borrowed)  Usable Subnets Created: 6  2 X - 2 (x = bits borrowed)  Total Host Addresses per Subnet: 32  2 y (y = host bits left over)  Usable Host Addresses per Subnet: 30  2 y - 2 (y = host bits left over)

48. Subnetting Example Not Usable

49. IP Addressing Scheme Not Usable = Network Address Not Usable=Broadcast Address Network Address

50. Subnet Practice Given the Network Address and Subnet Mask, Define: 1.The Range of Host Addresses 2.The Broadcast Address 3.The Next Network Address 10 24430 10 24430 228 255 255 255 252 252= 11111100 Place value of last bit borrowed = 4 Subnets Increase by a value of 4: Go to Section: 9.3.1.3 in Discovery 2 to Practice this Activity

51. Layer 3 IP Design Issues Common Layer 3 Problems caused by Poor Address Design and Planning 1.Overlapping subnets  When the address range of two separate subnets includes some of the same host or broadcast addresses  The Router IOS does not allow you to configure overlapping subnets on multiple interfaces of the same device  However, it will allow you to configure overlapping subnets on different devices, which will cause problems 2.Misconfigured subnet masks  On Hosts and routers 3.Insufficient addresses available through DHCP  If the subnetting design does not allow for enough available host addresses, then the DHCP server will not be able to assign them

52. Overlapping Subnets

54. Misconfigured Subnet Mask

56. Insufficient addresses available If the IP subnetting scheme does not allow for enough host addresses in each subnet, some hosts will be unable to receive an IP address from the DHCP server. ▫When a host can not receive an address from a DHCP server, it automatically assigns itself an address on the 169.254.0.0 network. ▫A duplicate IP address Error can also occur on a host ▫Use the show ip dhcp binding command from the command prompt to check whether the DHCP server has available addresses

57. Determine # of Hosts Available 62 Go to Section: 9.3.3.2 in Discovery 2 to Practice this Activity

58. 9.4: Layer 3 Routing issues Common Symptoms of Layer 3 Routing Errors: 1.Network failures 2.Poor network performance Routing problems are usually the result of : 1.Static (manual) route entry errors 2.Routing protocol configuration errors 3.Routing Protocol operation errors ▫They can also be caused by failures at the lower layers of the OSI model

59. Interpreting the Routing Table When there is a change in the network status, routes can be lost, or an incorrect route can be installed into the routing table. The primary tool to use when troubleshooting Layer 3 routing problems is examining the Routing Table ▫Router: Show ip route The routing table consists of route entries from the following sources: ▫Directly connected networks ▫Static routes ▫Dynamic routing protocols Any time a routing problem is suspected, use the show ip route command to ensure that all the expected routes are installed in the routing table.

60. Examining Routing Tables

61. Connected Routes Connected Route Problems ▫Directly connected routes are automatically installed in the routing table when an IP address is configured on an interface, and the interface is enabled using the no shutdown command. Causes of Connected Route Problems 1.An Interface has an invalid IP Address 2.An Interface is not enabled Troubleshooting Connected Routes 1.Use the show interfaces or show ip interface brief command to verify that the Interface is UP and has a valid IP address

62. Static and Default Routes Static and Default Route Problems ▫Static and default routes must use either an exit interface on the router or the IP address of the next hop router. Causes of Problems ▫When a static or default route does not appear in the routing table, the problem is most likely a configuration error. 1.If the next hop address is not in the correct IP address range of a directly connected network, then the route will not be vaild Troubleshooting Static and Default Routes 1.Verify that the configuration statements are correct with the show run command 2.Verify that the exit interfaces used by the routes are in an up/up state with the show ip interfaces brief command

63. Dynamic Routes Dynamic Route Problems ▫Dynamic Routes are learned from other routers through the use of routing protocols ▫In order for dynamic routes to be correct, the routing protocol must be operating properly Causes of Dynamic Route Problems ▫A missing route could be caused by a misconfiguration on one or more of the routers on the path to the destination. 1.Missing Network Statements in the Routing Protocol configuration of another router can cause dynamic routes to not be advertised properly Troubleshooting Dynamic Routes 1.Use show ip protocols to check the configuration of routing protocols 2.Use show run to check the routing protocol configuration 3.Use show ip interfaces brief to view interface status 4.Use the debug ip rip command to verify routing updates

64. Layer 3 RIP Issues Issues related specifically to RIP: 1.RIP Version mismatch between routers 2.Incorrect or missing network statements

65. RIP Network Statement When configuring RIP, the network statement does two things: 1.It enables the routing protocol to send and receive updates on all the local interfaces that belong to that network. 2.It includes that network in its routing updates to its neighboring routers. A missing or incorrect network statement results in inaccurate routing updates and can prevent an interface from sending or receiving routing updates.

66. RIP Configuration

67. Troubleshooting RIP issues Tools for troubleshooting dynamic routing issues: 1.Use TCP/IP utilities to test connectivity ▫Ping to test connectivity ▫Traceroute to test connectivity ▫Telnet to verify connectivity and make configuration changes. 2.Use Debug commands to view updates ▫Debug ip rip – to view RIP updated in real-time 3.Use Show commands to verify configuration & routes Show run –view version and network statements Show ip protocols – view rip configuration Show ip interfaces – view interface status Show ip route – view routes Show interfaces – view interface status

68. Debug ip rip

69. 9.5: Troubleshooting Layer 4 Issues Layer 4 is responsible for transporting data packets and specifies the port number used to reach specific applications. Layer 4 network problems can arise at the edge of the network where security technologies are examining and modifying the traffic. Many problems at Layer 4 are caused by misconfigured firewalls, which might deny traffic based on port numbers, even though this traffic should be forwarded.

70. Layer 4 Issues Symptoms of Layer 4 problems: 1.Security problems 2.Intermittent network problems 3.Trouble using some services Causes of Layer 4 Problems 1.Firewall misconfiguration  Example: ftp active connections blocked 2.Incorrect port assignments for applications

71. Well Known Port Numbers 21

72. Upper Layer Issues Most of the upper layer protocols provide user services that are typically used for network management, file transfer, distributed file services, terminal emulation, and email. ▫It can be difficult to isolate problems to the upper layers, especially if the client configuration does not reveal any obvious problems. Symptoms of Upper Layer issues: 1.Slow application performance 2.Application error messages 3.Unable to access application services or web services Common Causes of Upper layer issues: 1.Software configuration errors on clients and servers

73. Troubleshoot upper layer issues In order to determine if an issue exists with the upper layers, you must RULE OUT basic connectivity issues first Use the "divide and conquer" method – start at layer 3 1.Ping default gateway 2.Verify end to end connectivity – ping destination 3.Verify router configuration 4.Verify nat configuration 5.Verify firewall filtering If connectivity is not the issue, but the end device is still not operating as expected, the problem has been isolated to the upper layers.

74. Divide and Conquer Approach – Rule out Connectivity Issues

75. Upper Layer Issues Upper layer problems prevent services from being provided to application programs. A problem at the upper layers can result in unreachable or unusable resources, even when the lower layers are functional. Upper layer issues can exist even with full network connectivity Problems with upper layer functions usually affect just a few applications, perhaps even only one

76. Causes of Upper Layer Issues Most upper layer network problems are caused by Misconfigured client applications 1.If more than one application is affected, a DNS server issue may be the problem ▫Use the nslookup command from the command line to verify DNS functioning on a client 2.A mismatch between the way a client encrypts or compresses the data and the way the server interprets it can cause applications to not function or to function poorly. ▫When a problem occurs on a single host or workstation, it may be a problem with the way the information is being interpreted in the host software.

77. DHCP issues When a host can not obtain an IP address, or assigns itself an IP Address in the 169.254.0.0 network, that is an indication of a DHCP error DHCP Issues are usually caused by: 1.Physical connectivity issue with the host or DHCP server 2.DHCP Server misconfiguration 3.IP Address conflicts between 2 hosts

78. DHCP troubleshooting DHCP troubleshooting: 1.Use the ipconfig /all command 2.Check the physical connectivity of devices 3.Verify that the DHCP server is correctly configured and has available IP addresses to lease 4.Check for any address conflicts  Use the show ip dhcp conflict command 5.Test to ensure that the issue is actually with DHCP  Assign a static IP address to the host and ping the DHCP server

79. NAT issues Usually the first indication that there is a NAT problem is that users cannot reach sites located on the Internet. NAT issues: ▫Incorrectly designated inside & outside interfaces  The inside interface connects to the local network, which uses private IP address space  The outside interface connects to the public network, usually the ISP ▫NAT Pool misconfiguration  The IP address pool and static NAT translation entries must use IP addresses that are on the same local IP network as the outside interface ▫No route to the Internet for translated addresses  If external users must be able to reach specific servers on the internal network, be sure that static NAT translations are configured.

80. NAT Configuration

81. NAT Troubleshooting 1.Verify NAT operation: ▫Router# show ip nat translations 2.Clear NAT translations: ▫ Router# clear ip nat translation * 3.Verify that there is a route to the Internet for the translated addresses ▫Router# traceroute …..

82. Troubleshooting Tools 1.Telnet is an excellent tool to use when troubleshooting upper layer issues ▫Use Telnet to access networking devices remotely ▫The ability to reach devices using Telnet indicates that the lower layer connectivity exists between the devices. ▫However, Telnet is an insecure protocol 2.SSH is a more secure method for remote device access.

83. Troubleshooting Chart LayerSymptomsCausesTroubleshooting Layer 1: Physical Loss of connectivity Network congestion High collision count Console error messages Cable Errors Interference Router Boot Errors Interface Hardware errors Layer 2: Data Link Excessive Broadcasts Loss of connectivity Console Error Message Excessive noise Excessive collisions Excessive runts Late collisions Misconfigured NIC Faulty NIC drivers Large Broadcast Domains Encapsulation errors Clock Rate errors Layer 3: Network Layer 4: Transport Upper Layers