Chapter 14 Troubleshooting, Performance, and Security

Objectives  Describe and outline good troubleshooting practices  Effectively troubleshoot common hardware- and software-related problems  Monitor system performance using command-line and graphical utilities  Identify and fix common performance problems Linux+ Guide to Linux Certification, 3e2

Objectives (continued)  Describe the different facets of Linux security  Increase the security of a Linux computer  Outline measures and utilities that can be used to detect a Linux security breach Linux+ Guide to Linux Certification, 3e3

Troubleshooting Methodology 4Linux+ Guide to Linux Certification, 3e Figure 14-1: The maintenance cycle

Troubleshooting Methodology (continued)  Monitoring: observing log files and running performance utilities system to identify problems and their causes  Proactive maintenance: minimizing chance of future problems e.g., perform regular system backups Linux+ Guide to Linux Certification, 3e5

Troubleshooting Methodology (continued)  Reactive maintenance: correcting problems when they arise Documenting solutions Developing better proactive maintenance methods  Documentation: system information stored in a log book for future references All maintenance actions should be documented  Troubleshooting procedures: tasks performed when solving system problems Linux+ Guide to Linux Certification, 3e6

Troubleshooting Methodology (continued) Linux+ Guide to Linux Certification, 3e7 Figure 14-2: Common troubleshooting procedures

Troubleshooting Methodology (continued)  Two troubleshooting golden rules: Prioritize problems according to severity ○ Spend reasonable amount of time on each problem given its priority ○ Ask for help if you can’t solve the problem Try to solve the root of the problem ○ Avoid missing underlying cause ○ Justify why a certain solution is successful Linux+ Guide to Linux Certification, 3e8

Resolving Common System Problems  Three categories of problems: Hardware-related Software-related User interface-related Linux+ Guide to Linux Certification, 3e9

Hardware-Related Problems  Often involve improper hardware or software configuration SCSI termination Video card and monitor configuration All hardware is on Hardware Compatibility List POST test alerts ○ Loose hardware connections Problems specific to the type of hardware ○ View output of dmesg command ○ View content of /var/log/boot.log, /var/log/messages Linux+ Guide to Linux Certification, 3e10

Hardware-Related Problems (continued)  Absence of device drivers prevent OS from using associated devices dmesg command: displays the hardware that is detected by the Linux kernel lsusb command: displays a list of USB devices detected by the Linux kernel lspci command: displays a list of PCI devices detected by the Linux kernel Compare outputs of commands to output of lsmod to determine if driver module is missing from kernel Linux+ Guide to Linux Certification, 3e11

Hardware-Related Problems (continued)  Hardware failure can render a device unusable HDDs most common hardware components to fail If HDD containing partitions mounted on noncritical directories fails: ○ Power down computer and replace failed HDD ○ Boot Linux system ○ Use fdisk to create partitions on replaced HDD ○ Use mkfs to create filesystems ○ Restore original data ○ Ensure /etc/fstab has appropriate entries to mount filesystems Linux+ Guide to Linux Certification, 3e12

Hardware-Related Problems (continued)  If HDD containing / filesystem fails: Power down computer and replace failed HDD Reinstall Linux on new HDD Restore original configuration and data files Linux+ Guide to Linux Certification, 3e13

Software-Related Problems: Application-Related Problems  Missing program libraries/files, process restrictions, or conflicting applications  Dependencies: prerequisite shared libraries or packages required for program execution Programs usually check at installation Package files may be removed accidentally Linux+ Guide to Linux Certification, 3e14

Software-Related Problems: Application-Related Problems (continued)  rpm –V command: identify missing files in a package or package dependency  ldd command: display shared libraries used by a program  ldconfig command: updates list of shared library directories (/etc/ld.so.conf) and list of shared libraries (/etc/ld.so.cache) Linux+ Guide to Linux Certification, 3e15

Software-Related Problems: Application-Related Problems (continued)  Too many running processes Solve by killing parent process of zombie processes  Filehandles: connections programs make to files  ulimit command: modify process limit parameters in current shell Can also modify max number of filehandles Linux+ Guide to Linux Certification, 3e16

Software-Related Problems: Application-Related Problems (continued)  /var/log directory: contains most system log files Some are hard linked to /var/log directory  If applications stop functioning due to difficulty gaining resources, restart using SIGHUP Do determine if another process trying to access the same resources attempt to start application in Single User Mode ○ If resource conflict is the cause of the problem, download newer version of application or application fix Linux+ Guide to Linux Certification, 3e17

Software-Related Problems: Operating System-Related Problems  Most software-related problems related to OS X windows, boot loader, and filesystem problems  Problem detecting video card or monitors by the kernel To isolate problem starting X Windows or gdm: ○ View /var/log/Xorg.0.log file ○ Execute xwininfo or xdpyinfo Linux+ Guide to Linux Certification, 3e18

Software-Related Problems: OS-Related Problems (continued)  LILO problems: place “linear” in, remove “compact” from /etc/lilo.conf file  GRUB problems: typically result of missing files in /boot directory  Ensure Linux kernel resides before 1024 th cylinder and lba32 keyword is in configuration file Eliminates BIOS problems with large HDDs Linux+ Guide to Linux Certification, 3e19

Software-Related Problems: OS-Related Problems (continued)  If filesystem on partition mounted to noncritical directory becomes corrupted: Unmount filesystem Run fsck command with –f (full) option If fsck command cannot repair filesystem, use mkfs command to re-create the filesystem Restore filesystem’s original data Linux+ Guide to Linux Certification, 3e20

Software-Related Problems: OS-Related Problems (continued)  If / filesystem is corrupted: Boot from Fedora installation media and enter System Rescue At shell prompt within System Rescue: ○ Use mkfs to recreate the filesystem ○ Use backup utility to restore original data to the re- created / filesystem Exit System Rescue and reboot system  Knoppix Linux and BBC Linux: bootable Linux distributions with many filesystem repair utilities Linux+ Guide to Linux Certification, 3e21

Software-Related Problems: User Interface-Related Problems  Assistive technologies: tools that users can use to modify their desktop experience Assistive Technologies Preference utility within GNOME Desktop Environment ○ Preferred Applications to configure Web browser, multimedia player and terminal applications to be opened automatically ○ Mouse Accessibility to configure speed and click behavior ○ Keyboard Accessibility to configure keyboard related assistive technologies Linux+ Guide to Linux Certification, 3e22

Software-Related Problems: User Interface-Related Problems (continued) Linux+ Guide to Linux Certification, 3e23 Figure 14-3: The Assistive Technologies Preferences utility

Performance Monitoring  Jabbering: failing hardware components send large amounts of information to CPU  Other causes of poor performance: Software monopolizes system resources Too many processes Too many read/write requests to HDD Rogue processes Linux+ Guide to Linux Certification, 3e24

Performance Monitoring (continued)  To solve software performance issues: Remove software from the system Move software to another Linux system Add CPU or otherwise alter hardware  Bus mastering: peripheral components perform tasks normally executed by CPU Linux+ Guide to Linux Certification, 3e25

Performance Monitoring (continued)  To increase performance: Add RAM Upgrade to faster HDDs Disk Striping RAID Keep CD/DVD drives on a separate HDD controller  Run performance utilities on a regular basis Record results in a system log book Eases identification of performance problems  Baseline: measure of normal system activity Linux+ Guide to Linux Certification, 3e26

Monitoring Performance with sysstat Utilities  System Statistics ( sysstat ) package: contains wide range of system monitoring utilities Use yum install sysstat command to install  mpstat (multiple processor statistics) command: displays CPU statistics Used to monitor CPU performance Can specify interval and number of measurements rather than displaying average values %sys should be smaller than %usr and %nice combined Linux+ Guide to Linux Certification, 3e27

Monitoring Performance with sysstat Utilities (continued)  iostat (Input/Output Statistics) command: measures flow of information to and from disk devices Displays CPU statistics similar to mpstat Displays statistics for each disk device on the system Output includes: ○ Transfers per second ○ Number of blocks read and written per second ○ Total number of blocks read and written for the device Linux+ Guide to Linux Certification, 3e28

Monitoring Performance with sysstat Utilities (continued)  sar (System Activity Reporter) command: displays various system statistics taken in the last day Provides more information than mpstat and iostat By default scheduled to run every 10 minutes ○ Output logged to a file in /var/log/sa directory -f option: View statistics from a specific file Can be used to take current system measurements Linux+ Guide to Linux Certification, 3e29

Monitoring Performance with sysstat Utilities (continued)  Additional sar options : -q option : Displays processor queue statistics ○ runq -sz value: Number of processes waiting for execution on processor run queue ○ plist -sz value: Indicates number of processes currently running ○ ldavg values: Represent average CPU load -W option: Displays number of pages sent to and taken from swap partition ○ Large number causes slower performance ○ Add RAM to resolve Linux+ Guide to Linux Certification, 3e30

Monitoring Performance with sysstat Utilities (continued) Linux+ Guide to Linux Certification, 3e31 Table 14-1: Common options to the sar command

Other Performance Monitoring Utilities  top command: displays CPU statistics, swap usage, memory usage and average CPU load  free command: displays total amounts of physical and swap memory and their utilizations Can be used to indicate whether more physical memory is required  vmstat command: displays memory, CPU, and swap statistics Can be used to indicate whether more physical memory is required Linux+ Guide to Linux Certification, 3e32

Security  Linux systems typically made available across networks such as the Internet More prone to security loopholes and attacks  Should improve local and network security  Understand how to detect intruders who breach the system Linux+ Guide to Linux Certification, 3e33

Securing the Local Computer  Limit access to physical computer itself Prevent malicious users from accessing files by directly booting the computer with their own device  Server closet: secured room to store servers  Remove floppy, CD, and DVD drives from workstations  Ensure BIOS prevents booting from USB ports Linux+ Guide to Linux Certification, 3e34

Securing the Local Computer (continued)  Ensure BIOS password is set  Set boot loader password in LILO or GRUB configuration file Prevents intruder from interacting with boot loader  Limit access to graphical desktops and shells Exit command-line shell before leaving computer ○ nohup command: prevents background processes from being killed when parent shell is killed or exited Lock screen using GNOME or KDE Linux+ Guide to Linux Certification, 3e35

Securing the Local Computer (continued)  Minimize root user’s time logged in  su (switch user) command: switch current user account to another Used to switch between root user and regular user  sudo command: perform commands as another user if you have the rights to do that listed in /etc/sudoers file Linux+ Guide to Linux Certification, 3e36

Protecting Against Network Attacks  Always a possibility that hackers can manipulate a network service by interacting with it in unusual ways  Buffer overrun: program information for a network service altered in memory Linux+ Guide to Linux Certification, 3e37

Network Security Essentials  Minimize number of running network services  nmap (network mapper) command: scans ports on network computers User can determine what network services are running  Ensure that services that are not needed are not automatically started when entering the runlevel Linux+ Guide to Linux Certification, 3e38

Network Security Essentials (continued)  Ensure network service daemons for essential services not run as root user when possible  Ensure that shell listed in /etc/passwd for daemons is set to /sbin/nologin Hacker will not be able to get BASH shell  New network service versions usually include fixes for known network attacks Keep network services up-to-date Linux+ Guide to Linux Certification, 3e39

Network Security Essentials (continued)  TCP wrapper: program that can start a network daemon Checks /etc/hosts.allow and /etc/hosts.deny files before starting a network daemon  Examine permissions for files and directories associated with system and network services Linux+ Guide to Linux Certification, 3e40

Configuring a Firewall  netfilter/iptables: used to configure a firewall Discard network packets according to chains of rules  Chains: specify general type of network traffic to apply rules to  Rules: match network traffic to be allowed or dropped  Three chain types: INPUT: incoming packets FORWARD: packets passing through computer OUTPUT chain: outgoing packets Linux+ Guide to Linux Certification, 3e41

Configuring a Firewall (continued)  iptables command: creates rules for a chain Can be based on source IP, destination IP, protocol used, or packet status  Stateful packet filter: Remembers traffic allowed in an existing session and adjust rules appropriately  Easier to use graphical utility to configure firewalls Linux+ Guide to Linux Certification, 3e42

Linux+ Guide to Linux Certification, 3e43 Table 14-2: Common iptables options

Configuring a Firewall (continued) Linux+ Guide to Linux Certification, 3e44 Figure 14-4: The Firewall Configuration utility

Configuring SELinux  SELinux: Security Enhanced Linux By default, configured and enabled during Fedora installation Series of kernel patches and utilities created by NSA ○ Enforces role-based security  To enable, edit /etc/selinux/config file  Configure SELINUXTYPE option  Reboot and relabel the system  sestatus command: view current SELinux status Linux+ Guide to Linux Certification, 3e45

Using Encryption to Protect Network Data  Use encryption algorithms to protect data before it is transmitted on a network  Asymmetric encryption: uses a pair of keys uniquely generated on each system Public key: freely distributed Private key: used only by the system, never distributed Can be used to authenticate messages  Digital signature: message that has been encrypted using a private key Linux+ Guide to Linux Certification, 3e46

Working with SSH  By default, SSH uses RSA to encrypt data and DSA to digitally sign data  System wide RSA and DSA key pairs are generated the first time SSH daemon is started Tunneling: enclosing network traffic within encrypted SSH packets  SSH identity: used to automatically authenticate to other computers using digital signatures  Manage keys using Password and Encryption Keys utility Linux+ Guide to Linux Certification, 3e47

Working with SSH (continued) Linux+ Guide to Linux Certification, 3e48 Figure 14-5: The Passwords and Encryption Keys utility

Working with GPG  Open source version of PGP  Each user has a key pair used for encryption and authentication Authentication uses trust model  Typically uses RSA and DSA key pairs for asymmetric encryption and digital signing  Can manage GPG keys and encrypt data using: gpg command Graphical utility such as Passwords and Encryption Keys utility Linux+ Guide to Linux Certification, 3e49

Detecting Intrusion  Log files can contain information or irregularities indicating an intrusion Review log files in /var/log associated with network services At minimum, review system log files associated with authentication  Pluggable Authentication Module (PAM): handles authentication requests by network applications Log file in /var/log/secure Linux+ Guide to Linux Certification, 3e50

Detecting Intrusion (continued)  Check /var/log/wtmp log file Lists users who receive BASH shells Use who command to view the file  lsof (list open files) command: lists files that are currently being edited  Periodically search for files that have SUID bit set  Tripwire: monitors important files and directories  Intrusion Detection System (IDS): program used to detect intruders on a Linux system Linux+ Guide to Linux Certification, 3e51

Detecting Intrusion (continued) Linux+ Guide to Linux Certification, 3e52 Table 14-3: Common Linux Intrusion Detection Systems

Summary  Administrators monitor the system, perform proactive/reactive maintenance, and document system information  Common troubleshooting procedures involve: Isolating and determining the cause of system problems and implementing and testing solutions that can be documented for future use  Invalid hardware settings, absence of device drivers, and hard disk failure are common hardware-related problems Linux+ Guide to Linux Certification, 3e53

Summary (continued)  Software-related problems can be application-related or OS-related  Users can use assistive technologies to modify their desktop experience  System performance is affected by a variety of hardware and software factors Using performance monitoring utilities to create a baseline is helpful for diagnosing future performance problems Linux+ Guide to Linux Certification, 3e54

Summary (continued)  Securing a Linux computer involves: Improving local and network security and monitoring to detect intruders  Greatly improve local security by: Restricting access to the computer and using root account only when required via su and sudo commands Linux+ Guide to Linux Certification, 3e55

Summary (continued)  Reduce chance of network attacks by: Reducing number of network services, implementing firewalls, SELinux, service updates, encryption, and TCP wrappers, and restricting services from running as root user and permissions on key files  Analyzing log files and key system files and running IDS applications can be used to detect intruders Linux+ Guide to Linux Certification, 3e56