Monitoring & Updating Objectives –to be aware of Unix monitoring utilities Contents –look at utilities for monitoring the system –what to monitor & why.

Slides:



Advertisements
Similar presentations
Unix Systems Performance Tuning Project of COSC 513 Name: Qinghui Mu Instructor: Prof. Anvari.
Advertisements

Drivers and the kernel1-1 Drivers and the kernel UNIX system has three layers: m The hardware m The operating system kernel m The user-level programs Kernel.
Computer Basics Hit List of Items to Talk About ● What and when to use left, right, middle, double and triple click? What and when to use left, right,
Install. Will your hardware work? Most things are compatible - a few are known not to be.
Linux can be generally divided into four major components: 1. KERNEL – OS, ultimate boss The kernel is the core program that runs programs and manages.
Booting the Linux Kernel Dr. Michael L. Collard 1.
Linux+ Guide to Linux Certification, Second Edition
CIS238/DL1 Chapter 15 Rebuilding the Linux Kernel Preparing the Source Code Locating the Source Code Installing the Source Code Read the Documentation.
Linux+ Guide to Linux Certification, Second Edition
Linux+ Guide to Linux Certification, Second Edition Chapter 3 Linux Installation and Usage.
Linux Installation Chapter II. Linux Distributions Pre-packaged, installable Linux Anyone can compile a distribution, have to inculde GPL Available for.
S.Ha.R.K. Workshop28/02/05 S.Ha.R.K. Installation HowTo Tullio Facchinetti University of Pavia - Italy.
Chapter 14 Chapter 14: Server Monitoring and Optimization.
1 Web Server Administration Chapter 3 Installing the Server.
Project 0: Linux & VM Dabbling CS-502, Fall Project 0: Linux & Virtual Machine Dabbling CS-502, Operating Systems.
Lesson 4-Installing Network Operating Systems. Overview Installing and configuring Novell NetWare 6.0. Installing and configuring Windows 2000 Server.
SUSE Linux Enterprise Server Administration (Course 3037) Chapter 1 Introduction to Managing the SUSE Linux Enterprise Server.
Overview Basic functions Features Installation: Windows host and Linux host.
Cambodia-India Entrepreneurship Development Centre - : :.... :-:-
NDT Tools Tutorial: How-To setup your own NDT server Rich Carlson Summer 04 Joint Tech July 19, 2004.
Installing Windows Vista Lesson 2. Skills Matrix Technology SkillObjective DomainObjective # Performing a Clean Installation Set up Windows Vista as the.
Linux+ Guide to Linux Certification Chapter Three Linux Installation and Usage.
Fundamentals of Networking Discovery 1, Chapter 2 Operating Systems.
Guide to Linux Installation and Administration, 2e1 Chapter 3 Installing Linux.
Hands-On Virtual Computing
LOGO Linux Installation. Linux Distribution Including shells, libraries, tools, compiler, servers, applications. Redhat, Fedora, Mandrake, SuSE, Debian,
COSC 4750 Customizing and maintenance. Installing software Redhat/Fedora (and linux in general) has a package installer, called rpm Many programs will.
Ch 6. Performance Rating Windows 7 adjusts itself to match the ability of the hardware –Aero Theme v. Windows Basic –Gaming features –TV recording –Video.
Xen Virtualization Last Update Copyright 2011 Kenneth M. Chipps Ph.D.
Chapter 8: Operating Systems and Utility Programs Catherine Gifford Dan Falgares.
Guide to Linux Installation and Administration, 2e1 Chapter 2 Planning Your System.
Chapter Nine NetWare-Based Networking. Introduction to NetWare In 1983, Novell introduced its NetWare network operating system Versions 3.1 and 3.1—collectively.
09/21/081 Ho Chi Minh city University of Technology Linux kernel R.M. Introduction of building Linux kernel from source.
Monitoring & Updating Objectives –to be aware of Unix monitoring utilities Contents –look at utilities for monitoring the system –what to monitor & why.
Installation Overview Lab#2 1Hanin Abdulrahman. Installing Ubuntu Linux is the process of copying operating system files from a CD, DVD, or USB flash.
1 What is a Kernel The kernel of any operating system is the core of all the system’s software. The only thing more fundamental than the kernel is the.
Guide to Linux Installation and Administration, 2e1 Chapter 10 Managing System Resources.
Linux Administration. Pre-Install Different distributions –Redhat, Caldera, mandrake, SuSE, FreeBSD Redhat Server Install –Check HCL –Significant issues.
Troubleshooting and Performance
Guide To UNIX Using Linux Third Edition Chapter 8: Exploring the UNIX/Linux Utilities.
Cosc 4750 Maintenance & Analysis. Maintenance Contracts Annual cost of 10%-12% of component’s list price. On-site maintenance –usually within hours.
CIS Lesson 5 Lesson 5 New Skills Boot time GRUB edits (review) Changing BIOS boot order on a VM (review) Mounting CD ISO and floppy Image files on.
CHAPTER 2. Overview 1. Pre-Installation Tasks 2. Installing and Configuring Linux 3. X Server 4. Post Installation Configuration and Tasks.
Linux Startup Process Presenter: Dipu Gupta.
1 FreeBSD Installation AFNOG X Cairo, Egypt May 2009 Hervey Allen.
Linux Operations and Administration Chapter Twenty Advanced Linux Administration.
PTA Linux Series Copyright Professional Training Academy, CSIS, University of Limerick, 2006 © Workshop V Files and the File System Part B – File System.
Chapter 8: Installing Linux The Complete Guide To Linux System Administration.
LOGO Linux Installation. Linux Distribution Including shells, libraries, tools, compiler, servers, applications. Redhat, Fedora, Mandrake, SuSE, Debian,
Linux Kernel Programming (LKP). LKP New sub-course New sub-course We will learn together We will learn together Evaluation of this part of course will.
Implementation of Embedded OS
2: Operating Systems Networking for Home & Small Business.
System initialization Unit objectives A.Outline steps necessary to boot a Linux system, configure LILO and GRUB boot loaders, and dual boot Linux with.
1 Setup and Compile Linux Kernel Speaker: Yi-Ji Jheng Date:
Spring 2007 Vmware and Linux kernel COMS W4118 Columbia University.
Linux Introduction Linux was developed in the early 1990’s by Linus Torvald computer science student at the University of Helsinki Linux is distributed.
Rebuilding Linux Kernel Dedicated to penguin lovers everywhere 26 September 20161Rebuilding kernel by Visakh M R.
1 COP 4343 Unix System Administration Unit 1: –Linux OS structure –Distributions –Hardware inventory –Disks and partitions –Installation steps –Boot loader.
Operating System Kernel Compilation
Guide to Linux Installation and Administration, 2e
Seamless Guest OS's and more!
Drivers and the kernel UNIX system has three layers: Kernel
5 SYSTEM SOFTWARE CHAPTER
CONFIGURING HARDWARE DEVICE & START UP PROCESS
Operating System Kernel Compilation
IS3440 Linux Security Unit 7 Securing the Linux Kernel
Accounting Accounting collects information about users and processes
5 SYSTEM SOFTWARE CHAPTER
Operating System Kernel Compilation
Presentation transcript:

Monitoring & Updating Objectives –to be aware of Unix monitoring utilities Contents –look at utilities for monitoring the system –what to monitor & why –monitoring with sar –reconfiguring the kernel –patching kernel –lilo & grub Practical –to examine monitoring output Summary

Why Use Monitoring? Monitoring is useful for benchmarking a system If the system appears slow or sluggish –use monitoring to look for problems –too many processes or users –disk activity concentrated on one disk Users complain that system isn't as good as it used to be –gather performance data for a period of time –compare with benchmarks Analysing monitoring statistics is subjective –there are no hard and fast absolutes –data has to be compared over a period of time

What Do We Monitor? Process (CPU) activity –monitor processor loading –know the most processor-hungry programs Disk access (I/O) activity –understand how your applications use storage devices –observe I/O activity and its distribution –know the most I/O intensive programs Memory utilisation –weigh amount memory against number of users and what they do –minimise swapping and paging activity –monitor kernel tables: don't allow them to overflow Network activity –learn the 'nEtiquette' –don't load the network with activities which should be performed locally All of the above means: KNOW YOUR SYSTEM

System Activity Reporter (sar) Read about it here: Use sar to monitor the system –can look at current system performance –can look at historical data collected via cron jobs in files /var/log/sa/sadd and /proc Install Gentoo package sysstat to get sar Setup cronjobs to retrieve data in /etc/cron.d/sysstat Collects data on cpu, disk, memory, paging etc. –over the next few slides we will see examples of sar reports Compare values over a period of time (days or weeks) –system monitoring is an iterative process –often when you attempt to cure one bottleneck, new problem is introduced Output not immediately meaningful Not provided by all Unix systems –on BSD based systems (and AIX) look for vmstat –vmstat uses different options and reports, but provides the same kind of information # emerge gnuplot # emerge plotutils # emerge sysstat # emerge gnuplot # emerge plotutils # emerge sysstat */10 * * * * root /usr/lib/sa/sa1 -d * * * root /usr/lib/sa/sa2 -A */10 * * * * root /usr/lib/sa/sa1 -d * * * root /usr/lib/sa/sa2 -A

Trouble-shooting With sar (1) Monitoring CPU usage –without any options (just time interval and number of samples) sar will report overall processor utilisation –look for zero idle time (machine overloaded?) –look for high system values: may be hardware problems # sar 10 5 Linux a-default (linux) 10/03/05 10:42:03 %user %system %idle 10:42: :42: :42: :42: :42: Average: # sar 10 5 Linux a-default (linux) 10/03/05 10:42:03 %user %system %idle 10:42: :42: :42: :42: :42: Average:

Trouble-shooting With sar (2) Monitoring memory –w -if swapping, not enough physical memory –B -large values, not enough physical memory Other memory options to use with sar –q if queue very large, too many processes –r look for small free swap space # sar -W :48:12 pswpin/s pswpout/s 20:48: :48: Average: # sar -W :48:12 pswpin/s pswpout/s 20:48: :48: Average: # sar -B :57:22 pgpgin/s pgpgout/s fault/s majflt/s 20:57: :57: Average: # sar -B :57:22 pgpgin/s pgpgout/s fault/s majflt/s 20:57: :57: Average:

Trouble-shooting With iostat Monitor disk performance –d -device utilization report –look for uneven balance across disks and fast service times –p- analyses transfers to/from partitions –look for anomalies and high values # iostat –d 5 1 hda Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn hda Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn hda # iostat –d 5 1 hda Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn hda Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn hda # iostat –d –p 5 1 hda Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn hda hda hda hda # iostat –d –p 5 1 hda Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn hda hda hda hda

Performance Monitoring: Conclusions Learn your system –you must understand all your system resourses and their interaction –you must know who does what, when, where and why Accumulate full statistics when the system is idle –this will give you a common reference point for the system behaviour Monitor performance when the system is busy –this will help you to learn 'normal' values When problem strikes –define it as well as you can –use monitoring tools to identify the cause –correct the problem –continue monitoring Monitoring and tuning processes are iteractive

Other Utilities Various system provide various monitoring utilities These are some to look out for: vmstatvirtual memory and cpu statistics mpstatprocess statistics netstatnetwork statistics Don't forget about ps to look at current processes Systems with X-windows often have good graphical interfaces for display system operation in real time Accounting service –collects information about users and processes –requires vast amount of disk space –data collected through several commands run through cron

Tuning the Kernel The Unix kernel behaviour can be adjusted –tuneable parameters are used to set various kernel parameters –data structure sizes –system thresholds to handle the expected system load Beware - changes will affect system performance –not always in the desired manner! Kernel tuning and rebuilding is system specific All follow five basic steps –backup the existing kernel –tune your parameters –rebuild the kernel –reboot the system –hope it works better than before Kerneltuning increases performance

Preparing for new kernel Howto update/compile current kernel 4 ways of getting the kernel: –From as tar-ball (for generic linux system) –From Gentoo using emerge installing emerge genkernel package then using genkernel tool from command line: genkernel genkernel --bootsplash --no-install --no-clean -- menuconfig all –Using emerge -s sources and manually find a kernel to optimize –Using emerge gentoo-sources to recieve recommended vanilla kernels Get needed packages and development enviroment 1. First check current version of running kernel with uname –r In my case I had: gentoo-r1 2. Install kernel-source: 2a. emerge gentoo-sources Find out ALL about your hardware CPU typeGraphics board and chipsets MemorySCSI controllers Motherboard chipsetsAudio controllers Network cardsOther installed hardware

Kernel source home Carefully read the kernel documentation /usr/srcplaceholder of kernel and rpm sources /usr/src/linuxlink to linux[-kernel-version] /usr/src/linux.vanillalink to linux, needed for some patches /usr/src/linux/DocumentationLinux kernel doc’s (To make a generic custom kernel) Download latest kernel source from: Download needed patches from same site or distributor site. If needed apply patches, if needed only! 3. Backup old kernel source tree somewhere and then cd to /usr/src 4. Unpack kernel as it is under /usr/src/ (emerge does this for you) 5. Make a link called linux.vanilla -> /usr/src/linuxkernel-whatever 6. Apply patches, for example alan cox ac found in people at kernel.org There can be vendor specific patches as well. Patch order is important.

Apply Kernel patches, if needed If no kernel patches, skip step 7, goto the dry run! Apply the patches, kernel source tree For a compressed patch, enter the following command : 7a. cd /usr/src/linux-whatever 7b. zcat /tmp/patch-whatever | patch -p1 2> out Accordingly the command for an uncompressed patch is 7b. patch -p1 out Allways ”dry run” first time on new kernels –Make a dry run compile, helps up, sometimes kernel wont compile if this step is omitted) 8a. cd linux.vanilla 8b. cp.configure configure.orig (Backup your old configuration, if exist) 9. zcat /proc/config.gz >.config (Get the running kernel configuration, good start point) 10. make oldconfig (11. make mrproper) for non rpm kernel sources. 12. make clean 13. make bzImage 14. make modules DONT install anything yet!

Ways to configure the kernel Add our EXTRAVERSION to separate the new kernel from current/old –First edit the /usr/src/linux.vanilla/Makefile.to be able separating your kernel compiling attempts : EXTRAVERSION = -ac-IKEA1 Backup the.config file –First time this file does not exist, backup after you run the make config above cp.config config.old Installing the running kernel.config is a good startpoint in most cases zcat /proc/config.gz >.config make oldconfig Normal ”config steps” and how the ".config" file is made –Current running kernel config options is in /proc/config.gz –If.config does not exist, a generic config will be used with your new kernel –.config is created if it does not exist or modified if it exists when configuring the kernel with one of these make options: Open the kernel configuration menu. make oldconfig-use.config as template without questions make config-classic config for text terminals (much coffe is needed) make menuconfig-modern vt100/ansi terminals make xconfig-X-Windows GUI General config options inside the ”menues” –M=module Y=Yes, static and N=No skip –Modules are loaded into kernel when needed Static is there all the time

Kernel options, while configuring Depending on your #make config# choice –We recommend using the make menuconfig or make xconfig –You will be presented a large menu, where you are supposed to select proper kernel support for various hardware and inner kernel workings. –Static selections will consume more memory but will respond fast –Dynamic Module will consume less memory but will respond slower –Filesystem for booting must be static, or system will not load, they live in initrd You must now walk through all menu’s –To see what you can disable –To see what you need to support as static or module –Add support for at least what you found in ”Find out ALL about your hardware” Now it is time for step 15, enter the kernel configuration: 15. make menuconfig (or some other shown like make xconfig) COMPILING KERNEL IN X-WINDOWS IS A SLOW PROCESS!

Adding support for 3comNIC example Walk to: ”Network device support --->” Hit enter on ”Select” Walk down to ”Ethernet (10 or 100Mbit) --->” Hit enter on ”Select” Walk down to ”3COM cards” Press spacebar to check/open 3COM cards Walk down to ” 3c590/3c900 series (592/595/597) "Vortex/Boomerang" support” press spacebar or m or y to select or n to deselect Now you have added this card to.config file! You can move with arrows to exit and exit up to main menu again. Walk around exactly like this to select any other –This can take long time, read about all devices you add support –It is well worth the trouble, optimized kernels offer from 3% up to 40% preformance increase –Dont forget why we are here, we want to add support for our hardware!

Make the new kernel and install it Exit the configuration section Walk back to main menu and move to exit Press enter Question will come up if you want to save Select YES Make the kernel type: 16. make(make clean or make distclean erases.config and all your hard work in menuconfig) Extraversion will be added to output kernel now linux r3-ac-IKEA1 Backup your old kernel 17a. cp /boot/bzImage /boot/bzImage.old The new kernel image needs to be copied to /boot: 17b. cp /usr/src/linux/i386/boot/bzImage /boot/bzImage OR if you like: cp /usr/src/linux/i386/boot/bzImage /boot/linux r3-ac-IKEA1 Build The Kernel's loadable Modules and install them type: 18. make modules_install

Manual install the new kernel, details Install the Kernel Module A) export INSTALL_PATH=/boot make install B) make modules_install -All modules will copy to /lib/modules/kernelversion Copy The New Kernel To The /boot Partition C) make install -The new kernel is copied to /boot -The new System.map is copied to /boot -Check that they are really copied into /boot -The old kernel should still be there in /boot and the new with your subversion added Depending on how you boot your linux –INITRD initial run demon, a miniroot system on ram disk. –INITRD is used to boot on raid, or special net boots E) Making the initial run demon (minikernel boot, for special hardware needed at boot) cd /boot mkinitrd --omit-raid-modules /boot/initrd-kernelversion-extraversion.img \ kernelversion.extraversion (Sometimes you must "omit" some modules to make it work, dont omit modules nessesary for booting!)

Boot Loader for your kernel Most popular are –GRUB –LILO LInux LOader Bootloader common files –Your kernel need a bootloader, it must be configured to tell where to find the kernel and load the system /boot/boot.bBoostrap sector on disk, keeps LILO or GRUB start /boot/boot.0300Boostrap backup sector backup on disk, keeps LILO or GRUB start /boot/messageBoot message, can be a picture

LILO Add entry for your new kernel: –In top of lilo.conf, so new kernel auto starts, change: default=linux –In /etc/lilo.conf add after last line: image=/boot/linux r3-ac-IKEA1 label=linux initrd=/boot/initrd r3-ac-IKEA1.img read-only root=/dev/sda3 Update the boot.b lilo -v Reboot and test new kernel reboot Lilo is preferred over GRUB in optimated server systems

GRUB Read the GRUB documentation before continue Read the GRUB documentation before continue If you did the dangerous ”make install”, and use GRUB, the new configuration is already added and the old GRUB configuration is named menu.lst.old To manually add/check entry for your new kernel in /boot/menu.1st: In top of menu.1st, change to: default=1 After first ”title” bofore the next ”title” add a new title for the new kernel to auto start: title Linux ( r3-ac-IKEA1 ) root (hd0,0) kernel /boot/vmlinuz r3-ac-IKEA1 ro root=/dev/sda3 initrd /boot/initrd r3-ac-IKEA1.img Reboot and test new kernel reboot GRUB is popular in modern workstations and some high end systems

KERNEL PANIC after reboot Alt 1. Choose another kernel to boot Alt 2. Boot on your Gentoo live cd –A) Let in continue till fully booted, dont install anything! –B) Open a command liner –C) Mount your boot and root partitions # mount /dev/sdb3 /mnt/gentoo # mount /dev/sdb1 /mnt/gentoo/boot –D) Open grub.conf or lilo.conf dependent on your bootloader (here grub): # cd /boot/grub # nano grub.conf –E) Change back kernel (and initrd if you have it) to backed up kernel kernel = /boot/bzImage.old root=/dev/sda3 F) Now save the file Reboot and you should be back again for another try, better luck this time

Kernel afterwork, just a look Add APC UPS driver –We added alan cox patch to our kernel in order to support UPS USB –Now we need to compile the main driver for UPS control Prepare compiling APC UPS driver: cd /usr/src/extras/apcupsd-whatever export CFLAGS="-g -Wall“; export LDFLAGS="-g -Wall“; make distclean The./config is insane !?:./configure --with-upstype=usb --with-upscable=usb --enable-oldnet --enable-usb --enable- powerflute --enable-cgi --with-serial-dev=/dev/usb/hid/hiddev[0-9] --enable-http --enable-net --disable-nls --enable-apcsmart --disable-dumb --enable-pthreads --bindir=/sbin --with- libwrap=yes --enable-snmp --with-cgi-bin=/home/httpd/cgi-bin --with-css-dir=/home/httpd/cgi- bin Now make the driver make make install make clean If you want to backout you need to type make uninstall