CIS 191 – Lesson 11 Lesson Module Status Slides – draft Properties - done Flashcards - 1 st minute quiz – done Web Calendar summary – done Web book pages – Commands – done Howtos – na Skills pacing - done Lab – done Depot – NA VMs - NA

CIS Lesson 11 Quiz Please take out a blank piece of paper, switch off your monitor, close your books, put away your notes and answer these questions: What are three ways CUPS can be configured? What URL is used for the CUPS web based management tool? What two commands can used to delete print jobs?

CIS 191 – Lesson 11 ObjectivesAgenda Choose appropriate backup strategies Perform full, incremental and differential backups Restore a system from one or more backups Describe the issues around pathnames and symbolic links with respect to backups Quiz Questions from last week Backup and restore overview tar  compression  cpio  dump Wrap up  = hands on exercise for topic Backup and Restore

CIS 191B Lesson 8 System Admin User Mgmt Lesson 9 File System Management Holiday Lesson 12 System Monitoring Lesson 11 Backup and Restore Lesson 10 Printing Lesson 13 The kernel Review Final Course Skills Pacing

CIS Lesson 11 Last weeks Assignment Questions on previous material or assignments? mingetty -> login -> bash? windows print share? CUPS device URI?

CIS Lesson 11 Backup and Restore

CIS Lesson 11 Backup and Restore The Backup Process Losing data Backup Policies Devices Problems with Backup Programs Restoring Data File Permissions, Types, and Structures Defragmentation Protecting the Backup Media

CIS Lesson 11 Backup and Restore Losing data How to lose data Software bugs Pilot errors Hardware failures Hackers Many more ways … It will happen!

CIS Lesson 11 Backup and Restore Backup Policies Backup Policies (from textbook 9.1) Perform LAN backups from one system Label the media Choose reasonable backup interval Factor in modification frequency of files/filesystems Make daily dumps fit on one piece of media Make filesystems smaller than the backup media Keep media off-site Secure off-site media Limit activity during backups Verify your media Pay attention to media life Design your storage with backups in mind Prepare for the worst

CIS Lesson 11 Backup and Restore Backup devices Tape Devices ‒ Problems with tapes ‒ The buffer utility ‒ Tapes and random file access Writable CD and DVD ‒ CDs 650MB ($ each) ‒ DVDs 4.7 GB ($ each) ‒ DVDs (DL) 8.5 GB ($2-4 each) ‒ Blu-ray 25 GB ($8-12 each) ‒ 1-5 years to 100's of years (if stored properly) Other Removable Media Devices USB/Firewire Solid state, external drives Hard drives and RAID Arrays

CIS Lesson 11 Backup and Restore Types of Data System Data ‒ Basic information to set and run the system Application Software ‒ Best to reinstall from source installation disks Databases ‒ Most important data types User Data ‒ Includes site work User Data and Home Directories ‒ Includes , files from home directory for personal use

CIS Lesson 11 Backup and Restore Methodologies Epoch ‒ Arbitrary time interval that denotes a beginning Incremental ‒ All files that have been added or changed since epoch Differential ‒ Similar but easier than incremental, only uses two tapes File ghosting ‒ Anomaly of the backup and restoration process where deleted files turn up again!

CIS Lesson 11 Backup and Restore Archive Utilities dump/restore ‒ Oldest and most trusted backup utility ‒ Backs up filesystems, not files (block-based and fast) ‒ Multiple levels of incremental backups tar ‒ Used to archive or backup files, directories, hard disks cpio ‒ Copy input/output ‒ can use output of find command for wide range of file selection critera (by type, user, modification time, etc)

CIS Lesson 11 Backup and Restore Compression Utilities compress ‒ Traditional compression utility ‒ Not on CentOS 5 zip ‒ Used for compressing, archiving files into single archive ‒ compatible with WinZip gzip ‒ Used for compressing, uncompressing files one at a time bzip2 ‒ New, slower, but better compression device

CIS Lesson 11 Backup and Restore Backup Software Packages AMANDA - Kbackup - UNiBack - Taper - Arkeia - Rsync - Dar & KDar - Partimage (like Ghost) -

CIS Lesson 11 Backup and Restore Backup Software Packages AMANDA  Advanced Maryland Automatic Network Disk Archiver  Able to split backups across media  No GUI, command line only  Can backup other servers over the network  Uses native dump and tar utilities  Amanda also can use SAMBA top backup Windows hosts 

CIS Lesson 11 Backup and Restore Backup Software Packages Kbackup  Can use PGP to encrypt  Can use gzip or compress to compress archives  full or incremental backups  access remote devices for LAN backups 

CIS Lesson 11 tar

CIS Lesson 11 tar command tar file(s) c tvf x create table of contents (view) extract Note: The full path to each file is stored in the archive and these paths are used when restoring files tarfile archive file files to backup options (no – needed) v = verbose, double v (vv) provides more information

CIS Lesson 11 tar command Single file example Backup a web page file Oops! – file gets deleted View backed up file Restore the file

CIS Lesson 11 tar command Single file example create verbose tarfile to use file to back up (requires full path) view (table of contents) verbose tarfile to use extract verbose tarfile to use file in archive to restore (requires full path)

CIS Lesson 11 tar command Pay attention to the path stored in the tarfile Creating the tarfile from another directory using absolute path ~]# tar cvf sample2.tar /opt/lampp/htdocs/sample.html tar: Removing leading `/' from member names /opt/lampp/htdocs/sample.html ~]# tar tvf sample2.tar -rw-r--r-- root/root :13:10 opt/lampp/htdocs/sample.html ~]# ~]# tar cvf sample2.tar../opt/lampp/htdocs/sample.html tar: Removing leading `../' from member names../opt/lampp/htdocs/sample.html ~]# tar tvf sample2.tar -rw-r--r-- root/root :13:10 opt/lampp/htdocs/sample.html ~]# Note, the leading / gets stripped This path is used when restoring files Creating the tarfile from another directory using relative path path Note, the leading../ gets stripped This path is used when restoring files

CIS Lesson 11 tar command -C option tar tarfile files(s) The –C option can be used to set the starting directory for file to be restored to. Add a – to the other options as well

CIS Lesson 11 tar command -C option Change to the directory where the backup was made Or use the –C option to restore from another directory Find file to restore in archive

CIS Lesson 11 Class Exercise tar command 1.Change to the /opt/lampp/htdocs directory. 2.Backup sample.html using: tar cvf /root/sample.tar sample.html 3.Verify your tarfile contains sample.html What is the path to sample.html in the tarfile? [Table 1-4] 4.Delete then practice restoring sample.html:  To its original location Where did you have to be to do this? [Table 9-12]  To the /tmp directory Where did you have to be to do this? [Table 17-20] 5.Now use the tar man page to check out the –C directory option.  Delete the sample.html in the /tmp directory  Restore sample.html from /root to the /tmp directory with the –C option. What command did you use [Table 5-9]?

CIS Lesson 11 tar command Directory example Backup up files (including hidden and sub-directories) starting from "here" Delete some files Verify they were backed up Restore the files that were deleted Note how tar saves full path for each file backed up

CIS Lesson 11 tar command Directory example./xampp/contrib/xmlrss.php./xampp/contrib/mingstats.html./xampp/contrib/sql2xml.php./xampp/contrib/sqlite.pl./xampp/contrib/BabelSans-B.fdb./xampp/contrib/oracle.pl./xampp/contrib/interbase.php./xampp/contrib/postgresql.php./xampp/manuals.php./xampp/cgi.cgi./xampp/cds.php./xampp/webalizer.php./xampp/biorhythm.php./xampp/AnkeCalligraph.TTF./xampp/head.php./xampp/guestbook-zh.pl./xampp/start.php./xampp/AnkeCalligraph.fdb./xampp/mingswf.php./xampp/mysql.php./xampp/guestbook-nl.pl./xampp/navi.php./xampp/lang.php./xampp/security.php./xampp/lang/./xampp/lang/de.php./xampp/lang/it.php./xampp/lang/fr.php./xampp/lang/es.php./xampp/lang/pt_br.php./xampp/lang/nl.php./xampp/lang/zh.php./xampp/lang/no.php./xampp/lang/pl.php./xampp/lang/languages.php./xampp/lang/jp.php./xampp/splash-logo.php./xampp/langsettings.php./xampp/.version./xampp/status.php./xampp/phonebook.php./xampp/img/./xampp/img/rb.gif./xampp/img/status3.gif./xampp/img/logo-small.gif./xampp/img/logo-small.jpg./xampp/img/new.png./xampp/img/signature-kay.gif./xampp/img/lt.gif./xampp/img/benji-500x420.jpg./xampp/img/status4.gif./xampp/img/rt.gif./xampp/img/strichel.gif./xampp/img/head-linux.gif./xampp/img/head-xampp.gif./xampp/img/xampp-logo.jpg./xampp/img/head-fuer.gif./xampp/img/head-for.gif./xampp/img/head-solaris.gif./xampp/img/status2.gif./xampp/img/status1.gif./xampp/img/head-windows.gif./xampp/img/logo-big.gif./xampp/img/signature-oswald.gif./xampp/img/status5.gif./xampp/img/blank.gif./xampp/img/lb.gif./xampp/img/xampp-logo-new.gif./xampp/guestbook-es.pl./xampp/guestbook-de.pl./xampp/phpinfo.php./xampp/splash.php./xampp/guestbook-pt_br.pl./xampp/guestbook.dat./xampp/charset.php./xampp/test.php./xampp/softwarelist.inc./xampp/guestbook-it.pl./xampp/cds-fpdf.php./xampp/ssi.inc./xampp/php.php./xampp/sqlite/./xampp/sqlite/cdcol./xampp/sqlite/phonebook.sqlite./xampp/sqlite/.htaccess./xampp/xampp.css./xampp/ming.php./xampp/showcode.php./xampp/guestbook-fr.pl./xampp/guestbook-en.pl./xampp/iart.php./.hidden./favicon.ico./index.html create verbose tarfile to use files to back up (starting point)

CIS Lesson 11 tar command Directory example view (table of contents) Files to find in archive tarfile verbose extract Files to find to extract tarfile verbose

CIS Lesson 11 Class Exercise tar command 1.Change to the /opt/lampp/ directory. Note the htdocs and backup directories. 2.Backup the htdocs directory (including sub-directories) to htdocs tar the backup directory. What command did you use? [Table 21-24] 3.Verify your tarfile contains xampp/lang/jp.php What command did you use? From where? [Table 1-4] 4.Delete and then restore xampp/lang/jp.php from the backup directory What command did you use? [Table 9-12]

CIS Lesson 11 tar command Size can differ between tarfile and backed up file Tarfiles  Tarfile can be smaller than backed up file as it only saves the date, not unused portion of data blocks  Tarfile can be larger if backed up file is a sparse file  Additional empty blocks can be added to tar version

CIS Lesson 11 Tarfiles  Additional empty blocks can be added to tar version tar command du size can differ between tarfile and backed up file du report shows more disk space used than original for copy and tar extracts original file ls command show same size for original, copy and tar extracts Note: the –s option is to handle sparse files (discussed later)

CIS Lesson 11 Tarfiles  Additional empty blocks can be added to tar version ~]# stat messages /var/log/messages sparse/messages nosparse/messages File: `messages' Size: Blocks: 272 IO Block: 4096 regular file File: `/var/log/messages' Size: Blocks: 254 IO Block: 4096 regular file File: `sparse/messages' Size: Blocks: 272 IO Block: 4096 regular file File: `nosparse/messages' Size: Blocks: 272 IO Block: 4096 regular file ~]# tar command Size can differ between tarfile and backed up file stat command shows copy and files extracted from tarfile have additional blocks which explains why file size is the same but disk usage is higher.

CIS Lesson 11 tar command sparse files Sparse Files  Null blocks (containing only zeroes) are not stored on the disk  Can be problematic if utilities don't recognize them ~]# cd /var/log log]# ls -l lastlog -rw-r--r-- 1 root root Nov 22 05:18 lastlog log]# du -h lastlog 16K lastlog Note the reduced size of the file actually stored on the disk using du –h command

CIS Lesson 11 Sparse Files  tar command without using the sparse option (-S) for sparse file tar command sparse files The tarfile has expanded all the null blocks increasing disk space usage ~]# cd /var/log log]# tar cvf /root/lastlog.tar lastlog lastlog log]# cd /root ~]# ls -l lastlog.tar -rw-r--r-- 1 root root Nov 22 05:19 lastlog.tar ~]# du -h lastlog.tar 248K lastlog.tar ~]# tar xvf lastlog.tar lastlog ~]# ls -l lastlog -rw-r--r-- 1 root root Nov 22 05:18 lastlog ~]# du -h lastlog 240K lastlog

CIS Lesson 11 Sparse Files  tar command using the sparse option (-S) with sparse file tar command sparse files ~]# cd /var/log log]# tar cSvf /root/lastlog2.tar lastlog lastlog log]# cd /root ~]# tar xvf lastlog2.tar lastlog ~]# ls -l lastlog -rw-r--r-- 1 root root Nov 22 05:18 lastlog ~]# du -h lastlog 20K lastlog ~]# Using the -S option, the tar command handles the sparse file efficiently

CIS Lesson 11 tar command compression tar file(s) -C dir c t vf x tarfile The z option uses gzip compresssion and the j option uses bzip2 compression bzip2 is slower, but compresses more gzip is faster, but compreses less

CIS Lesson 11 tar command compression Backing up htdocs with no compression

CIS Lesson 11 tar command compression Post-compression using gzip of archive

CIS Lesson 11 tar command compression Post-compression using bzip2 of archive

CIS Lesson 11 tar command compression Using gzip compression option

CIS Lesson 11 tar command compression Using the bzip2 compression option

CIS Lesson 11 tar command compression backup]# ls -Slr total rw-r--r-- 1 root root Nov 22 13:14 htdocs.tar.bz2 -rw-r--r-- 1 root root Nov 22 13:18 htdocs2.tar.bz2 -rw-r--r-- 1 root root Nov 22 13:13 htdocs.tar.gz -rw-r--r-- 1 root root Nov 22 13:15 htdocs2.tar.gz -rw-r--r-- 1 root root Nov 22 13:13 htdocs.tar backup]# Conclusions: compression option slightly better than two steps bzip2 compresses more that gzip 2 steps

CIS Lesson 11 tar command compression FYI, not specifying the z option still works when restoring a file

CIS Lesson 11 tar command compression FYI, not specifying the j option still works when restoring a file

CIS Lesson 11 Class Exercise tar command 1.Change to the /opt/lampp/htdocs directory. 2.Make four backups of htdocs into the /root directory using both compression options and using post compression with gzip and bzip2. a)-z option b)-j option c)2 steps with gzip command d)2 steps with bzip2 command 3.Verify your backup by deleting, restoring and viewing sample.html for each backup above What commands did you use for 2a and 3? [Table 17-20] What commands did you use for 2b and 3? [Table 5-8] What commands did you use for 2c and 3? [Table 13-16] What commands did you use for 2d and 3? [Table 21-24]

CIS Lesson 11 tar command compression tar file(s) c t vf x Backups can be made to devices as well zjzj

CIS Lesson 11 tar command using devices bin]# fdisk /dev/sda Command (m for help): n First cylinder ( , default 582): Using default value 582 Last cylinder or +size or +sizeM or +sizeK ( , default 652): +500M Command (m for help): p Disk /dev/sda: 5368 MB, bytes 255 heads, 63 sectors/track, 652 cylinders Units = cylinders of * 512 = bytes Device Boot Start End Blocks Id System /dev/sda1 * Linux /dev/sda Linux swap / Solaris /dev/sda Linux /dev/sda Extended /dev/sda Linux /dev/sda Linux /dev/sda Linux /dev/sda Linux Command (m for help): Make a 500 MB partition

CIS Lesson 11 tar command using devices Use partprobe so kernel will use new partition table Command (m for help): x Expert command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot. Syncing disks. bin]# partprobe bin]#

CIS Lesson 11 tar command using devices You don't need to backup socket files. They are created automatically by the kernel when needed Change to the directory to back up. Note /dev/sda8 is not mounted

CIS Lesson 11 tar command using devices Restoring deleted file from backup archive on a device Finding file in archive Oops! …. file gets deleted Viewing restored file

CIS Lesson 11 tar command Examples Don't backup and restore /proc (this is real time kernel data) !! Perform full backup of entire file tree -p, --same-permissions, --preserve-permissions Keeps permissions of extracted files the same as the originals.

CIS Lesson 11 tar command Examples Backup files in /opt after May 5, 1993 This allows differential and incremental backups to be done. Both types are based on a date. Either the last backup (incremental) or the last full backup (differential). To do differential or incremental backups you will need a mechanism to track the dates or use a higher level backup program that makes use of tar.

CIS Lesson 11 Class Exercise tar command 1.Create a 500 MB partition on /dev/sda 2.Backup the entire /opt directory to this partition using tar What commands did you use? [Table 1-4] 3.Verify your backup by deleting, restoring and viewing sample.html for each backup above What commands did you use? [Table 9-12] 4.Delete the 500 MB partition

CIS Lesson 11 cpio

CIS Lesson 11 cpio command find files by attribute Note: -mount = don’t descend directories on other filesystems. -mtime n = True if the file's data was modified n days ago. Sometimes the files we want to back up may or may not be in one directory

CIS Lesson 11 cpio command Backup all files modified in the last week verbose output 5K blocks (good practice)

CIS Lesson 11 cpio command opt]# cpio -vitB < /tmp/level-7.bak cpio: Removing leading `/' from member names -rwxr--r-- 1 root root 295 Nov 15 19:17 root/bin/undoLab06 cpio: Removing leading `/' from member names drwxr-xr-x 2 root root 0 Nov 15 19:19 root/bin/backups cpio: Removing leading `/' from member names -rw-r--r-- 1 root root 0 Nov 15 19:17 root/bin/backups/motd cpio: Removing leading `/' from member names -rw-r--r-- 1 root root 124 Nov 15 19:17 root/bin/backups/.bashrc cpio: Removing leading `/' from member names -rw-r--r-- 1 root root 176 Nov 15 19:17 root/bin/backups/.bash_profile cpio: Removing leading `/' from member names -rw-r--r-- 1 root root 21 Nov 15 19:17 root/bin/backups/issue cpio: Removing leading `/' from member names -rwxr--r-- 1 root root 1488 Nov 15 19:17 root/bin/doLab06 1 block opt]# Verify files backed up inputtable of contents

CIS Lesson 11 cpio command Selective file restore Retain previous file modification time Verify file was restored

CIS Lesson 11 Class Exercise cpio command 1.Prove to yourself that you can create backup of all files modified in the last 7 days using cpio. Put your backup in /tmp. What command did you use? [Table 17-20] 2.Verify the contents of your backup. What command did you use? [Table 5-8] 3.Delete and restore one of the files you backup. What command did you use? [Table 13-16]

CIS Lesson 11 dump

CIS Lesson 11 dump command dump uf files-to-backup file device The dump level defines the different levels of incremental backups. Each backup level has a date associated to indicate when the last backup at that level took place. Level 0 is a full backup; level 1 backs up files that changed since the last level 0 backup; level 2 backs up files that changed since the last level 1 backup, and so forth. dump level Update the file /etc/dumpdates Backup device or file

CIS Lesson 11 dump command dump levelupdate the file /etc/dumpdates backup device or file what to backup

CIS Lesson 11 dump command table of contents create a list of all files in the dump record of the last dump

CIS Lesson 11 dump command We are going to delete (really just move it) a file that is a symbolic link in the /etc directory

CIS Lesson 11 dump command run restore in interactive mode Use ls and cd to navigate to file we want to restore Add it to the list for extraction

CIS Lesson 11 dump command * shows file marked for extraction extract the file quit the program

CIS Lesson 11 dump command /]# ls -li /etc/grub.conf /tmp/grub.conf lrwxrwxrwx 1 root root 22 Nov 23 12:06 /etc/grub.conf ->../boot/grub/grub.conf lrwxrwxrwx 1 root root 22 Oct 12 03:28 /tmp/grub.conf ->../boot/grub/grub.conf

CIS Lesson 11 Opinion Section From: Linus Torvalds To: Neil Conway Subject: Re: [PATCH] SMP race in ext2 - metadata corruption. Date: Fri, 27 Apr :59: (PDT) Cc: Kernel Mailing List [ linux-kernel added back as a cc ] On Fri, 27 Apr 2001, Neil Conway wrote: > > I'm surprised that dump is deprecated (by you at least ;-)). What to use instead for backups on machines that can't umount disks regularly? Note that dump simply won't work reliably at all even in 2.4.x: the buffer cache and the page cache (where all the actual data is) are not coherent. This is only going to get even worse in 2.5.x, when the directories are moved into the page cache as well.Mailingdata So anybody who depends on "dump" getting backups right is already playing russian rulette with their backups. It's not at all guaranteed to get the right results - you may end up having stale data in the buffer cache thatends up being "backed up". Dump was a stupid program in the first place. Leave it behind. > I've always thought "tar" was a bit undesirable (updates atimes or > ctimes for example). Right now, the cpio/tar/xxx solutions are definitely the best ones, and will work on multiple filesystems (another limitation of "dump"). Whatever problems they have, they are still better than the _guaranteed_(*) data corruptions of "dump". However, it may be that in the long run it would be advantageous to have a "filesystem maintenance interface" for doing things like backups and defragmentation.. Linus (*) Dump may work fine for you a thousand times. But it _will_ fail under the right circumstances. And there is nothing you can do about it.

CIS Lesson 11 wrap up

CIS Lesson 11 New commands: cpio- Flexible backup and restore utility dump- Backup utility restore- restore from dumps tar- Simple backup and restore utility New Files and Directories: /etc/dumpdates

CIS 191 – Lesson 11 Next Class Assignment: Check Calendar Page on web site to see what is due next week: Lab 9 Test 1 (take home) due beginning of next class No quiz next week!

CIS 191 – Lesson 11 Backup