1. Backups
Chapter 10

2. Introduction
At most sites, the information stored on computers is worth more than the computers themselves.
It is also much harder to replace.
Protecting this information is one of the system administrator’s most important tasks
and, unfortunately, most tedious
Chapter 10 - Backups

3. Introduction
If executed correctly, backups allow administrators to restore filesystems (or any part of a filesystem) to the condition it was in at the time of the last backup.
Backups must be done carefully and on a strict schedule
The backup system and backup media must be tested regularly to verify that they are working correctly.
Chapter 10 - Backups

4. Introduction Chapter Overview
We begin this chapter with some general backup philosophy,
Followed by a discussion of the most commonly used backup devices and media
Their strengths, weaknesses, and costs
Next, we discuss the standard UNIX backup and archiving commands
and give some suggestions as to which commands are best for which situations
Chapter 10 - Backups

5. Introduction
We then talk about how to design a backup scheme and review the mechanics of the UNIX commands dump and restore.
Finally we take a look at Amanda, a free network backup package and offer some comments about its commercial alternatives
Chapter 10 - Backups

6. 1. Motherhood and apple pie
Before we get into the meat and potatoes of backups, we want to pass on some general hints that we have learned over time
(usually the hard way)
None of these suggestions is an absolute rule, but you will find that the more of them you follow, the smoother your dump process will be.
Chapter 10 - Backups

7. 1. Motherhood and apple pie
Perform all dumps from one machine
rdump allows you to perform dumps over the network.
Although there is some performance penalty for doing this, the ease of administration makes it worthwhile.
We have found the best method is to run a script from a central location that executes rdump (via rsh or ssh) on each machine that needs to be dumped
Or use a software package that automates this process.
Chapter 10 - Backups

8. 1. Motherhood and apple pie
All dumps should go to the same backup device
If your network is too large to be backed up by a single tape drive, you should try to keep your backup system as centralized as possible.
Centralization makes administration easier and allows you to verify that all machines were dumped correctly.
Chapter 10 - Backups

9. 1. Motherhood and apple pie
Label your tapes
It is essential that you label each dump tape clearly and completely.
An unlabeled tape is a scratch tape.
Detailed information such as lists of filesystems and dump dates can be written on the cases.
Chapter 10 - Backups

10. 1. Motherhood and apple pie
You must be able to restore root and /usr filesystems without looking at dump scripts
label the dump tapes with their format, the exact syntax of the dump command used to create them, and any other information you would need to restore from them without referring to on-line documentation
Free and commercial labeling programs abound.
Save yourself a major headache and invest in one.
Chapter 10 - Backups

11. 1. Motherhood and apple pie
Pick a reasonable backup interval
The more often backups are done, the smaller amount of data that can be lost in a crash.
However, backups use a systems resources and an operator’s time.
On busy systems it is generally appropriate to back up filesystems with home directories every workday.
The question is: How much data are your users willing to lose?
Chapter 10 - Backups

12. 1. Motherhood and apple pie
Choose filesystems carefully
Filesystems that are rarely modified do not need to be backed up as frequently as user’s home directories.
If only a few files change on an otherwise static filesystem (such as /etc/passwd in the root filesystem), these files can be copied every day to another partition that is backed up regularly.
Chapter 10 - Backups

13. 1. Motherhood and apple pie
Make daily dumps fit on one tape
In a perfect world, you could do daily dumps of all your filesystems onto a single tape.
You can mount a tape every day before you leave work and run the dump later at night from cron.
This way, dumps occur at a time when files are not likely to be changing, and the dumps have minimal impact on users.
Chapter 10 - Backups

14. 1. Motherhood and apple pie
If you can’t fit your daily backups on tape, you have several options:
Buy a higher capacity tape device.
Buy a stacker or library and feed multiple pieces of media to one device
Change your dump sequence
Write a smarter script
Use multiple backup devices.
Chapter 10 - Backups

15. 1. Motherhood and apple pie
Make filesystems smaller than your dump device
dump is perfectly capable of dumping filesystems to multiple tapes.
But if a dump spans multiple tapes, an operator must be present to change tapes.
Unless you have a good reason to create a really large filesystem, don’t do it.
Chapter 10 - Backups

16. 1. Motherhood and apple pie
Keep tapes off-site
Most organizations keep backups off-site so that a disaster such as fire cannot destroy both the original data and the backups.
The speed with which tapes are moved off-site should depend on how often you need to restore files and how much latency you can accept.
Chapter 10 - Backups

17. 1. Motherhood and apple pie
Protect your backups
“What does a backup do? It reliably violates file permissions at a distance.”
Not only should you keep your tapes off-site, but you should also keep them under lock and key
Some companies feel so strongly about the importance of backups that they make duplicates.
Chapter 10 - Backups

18. 1. Motherhood and apple pie
Limit activity during dumps
Filesystem activity should be limited during dumps because changes can cause dump to make mistakes.
You can limit activity by
doing the dumps when few active users are around (at night or weekend) or
by making the filesystem accessible only to dump.
Most users will not like this option. They want 24/7 access.
Chapter 10 - Backups

19. 1. Motherhood and apple pie
Check your tapes
There are many horror stories about system administrators that did not discover problems with their dump regime until after a serious system failure.
Checks:
Have your dump software attempt to reread the tapes immediately after it has finished dumping.
It is often useful to run restore t to generate a table of contents for each filesystem and store the results to disk.
Chapter 10 - Backups

20. 1. Motherhood and apple pie
Develop a tape life cycle
Tapes have a finite life cycle.
It is great to recycle your media.
Most manufacturers quantify their recommended life in terms of the number of passes that a tape can stand.
A backup, a restore, and an mt fsf (file skip forward) each represent one pass.
Chapter 10 - Backups

21. 1. Motherhood and apple pie
Prepare for the worst
After you have established a backup procedure, explore the worst case scenario: your site is completely destroyed.
Determine how much data would be lost and how long it would take to get your system back to life.
Include the time it would take to get new hardware.
Then determine if you can live with your answers.
Chapter 10 - Backups

22. 2. Backup devices and media
Since many types of failure can damage several pieces of hardware at once, backups should be written to some sort of removable media
Mirroring is a great form of backup, but if the controller dies, then what….
Many kinds of media store data using magnetic particles. These are subject to damage by electrical and magnetic fields.
Chapter 10 - Backups

23. 2. Backup devices and media
Here are some specific hazards to avoid:
Audio speakers contain large electromagnets
Transformers and power supplies
including UPS boxes
Monitors use transformers and high voltages.
Many monitors retain an electrical charge even after being turned off. Color monitors are the worst.
Long Shelf life
All tapes will become unreadable over a period of years.
Most media will keep for three years. If you need longer than that, use optical media or re-record the data.
Chapter 10 - Backups

24. 2. Backup devices and media
The following sub-sections describe some of the media that can be used for backups.
Although cost and media capacity are both important considerations, it is important to consider throughput as well.
Fast media are more pleasant to deal with, and they allow more flexibility in the scheduling of dumps.
Chapter 10 - Backups

25. 2. Backup devices and media
Floppy disks
The most inconvenient way to store backups.
They are slow and do not hold much data.
Although the individual disks are cheap, they hold so little data that they are in fact the most expensive backup medium overall.
Chapter 10 - Backups

26. 2. Backup devices and media
Super floppies
Iomega Zip drives
100MB to 250MB
SCSI, USB, Parallel, and Serial interfaces
Imation
Drives can read and write normal floppies and 120MB media
Although these products are useful for exchanging data, their high media costs make them a poor choice for backups.
Chapter 10 - Backups

27. 2. Backup devices and media
CD-R and CD-RW
Price changes make these a far more attractive medium than they were a few years ago.
CD-R’s are not quite as durable as stamped CDs.
They are good for archiving, but not a good choice for backup media.
Recordable DVD technology, with capacity around 10GB, will change how this field is viewed.
Chapter 10 - Backups

28. 2. Backup devices and media
Removable hard disks
The removable drive market is becoming very competitive, and prices vary daily.
Lifetime is 5-10 years
The main advantage of these products is speed (close to normal drives)
They are attractive as backup devices for small systems and home machines, although the disks themselves are somewhat pricey
Chapter 10 - Backups

29. 2. Backup devices and media
8mm cartridge tapes
Small format videotapes - also called Exabyte drives
The original format held 2GB, and newer formats hold up to 7GB.
Hardware compression build into the drives will push this even higher.
The size of the tapes makes off-site storage convenient.
Susceptible to heat damage.
Chapter 10 - Backups

30. 2. Backup devices and media
DAT (4mm) cartridge tapes
Digital Audi Tape drives - or Digital Data Storage (DDS) Drives
Original format was 2GB, DDS-4 holds up to 20GB
DDS-4 drives seek rapidly and transfer data at up to about 2.5MB/s, making the drives relatively fast.
They also do not have a history of alignment problems.
Chapter 10 - Backups

31. 2. Backup devices and media
Travan tapes
The next generation of QIC tape technology
Travan drives use a linear recording technology and support media from 2.5GB to 10GB
Tapes cost about $3/GB (slightly more than others)
Marketing hype claims 1MB/s transfer rate.
UNIX drive support is sketchy.
Chapter 10 - Backups

32. 2. Backup devices and media
OnStream ADR
A relative newcomer to backups.
Based upon Linear recording technology
Supports media sizes of 15GB and 25GB
Drives are cheap and fast
Chapter 10 - Backups

33. 2. Backup devices and media
DLT
Popular backup device
reliable, affordable, and hold large amounts of data.
Tapes hold up to 40GB
Transfer rates at 6 MB/s
Media is expensive ($65 per tape)
Manufacturers boast that tapes will last 20 to 30 years
(will the drives still be running then?)
Chapter 10 - Backups

34. 2. Backup devices and media
AIT
Sony’s 8mm product on steroids
AIT-2 claims
6 MB/s native transfer rate and
media capacity of 50GB
Tapes
Long life cycle
Contain an EEPROM to give the media some smarts.
Chapter 10 - Backups

35. 2. Backup devices and media
Mammoth
Produced by Exabyte - after falling out with Sony
Mammoth-2 drives offer blindingly fast 12MB/s native transfer rate
far faster than other tape drives in this price range.
Chapter 10 - Backups

36. 2. Backup devices and media
Jukeboxes, stackers, and tape libraries
With the low cost of disks these days, most sites have so much disk space that a full backup requires multiple tapes, even at 20GB per tape.
One solution for these sites is a stacker, jukebox, or tape library.
Chapter 10 - Backups

37. 2. Backup devices and media
Hard disks
We would be remiss if we did not mention the decreasing cost of hard drives as a reason to consider disk to disk backups.
Although we suggest that you not duplicate one disk to another within the same physical machine, hard disks can be a good low-cost solution for storage over a network.
Chapter 10 - Backups

38. 2. Backup devices and media
Summary of media types
Chapter 10 - Backups

39. 2. Backup devices and media
What to buy
DAT and Exabyte drives are excellent solutions for small workgroups
DLT, AIT, and Mammoth-2 are all roughly comparable and target University and Corporate environments.
Chapter 10 - Backups

40. 3. Setting up an incremental backup regime
The dump and restore commands are the most common way to create and restore from backups.
These programs have been part of UNIX for a very long time, and their behavior is well known.
At most sites, dump and restore are the underlying commands used by automated backup software.
Chapter 10 - Backups

41. 3. Setting up an incremental backup regime
Dumping filesystems
The dump command builds a list of files that have been modified since a previous dump, then packs those files into a single large file to archive on an external device.
dump has several advantages over other utilities described later in this chapter
Backups can span multiple tapes
Files of any types (even devices) can be backed up and restored
Permissions, ownerships, and modification times are preserved.
Files containing holes are handled correctly
Backups can be performed incrementally (with only recently modified files being written to tape)
Chapter 10 - Backups

42. 3. Setting up an incremental backup regime
dump understands the layout of raw filesystems, and it reads the filesystem’s inode tables directly to decide which files must be backed up
This allows dump to be very efficient
It also imposes a few limitations:
Every filesystem must be dumped individually
Only filesystems on the local machine can be dumped
you can overcome this with rdump
Chapter 10 - Backups

43. 3. Setting up an incremental backup regime
The most important feature of dump is its support for the concept of an “incremental” backup.
Incremental dumps make it possible to back up only filesystems that have changed since the last backup.
When you do a dump, you assign it a backup level (0-9).
A level 0 backup places the entire filesystem on the tape.
A level N dump backs up all files that have changed since the last dump of less than N
Chapter 10 - Backups

44. 3. Setting up an incremental backup regime
The first argument to dump must be the incremental dump level.
dump uses the /etc/dumpdates file to determine how far back to go.
The u flag causes dump to automatically update /etc/dumpdates when the dump completes.
Why should it be after it completes?
Chapter 10 - Backups

45. 3. Setting up an incremental backup regime
Dump sequences
Because dump levels are arbitrary, dumps can be performed on various schedules.
The schedule that is right for you depends on:
The activity of your filesystem
The capacity of your dump device
The amount of redundancy you want.
The number of tapes you want to buy.
Chapter 10 - Backups

46. 3. Setting up an incremental backup regime
A simple schedule
If your total disk space is smaller than the capacity of your tape device, you can use a trivial dump schedule
Do level zero dumps of every filesystem each day.
Reuse a group of tapes, but every N days, keep the tape forever
Don’t reuse the exact same tape for every night’s dump. It is better to rotate among a set of tapes so that even if one night’s dump is blown, you can still fall back to the previous night.
Chapter 10 - Backups

47. 3. Setting up an incremental backup regime
A moderate schedule
Assign a tape to each day of the week, and each month of the year
Every day do a level 9 dump to the daily tape.
Every week do a level 5 dump to the weekly tape.
And every month do a level 3 dump to the monthly tape.
Do a level 0 at least once per year.
Chapter 10 - Backups

48. 4. Restoring from dumps
Most of the many variations of the program that extracts data from dump tapes are called restore.
We will first discuss restoring individual files (or a small set of files), then explain how to restore entire filesystems.
Chapter 10 - Backups

49. 4. Restoring from dumps Restoring individual files
The first step to take when you are notified of a lost file is to determine which tapes contain versions of the file.
Users often want the most recent version of a file, but that is not always the case.
It is helpful if you can browbeat the users into telling you not only what files are missing, but also when they were lost and when they were last modified
Chapter 10 - Backups

50. 4. Restoring from dumps
If you do not keep on-line catalogs, you must mount tapes and repeatedly attempt to restore the missing files until you find the correct tape.
If the user remembers when the files were last changed, you may be able to make an educated guess about which tape the files might be on.
Chapter 10 - Backups

51. 4. Restoring from dumps
After determining which tapes you want to extract from, create and cd into a temporary directory (/var/restore) where a large hierarchy can be created
Most versions of restore must create all of the directories leading to a particular file before it can be restored.
Don’t use /tmp
Your work could be wiped out if the machine crashes and reboots before the restored data has been moved to its original location.
Chapter 10 - Backups

52. 4. Restoring from dumps restore has many command options.
i – for interactive restores of individual files and directories
r – for a complete restore of a filesystem
x – for non interactive restores of specified files
In general, restore i is the easiest way to restore a few files or directories from a tape.
However, it will not work if the tape does device cannot be moved backward a record at a time (a problem with some 8mm tape drives)
Chapter 10 - Backups

53. 4. Restoring from dumps Restoring entire filesystems
With luck you will never have to restore an entire filesystem after a system failure.
Before you begin a full restore, you must create and mount the target filesystem (see Chapter 8)
cd to the mount point,
Put the first tape of the most recent level 0 dump in the drive
Type restore r
Chapter 10 - Backups

54. 5. Dumping and restoring for upgrades
When you perform an OS upgrade, you must back up all filesystems with a level 0 dump and, possibly, restore them.
The restore is needed only if the new OS uses a different filesystem format, or if you are changing the partitioning of your disk.
A complete set of backups also gives you the option of reinstalling the old OS if the new version does not prove satisfactory.
You should also do a level 0 after the install.
System file time stamps may have the creation date, which will mess up dump’s incremental backups
Chapter 10 - Backups

55. 6. Using other archiving programs
dump is not the only program you can use to archive files to tapes; however, it is usually the most efficient way to back up an entire system.
tar, cpio, and dd can also move files from one medium to another
Chapter 10 - Backups

56. 6. Using other archiving programs
tar: package files
tar reads multiple files or directories and packages them into one file, often a tape file.
tar is a useful way to back up any files whose near-term recovery you anticipate.
tar is also useful for moving directory trees from place to place.
Chapter 10 - Backups

57. 6. Using other archiving programs
cpio: Sys V-ish archiving
cpio is similar to tar in functionality.
It dates from the beginning of time and is rarely used today.
When using cpio, read your man pages carefully; the options vary greatly among systems.
Chapter 10 - Backups

58. 6. Using other archiving programs
dd: twiddle bits
dd is a file copying and conversion program.
Unless it is told to do some sort of conversion, dd just copies from its input to its output
One historical use for dd was to create a copy of an entire filesystem.
A better way is to use newfs, dump, and restore.
Chapter 10 - Backups

59. 6. Using other archiving programs
volcopy: duplicate filesystems
volcopy makes an exact copy of a filesystem on another device.
It is available on Solaris, HP-UX, and Linux systems.
You can use the volcopy command to back up a filesystem to a removable disk or to make a complete copy of a filesystem on tape.
Chapter 10 - Backups

60. 7. Using multiple files on a single tape
In reality, magnetic tape contains one long string of data. However, it’s often useful to store more than one “thing” on a tape, so tape drives and their UNIX drivers conspire to provide you with a bit more structure.
When dump or some other command writes a stream of bytes out to a tape device then closes the file, an “end of file” marker is automatically placed on the tape.
Chapter 10 - Backups

61. 7. Using multiple files on a single tape
You can use the mt command to position a tape at a particular stream or “file set” as mt calls them.
mt is especially useful if you put multiple files (for example multiple dumps) on a single tape.
It also has some of the most interesting error messages of any UNIX utility.
Consult your manual for an exact list of commands supported by mt.
Chapter 10 - Backups

62. 8. Amanda
The Advanced Maryland Automatic Network Disk Archiver, Amanda, is a sophisticated network backup system that can replace the home-grown scripts used by many sites.
It supports multiple flavors of UNIX and many different kinds of backup media.
The latest information and source code are available from
Chapter 10 - Backups

63. 9. Commercial backup products
There are a lot of other commercial products available.
ADSM/TSM
Veritas
Legato
For a lot more information check out the web site or the book on the next page
Chapter 10 - Backups

64. 10. Recommended Reading Unix Backup and Recovery By Curtis W. Preston
O’Reilly
1999
Chapter 10 - Backups

65. Chapter 10 - Backups