Embed Size (px)
Citation preview
CIS 191 - Lesson 11
Backupand
Restore
CIS 191 - Lesson 11
Backup and RestoreThe Backup Process
• Losing data
• Backup Policies
• Devices
• Problems with Backup Programs
• Restoring Data
• File Permissions, Types, and Structures
• Defragmentation
• Protecting the Backup Media
CIS 191 - Lesson 11
Backup and RestoreLosing data
How to lose data
• Software bugs
• Pilot errors
• Hardware failures
• Hackers
• Many more ways …
It will happen!
CIS 191 - Lesson 11
Backup and RestoreBackup Policies
Backup Policies (from textbook 9.1)• Perform LAN backups from one system • Label the media• Choose reasonable backup interval• Factor in modification frequency of files/filesystems • Make daily dumps fit on one piece of media • Make filesystems smaller than the backup media• Keep media off-site• Secure off-site media• Limit activity during backups• Verify your media• Pay attention to media life• Design your storage with backups in mind• Prepare for the worst
CIS 191 - Lesson 11
Backup and RestoreBackup devices
• Tape Devices‒ Problems with tapes ‒ The buffer utility ‒ Tapes and random file access
• Writable CD and DVD ‒ CDs 650MB ($.15-30 each)‒ DVDs 4.7 GB ($.20-43 each)‒ DVDs (DL) 8.5 GB ($2-4 each)‒ Blu-ray 25 GB ($8-12 each)‒ 1-5 years to 100's of years (if stored properly)
• Other Removable Media Devices• USB/Firewire• Solid state, external drives
• Hard drives and RAID Arrays
CIS 191 - Lesson 11
Backup and RestoreTypes of Data
• System Data‒ Basic information to set and run the system
• Application Software‒ Best to reinstall from source installation
disks• Databases
‒ Most important data types• User Data
‒ Includes site work• User Data and Home Directories
‒ Includes e-mail, files from home directory for personal use
CIS 191 - Lesson 11
Backup and RestoreMethodologies
• Epoch‒ Arbitrary time interval that denotes a
beginning • Incremental
‒ All files that have been added or changed since epoch
• Differential‒ Similar but easier than incremental, only
uses two tapes
File ghosting‒ Anomaly of the backup and restoration process
where deleted files turn up again!
CIS 191 - Lesson 11
Backup and RestoreArchive Utilities
• dump/restore‒ Oldest and most trusted backup utility‒ Backs up filesystems, not files (block-based
and fast)‒ Multiple levels of incremental backups
• tar‒ Used to archive or backup files, directories,
hard disks• cpio
‒ Copy input/output ‒ can use output of find command for wide
range of file selection critera (by type, user, modification time, etc)
CIS 191 - Lesson 11
Backup and RestoreCompression Utilities
• compress‒ Traditional compression utility‒ Not on CentOS 5
• zip‒ Used for compressing, archiving files into
single archive‒ compatible with WinZip
• gzip‒ Used for compressing, uncompressing files
one at a time• bzip2
‒ New, slower, but better compression device
CIS 191 - Lesson 11
Backup and RestoreBackup Software Packages
• AMANDA - http://www.amanda.org/
• Kbackup - http://kbackup.sourceforge.net/
• UNiBack - http://www.orbitsw.co.uk/uni_index.html
• Taper - http://taper.sourceforge.net/
• Arkeia - http://www.arkeia.com/
• Rsync - http://rsync.samba.org/
• Dar & KDar - http://kdar.sourceforge.net/
• Partimage (like Ghost) - http://partimage.org/Main_Page
CIS 191 - Lesson 11
Backup and RestoreBackup Software Packages
• AMANDA Advanced Maryland Automatic Network Disk
Archiver Able to split backups across media No GUI, command line only Can backup other servers over the network Uses native dump and tar utilities Amanda also can use SAMBA top backup
Windows hosts http://www.amanda.org/
CIS 191 - Lesson 11
Backup and RestoreBackup Software Packages
Kbackup Can use PGP to encrypt Can use gzip or compress to compress archives full or incremental backups access remote devices for LAN backups http://kbackup.sourceforge.net/
CIS 191 - Lesson 11
tar
CIS 191 - Lesson 11
tar command
tar file(s)ctvfx
createtable of contents (view)
extract
Note: The full path to each file is stored in the archive and these paths are used when restoring files
tarfilearchive file
files tobackup
options(no – needed)
v = verbose, double v (vv) provides more information
CIS 191 - Lesson 11
tar commandSingle file example
Backup a web page file
Oops! – file gets deleted
View backed up file
Restore the file
CIS 191 - Lesson 11
tar commandSingle file example
create
verbose tarfileto use
file to back up(requires full path)
view (table of contents)
verbose tarfileto use
extract
verbose tarfileto use
file in archive to restore(requires full path)
CIS 191 - Lesson 11
tar commandPay attention to the path stored in the tarfile
Creating the tarfile from another directory using absolute path
[root@benji ~]# tar cvf sample2.tar /opt/lampp/htdocs/sample.htmltar: Removing leading `/' from member names/opt/lampp/htdocs/sample.html[root@benji ~]# tar tvf sample2.tar-rw-r--r-- root/root 194 2008-10-24 10:13:10 opt/lampp/htdocs/sample.html[root@benji ~]#
[root@benji ~]# tar cvf sample2.tar ../opt/lampp/htdocs/sample.htmltar: Removing leading `../' from member names../opt/lampp/htdocs/sample.html[root@benji ~]# tar tvf sample2.tar-rw-r--r-- root/root 194 2008-10-24 10:13:10 opt/lampp/htdocs/sample.html[root@benji ~]#
Note, the leading / gets stripped
This path is used when restoring files
Creating the tarfile from another directory using relative path path
Note, the leading ../ gets stripped
This path is used when restoring files
CIS 191 - Lesson 11
tar command-C option
The –C option can be used to set the starting directory for file to be restored to. Add a – to the other options as well
CIS 191 - Lesson 11
tar command-C option
Change to the directory where the backup was made
Or use the –C option to restore from another directory
Find file to restore in archive
CIS 191 - Lesson 11
tar commandDirectory example
Backup up files (including hidden and sub-directories) starting from "here"
Delete some files
Verify they were backed up
Restore the files that were deleted
Note how tar saves full path for each file backed up
CIS 191 - Lesson 11
tar commandDirectory example
./xampp/contrib/xmlrss.php
./xampp/contrib/mingstats.html
./xampp/contrib/sql2xml.php
./xampp/contrib/sqlite.pl
./xampp/contrib/BabelSans-B.fdb
./xampp/contrib/oracle.pl
./xampp/contrib/interbase.php
./xampp/contrib/postgresql.php
./xampp/manuals.php
./xampp/cgi.cgi
./xampp/cds.php
./xampp/webalizer.php
./xampp/biorhythm.php
./xampp/AnkeCalligraph.TTF
./xampp/head.php
./xampp/guestbook-zh.pl
./xampp/start.php
./xampp/AnkeCalligraph.fdb
./xampp/mingswf.php
./xampp/mysql.php
./xampp/guestbook-nl.pl
./xampp/navi.php
./xampp/lang.php
./xampp/security.php
./xampp/lang/
./xampp/lang/de.php
./xampp/lang/it.php
./xampp/lang/fr.php
./xampp/lang/es.php
./xampp/lang/pt_br.php
./xampp/lang/nl.php
./xampp/lang/zh.php
./xampp/lang/no.php
./xampp/lang/pl.php
./xampp/lang/languages.php
./xampp/lang/jp.php
./xampp/splash-logo.php
./xampp/langsettings.php
./xampp/.version
./xampp/status.php
./xampp/phonebook.php
./xampp/img/
./xampp/img/rb.gif
./xampp/img/status3.gif
./xampp/img/logo-small.gif
./xampp/img/logo-small.jpg
./xampp/img/new.png
./xampp/img/signature-kay.gif
./xampp/img/lt.gif
./xampp/img/benji-500x420.jpg
./xampp/img/status4.gif
./xampp/img/rt.gif
./xampp/img/strichel.gif
./xampp/img/head-linux.gif
./xampp/img/head-xampp.gif
./xampp/img/xampp-logo.jpg
./xampp/img/head-fuer.gif
./xampp/img/head-for.gif
./xampp/img/head-solaris.gif
./xampp/img/status2.gif
./xampp/img/status1.gif
./xampp/img/head-windows.gif
./xampp/img/logo-big.gif
./xampp/img/signature-oswald.gif
./xampp/img/status5.gif
./xampp/img/blank.gif
./xampp/img/lb.gif
./xampp/img/xampp-logo-new.gif
./xampp/guestbook-es.pl
./xampp/guestbook-de.pl
./xampp/phpinfo.php
./xampp/splash.php
./xampp/guestbook-pt_br.pl
./xampp/guestbook.dat
./xampp/charset.php
./xampp/test.php
./xampp/softwarelist.inc
./xampp/guestbook-it.pl
./xampp/cds-fpdf.php
./xampp/ssi.inc
./xampp/php.php
./xampp/sqlite/
./xampp/sqlite/cdcol
./xampp/sqlite/phonebook.sqlite
./xampp/sqlite/.htaccess
./xampp/xampp.css
./xampp/ming.php
./xampp/showcode.php
./xampp/guestbook-fr.pl
./xampp/guestbook-en.pl
./xampp/iart.php
./.hidden
./favicon.ico
./index.html
createverbose
tarfileto use
files to back up(starting point)
CIS 191 - Lesson 11
tar commandDirectory example
view(table of contents)
Files to find in archivetarfileverbose
extractFiles to find to extracttarfile
verbose
CIS 191 - Lesson 11
tar commandSize can differ between tarfile and backed up file
Tarfiles Tarfile can be smaller than backed up file as it only
saves the date, not unused portion of data blocks Tarfile can be larger if backed up file is a sparse file Additional empty blocks can be added to tar version
CIS 191 - Lesson 11
Tarfiles Additional empty blocks can be added to tar version
tar commanddu size can differ between tarfile and backed up file
du report shows more disk space used than original for copy and tar extracts
original file
original file
ls command show same size for original, copy and tar extracts
Note: the –s option is to handle sparse files (discussed later)
CIS 191 - Lesson 11
Tarfiles Additional empty blocks can be added to tar version
[root@benji ~]# stat messages /var/log/messages sparse/messages nosparse/messages File: `messages' Size: 127538 Blocks: 272 IO Block: 4096 regular file<snipped> File: `/var/log/messages' Size: 127538 Blocks: 254 IO Block: 4096 regular file<snipped> File: `sparse/messages' Size: 127538 Blocks: 272 IO Block: 4096 regular file<snipped> File: `nosparse/messages' Size: 127538 Blocks: 272 IO Block: 4096 regular file<snipped>[root@benji ~]#
tar commandSize can differ between tarfile and backed up file
stat command shows copy and files extracted from tarfile have additional blocks which explains why file size is the same but disk usage is higher.
CIS 191 - Lesson 11
tar commandsparse files
Sparse Files Null blocks (containing only zeroes) are not stored on
the disk Can be problematic if utilities don't recognize them
[root@benji ~]# cd /var/log[root@benji log]# ls -l lastlog-rw-r--r-- 1 root root 234476 Nov 22 05:18 lastlog
[root@benji log]# du -h lastlog16K lastlog
Note the reduced size of the file actually stored on the disk using du –h command
CIS 191 - Lesson 11
Sparse Files tar command without using the sparse option (-S) for
sparse file
tar commandsparse files
The tarfile has expanded all the null blocks increasing disk space usage
[root@benji ~]# cd /var/log[root@benji log]# tar cvf /root/lastlog.tar lastloglastlog[root@benji log]# cd /root[root@benji ~]# ls -l lastlog.tar-rw-r--r-- 1 root root 245760 Nov 22 05:19 lastlog.tar[root@benji ~]# du -h lastlog.tar248K lastlog.tar
[root@benji ~]# tar xvf lastlog.tarlastlog[root@benji ~]# ls -l lastlog-rw-r--r-- 1 root root 234476 Nov 22 05:18 lastlog[root@benji ~]# du -h lastlog240K lastlog
CIS 191 - Lesson 11
Sparse Files tar command using the sparse option (-S) with sparse
file
tar commandsparse files
[root@benji ~]# cd /var/log[root@benji log]# tar cSvf /root/lastlog2.tar lastloglastlog[root@benji log]# cd /root
[root@benji ~]# tar xvf lastlog2.tarlastlog[root@benji ~]# ls -l lastlog-rw-r--r-- 1 root root 234476 Nov 22 05:18 lastlog[root@benji ~]# du -h lastlog20K lastlog[root@benji ~]#
Using the -S option, the tar command handles the sparse file efficiently
CIS 191 - Lesson 11
tar commandcompression
tar file(s) -C dirct vfx
tarfile
The z option uses gzip compresssion and the j option uses bzip2 compression
bzip2 is slower, but compresses moregzip is faster, but compreses less
CIS 191 - Lesson 11
tar commandcompression
Backing up htdocs with no compression
CIS 191 - Lesson 11
tar commandcompression
Post-compression using gzip of archive
CIS 191 - Lesson 11
tar commandcompression
Post-compression using bzip2 of archive
CIS 191 - Lesson 11
tar commandcompression
Using gzip compression option
CIS 191 - Lesson 11
tar commandcompression
Using the bzip2 compression option
CIS 191 - Lesson 11
tar commandcompression
[root@benji backup]# ls -Slrtotal 2049-rw-r--r-- 1 root root 340646 Nov 22 13:14 htdocs.tar.bz2-rw-r--r-- 1 root root 340646 Nov 22 13:18 htdocs2.tar.bz2-rw-r--r-- 1 root root 343748 Nov 22 13:13 htdocs.tar.gz-rw-r--r-- 1 root root 343760 Nov 22 13:15 htdocs2.tar.gz-rw-r--r-- 1 root root 706560 Nov 22 13:13 htdocs.tar[root@benji backup]#
Conclusions:• compression option slightly better than two steps• bzip2 compresses more that gzip
2 steps
CIS 191 - Lesson 11
tar commandcompression
FYI, not specifying the z option still works when restoring a file
CIS 191 - Lesson 11
tar commandcompression
FYI, not specifying the j option still works when restoring a file
CIS 191 - Lesson 11
tar commandcompression
tar file(s)ct vfx
Backups can be made to devices as well
zj
CIS 191 - Lesson 11
tar commandusing devices
[root@benji bin]# fdisk /dev/sda
Command (m for help): nFirst cylinder (582-652, default 582):Using default value 582Last cylinder or +size or +sizeM or +sizeK (582-652, default 652): +500M
Command (m for help): p
Disk /dev/sda: 5368 MB, 5368709120 bytes255 heads, 63 sectors/track, 652 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System/dev/sda1 * 1 382 3068383+ 83 Linux/dev/sda2 383 447 522112+ 82 Linux swap / Solaris/dev/sda3 448 511 514080 83 Linux/dev/sda4 512 652 1132582+ 5 Extended/dev/sda5 512 549 305203+ 83 Linux/dev/sda6 550 556 56196 83 Linux/dev/sda7 557 581 200781 83 Linux/dev/sda8 582 643 497983+ 83 Linux
Command (m for help):
Make a 500 MB partition
CIS 191 - Lesson 11
tar commandusing devices
Use partprobe so kernel will use new partition table
Command (m for help): x
Expert command (m for help): wThe partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.The kernel still uses the old table.The new table will be used at the next reboot.Syncing disks.[root@benji bin]# partprobe[root@benji bin]#
CIS 191 - Lesson 11
tar commandusing devices
You don't need to backup socket files. They are created automatically by the kernel when needed
Change to the directory to back up. Note /dev/sda8 is not mounted
CIS 191 - Lesson 11
tar commandusing devices
Restoring deleted file from backup archive on a device
Finding file in archive
Oops! …. file gets deleted
Viewing restored file
CIS 191 - Lesson 11
tar commandExamples
Don't backup and restore /proc (this is real time kernel data) !!
Perform full backup of entire file tree
-p, --same-permissions, --preserve-permissions Keeps permissions of extracted files the same as the originals.
CIS 191 - Lesson 11
tar commandExamples
Backup files in /opt after May 5, 1993
This allows differential and incremental backups to be done. Both types are based on a date. Either the last backup (incremental) or the last full backup (differential).
To do differential or incremental backups you will need a mechanism to track the dates or use a higher level backup program that makes use of tar.
CIS 191 - Lesson 11
cpio
CIS 191 - Lesson 11
cpio commandfind files by attribute
Note: -mount = don’t descend directories on other filesystems.-mtime n = True if the file's data was modified n days ago.
Sometimes the files we want to back up may or may not be in one directory
CIS 191 - Lesson 11
cpio command
Backup all files modified in the last week
verbose output
5K blocks (good practice)
CIS 191 - Lesson 11
cpio command
[root@benji opt]# cpio -vitB < /tmp/level-7.bakcpio: Removing leading `/' from member names-rwxr--r-- 1 root root 295 Nov 15 19:17 root/bin/undoLab06cpio: Removing leading `/' from member namesdrwxr-xr-x 2 root root 0 Nov 15 19:19 root/bin/backupscpio: Removing leading `/' from member names-rw-r--r-- 1 root root 0 Nov 15 19:17 root/bin/backups/motdcpio: Removing leading `/' from member names-rw-r--r-- 1 root root 124 Nov 15 19:17 root/bin/backups/.bashrccpio: Removing leading `/' from member names-rw-r--r-- 1 root root 176 Nov 15 19:17 root/bin/backups/.bash_profilecpio: Removing leading `/' from member names-rw-r--r-- 1 root root 21 Nov 15 19:17 root/bin/backups/issuecpio: Removing leading `/' from member names-rwxr--r-- 1 root root 1488 Nov 15 19:17 root/bin/doLab061 block[root@benji opt]#
Verify files backed up
input table of contents
CIS 191 - Lesson 11
cpio commandSelective file restore
Retain previous file modification time
Verify file was restored
CIS 191 - Lesson 11
Class Exercisecpio command
1. Prove to yourself that you can create backup of all files modified in the last 7 days using cpio. Put your backup in /tmp.What command did you use? [Table 17-20]
2. Verify the contents of your backup.What command did you use? [Table 5-8]
3. Delete and restore one of the files you backup.What command did you use? [Table 13-16]
CIS 191 - Lesson 11
dump
CIS 191 - Lesson 11
dump command
dump uf files-to-backup0123456789
filedevice
The dump level defines the different levels of incremental backups.
Each backup level has a date associated to indicate when the last backup at that level took place.
Level 0 is a full backup; level 1 backs up files that changed since the last level 0 backup; level 2 backs up files that changed since the last level 1 backup, and so forth.
dump level
Update the file /etc/dumpdates
Backup device or file
CIS 191 - Lesson 11
dump commanddump level update the file /etc/dumpdates
backup device or file
what to backup
CIS 191 - Lesson 11
dump command
table of contents
create a list of all files in the dump
record of the last dump
CIS 191 - Lesson 11
dump command
We are going to delete (really just move it) a file that is a symbolic link in the /etc directory
CIS 191 - Lesson 11
dump commandrun restore in interactive mode
Use ls and cd to navigate to file we want to restore
Add it to the list for extraction
CIS 191 - Lesson 11
dump command
* shows file marked for extraction
extract the file
quit the program
CIS 191 - Lesson 11
dump command
[root@benji /]# ls -li /etc/grub.conf /tmp/grub.conf191815 lrwxrwxrwx 1 root root 22 Nov 23 12:06 /etc/grub.conf -> ../boot/grub/grub.conf193953 lrwxrwxrwx 1 root root 22 Oct 12 03:28 /tmp/grub.conf -> ../boot/grub/grub.conf
CIS 191 - Lesson 11
Opinion SectionFrom: Linus Torvalds <torvalds@xxxxxxxxxxxxx>To: Neil Conway <nconway.list@xxxxxxxxxxxx>Subject: Re: [PATCH] SMP race in ext2 - metadata corruption.Date: Fri, 27 Apr 2001 09:59:46 -0700 (PDT)Cc: Kernel Mailing List <linux-kernel@xxxxxxxxxxxxxxx>
[ linux-kernel added back as a cc ]
On Fri, 27 Apr 2001, Neil Conway wrote:>> I'm surprised that dump is deprecated (by you at least ;-)). What to use instead for backups on machines that can't umount disks regularly?
Note that dump simply won't work reliably at all even in 2.4.x: the buffer cache and the page cache (where all the actual data is) are not coherent. This is only going to get even worse in 2.5.x, when the directories are moved into the page cache as well.
So anybody who depends on "dump" getting backups right is already playing russian rulette with their backups. It's not at all guaranteed to get the right results - you may end up having stale data in the buffer cache thatends up being "backed up".
Dump was a stupid program in the first place. Leave it behind.
> I've always thought "tar" was a bit undesirable (updates atimes or > ctimes for example).
Right now, the cpio/tar/xxx solutions are definitely the best ones, and will work on multiple filesystems (another limitation of "dump"). Whatever problems they have, they are still better than the _guaranteed_(*) data corruptions of "dump".
However, it may be that in the long run it would be advantageous to have a "filesystem maintenance interface" for doing things like backups and defragmentation..
Linus
(*) Dump may work fine for you a thousand times. But it _will_ failunder the right circumstances. And there is nothing you can do about it.
CIS 191 - Lesson 11
wrap up
CIS 191 - Lesson 11
New commands:cpio - Flexible backup and restore utilitydump - Backup utilityrestore - restore from dumpstar - Simple backup and restore utility
New Files and Directories:/etc/dumpdates
