Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Filesystem Maintenance
René Serral-Gracià Xavier Martorell-Bofill1
1Universitat Politècnica de Catalunya (UPC)
May 26, 2014
Introduction Filesystems Disk verification LVM Backups
Lectures
1 System administration introduction2 Operating System installation3 User management4 Application management5 System monitoring6 Filesystem Maintenance7 Local services8 Network services9 Security and Protection
10 Virtualization
R. Serral-Gracià, et. al Filesystem 2
Introduction Filesystems Disk verification LVM Backups
Outline
1 IntroductionGoals
2 Filesystems
3 Disk verification
4 Logical Volume Manager (LVM)
5 Backups
R. Serral-Gracià, et. al Filesystem 3
Introduction Filesystems Disk verification LVM Backups
Goals
Knowledge
FilesystemsBackup toolsBackup media
Abilities
Filesystem ampliationFilesystem verificationPerform and restore backups
R. Serral-Gracià, et. al Filesystem 4
Introduction Filesystems Disk verification LVM Backups
Outline
1 Introduction
2 Filesystems
3 Disk verification
4 Logical Volume Manager (LVM)
5 Backups
R. Serral-Gracià, et. al Filesystem 5
Introduction Filesystems Disk verification LVM Backups
Filesystems (I)
FAT (FAT16) –> DOSSmall disks (< 4GB)File names 8+3
FAT32 (VFAT) –> Win95Larger disksLong filenamesPartial definition of soft-linksNo owner or file access privileges
exFATFAT32 ExtensionTheoretical maximum capacity of 64ZiB (512TiB real)
NTFS –> WinNT, XP, Vista, Windows 7Integrates ownership and privileges (create, modify,access...)Maps to Windows NT security model
R. Serral-Gracià, et. al Filesystem 6
Introduction Filesystems Disk verification LVM Backups
Filesystems (II)
ext2UNIX FilesystemSoft/hard linksAccess privilegesLong filenames
ext3Adds journaling (eases error recovery)
reiserfsFiles and directories organized similarly to a databaseFeatures journalingVery efficient in small filesNo internal block fragmentation
R. Serral-Gracià, et. al Filesystem 7
Introduction Filesystems Disk verification LVM Backups
Filesystems (III)
xfsjournalingDynamic i-node managementACLsVery large disk sizesFilesystem activity log
jfsjournalingDynamic i-node managementACLs and MAC (Mandatory Acess Control)Very large disk sizes
R. Serral-Gracià, et. al Filesystem 8
Introduction Filesystems Disk verification LVM Backups
Filesystems (i IV)
ext464 bits addressing, improvements in journalingDelayed allocationExtents1 exbibyte (EiB) maximum size
btrfsExtentsOnline resizingOnline balancingOnline filesystem check
R. Serral-Gracià, et. al Filesystem 9
Introduction Filesystems Disk verification LVM Backups
Journal filesystems
Journal: disk operation registryEases the recovery of the FS in case of crash or errorSlightly decrease in disk operations performance
Journal outside the buffer cacheJournal can be stored in another disk or partition
Ext3/4, reiserfs, JFS, XFS, NTFS, BTRFS have journal
R. Serral-Gracià, et. al Filesystem 10
Introduction Filesystems Disk verification LVM Backups
Outline
1 Introduction
2 Filesystems
3 Disk verificationDisk fragmentationFilesystem ampliationDisk quota management
4 Logical Volume Manager (LVM)
5 Backups
R. Serral-Gracià, et. al Filesystem 11
Introduction Filesystems Disk verification LVM Backups
Disk verification (I)
Reason for errors
Hardware errorsPower shortageOperating system bugsAdministration errors
Incorrect machine shutdown
Never verify a filesystem while mountedHigh probability of disk corruptionVerification access skips the buffer cache and acts directluon the device
R. Serral-Gracià, et. al Filesystem 12
Introduction Filesystems Disk verification LVM Backups
Disk verification (II)
Logical verification
Filesystem metadataDirectory structureLost data recovery
Directory lost+found
Physical recovery
Disk blocks with Input/output errorsCommand: badblocks
R. Serral-Gracià, et. al Filesystem 13
Introduction Filesystems Disk verification LVM Backups
Disk fragmentation
Faster access to consecutive disk blocksFaster access to closer files within the diskFaster access Depending on the disk region
Zona mitja vs. extrems del disc
R. Serral-Gracià, et. al Filesystem 14
Introduction Filesystems Disk verification LVM Backups
Filesystem ampliation
Install and configure the new diskPartitionor recycle existing ones. . .
Create filesystemDecide mountpointsTransfer the required data to the new partitionMount the partition
Update /etc/fstab
Maybe you have to reorganize existing directories/home→ /homeA + /homeB/home→ /home/students + /home/professors
R. Serral-Gracià, et. al Filesystem 15
Introduction Filesystems Disk verification LVM Backups
Exercise
Planificar i definir possibles ampliacions dels segÃ14ents
directoris
/home
/usr/local
/var
R. Serral-Gracià, et. al Filesystem 16
Introduction Filesystems Disk verification LVM Backups
Disk quota (I)
QuotaAbility to limit the amount of data a user (or user group) is ableto use in a filesystem (partition)
Requires
Support from the filesystemSupport from the kernel
R. Serral-Gracià, et. al Filesystem 17
Introduction Filesystems Disk verification LVM Backups
Quota management (II)
Partition preparation
Mounted using options ’usrquota’ and/or ’grpquota’It can be done from /etc/fstab
/dev/sda9 /home ext4 defaults,usrquota,grpquota 1 1
quotacheck command to create the quota filesquotacheck -v -a -g -u -m
verbose all group user no-remount
Creates/aquota.user/aquota.group
R. Serral-Gracià, et. al Filesystem 18
Introduction Filesystems Disk verification LVM Backups
Quota management (III)
Quota enablingquotaon -v -a -g -u
verbose all group user
Activates quota mechanisms, usually from /etc/init.d/
Quota disabling/sbin/quotaoff
Quota editing (edquota)Disk quotas for user xavim (uid 500):Filesystem blocks soft hard inodes soft hard/dev/sdb1 3 16 32 2 0 0
Data blocs and i-nodes quotaIt is not possible to edit the used blocks/inodes, but thelimits of the quota can be changed
R. Serral-Gracià, et. al Filesystem 19
Introduction Filesystems Disk verification LVM Backups
Quota management (and IV)
Visualize quotas: quota -v
Disk quotas for user xavim (uid 500):Filesystem blocks quota limit grace files quota limit grace/dev/sdb1 32* 16 32 6days 2 0 0 -
∗ We are above the quotas, within the "hard" limit!!
"Grace period"Grace time where the user can reach the hard limit, it onlyraises warningsIf the grace period expires, then the system does not allowto go above the soft limit
R. Serral-Gracià, et. al Filesystem 20
Introduction Filesystems Disk verification LVM Backups
Other maintenance tasks
Monitoring
Free space (df)Most systems reserve a (5%) of the space to be exclusivelyused by root
Occupied space (du)
Synchronization
Write to disk the modified bufferssyncUpdate daemon
R. Serral-Gracià, et. al Filesystem 21
Introduction Filesystems Disk verification LVM Backups
Outline
1 Introduction
2 Filesystems
3 Disk verification
4 Logical Volume Manager (LVM)
5 Backups
R. Serral-Gracià, et. al Filesystem 22
Introduction Filesystems Disk verification LVM Backups
Logical Volume Manager (LVM) (I)
Physical Volumes (PV)
/dev/sda1 /dev/sdb1 /dev/sdb2
Volume Group
homeext3
swap / (root)ext4
usrbtrfs
Logical Volumes (LV)
R. Serral-Gracià, et. al Filesystem 23
Introduction Filesystems Disk verification LVM Backups
Logical Volume Manager (and II)
High level abstraction of the disk spaceAggregates multiple physical partitions/disks
Allows to add more partitions to the volumeIt allows logical partitions within the volume
They can be assigned logical namesCustomized distribution among the physical volumes
ResizingMove
Example: /etc/fstab/boot /dev/sda1 ...swap /dev/vg00/swap .../ /dev/vg00/root .../home /dev/vg00/home .../usr /dev/vg00/usr ...
R. Serral-Gracià, et. al Filesystem 24
Introduction Filesystems Disk verification LVM Backups
Outline
1 Introduction
2 Filesystems
3 Disk verification
4 Logical Volume Manager (LVM)
5 BackupsFull BackupIncremental BackupReverse Incremental Backup
R. Serral-Gracià, et. al Filesystem 25
Introduction Filesystems Disk verification LVM Backups
Backups
Data to copyUser data (home, mail, ... )Program data (BBDDs, CVS, web, ...)System configurationBinary?
Backup frequencyData confidenceData importance
Backup typesFull Backup (all)Incremental Backup (only changes)Reverse Incremental Backup (only changes)
R. Serral-Gracià, et. al Filesystem 26
Introduction Filesystems Disk verification LVM Backups
Full Backup
Always copy all the dataFast to restoreLarge size
Sistema
backup
backup
backup
Temps
R. Serral-Gracià, et. al Filesystem 27
Introduction Filesystems Disk verification LVM Backups
Incremental Backup
Only backup the changed filesAdvantages
Small size
It can be done in any media
InconveniencesSlower to restore
The first one is equal to a fullbackup
Do not create a large backup sequence
Sistema
backup total
backup
Restaura
cióbackup
Tem
ps
R. Serral-Gracià, et. al Filesystem 28
Introduction Filesystems Disk verification LVM Backups
Reverse Incremental Backup
Everything is copied but in the former backup only thechanges are kept
Advantages
Fast to restore
Little space
InconveniencesOnly possible in randomaccess media
Sistema
=backup
backup
backup =Temps
Restaura
ció
R. Serral-Gracià, et. al Filesystem 29
Introduction Filesystems Disk verification LVM Backups
Backups
Physical support
Floppy, disc, CD, tape,network. . .To consider:
Cost size ratioReliability
AvailabilityUsabilitySpeed
Copy location
Accident protectionFireproof boxesKeep some backups outside the company premisesStealing protection
R. Serral-Gracià, et. al Filesystem 30
Introduction Filesystems Disk verification LVM Backups
Exercise
Define a backup policy (data to backup, backup type,frequency, device, compression, . . . ) for a multi-user serverwithin a company with:
500 Gb. disk and 80 usersMail
50Mb per userWeb pages
20 Mb per user100 Mb company web
Code repository10 GB distributed among 20 projectsOnly 5 active projects
R. Serral-Gracià, et. al Filesystem 31
Introduction Filesystems Disk verification LVM Backups
Other considerations
When having different servers it is recommended toDefine specific backup machines
cheapereasy to administer
Tools: tar+rsync/ssh, amanda, bacula
R. Serral-Gracià, et. al Filesystem 32
Introduction Filesystems Disk verification LVM Backups
Personal Homework
Task automationProgramming language: bash, perlInformation search: find, grep. . .
R. Serral-Gracià, et. al Filesystem 33