OReilly Open Source Convention 2000 Linux Filesystems for Real-World Needs Æleen Frisch [email protected] e x ponential Consulting,

O’Reilly Open Source Convention 2000

Linux Filesystems for Real-World Needs

Æleen [email protected]

www.aeleen.com

exponential Consulting, LLCNorth Haven, Connecticut, USA

Copyright © 1999-2000,Exponential Consulting,

LLC

LinuxFilesystems

forReal-World

Needs

2

Itinerary

The Kernel’s VFS Local Filesystems Network Shared Filesystems Advanced Filesystems Futures


LLC

LinuxFilesystems

forReal-World

Needs

3

Virtual File System (VFS)

Kernel subsystem Provides a consistent interface for

low-level file I/O Filesystem needs only use that API and

provide specified functionality, andit is automatically supported.


LLC

LinuxFilesystems

forReal-World

Needs

4

Filesystems for Local Disks

Lots of supported types: ext2 CD-ROM: iso9660, Microsoft Joliet extensions ufs, fat, vfat, umsdos, ntfs, sysv, affs, adfs,

hfs, hpfs, qnx4, … minix procfs /dev/pts


LLC

LinuxFilesystems

forReal-World

Needs

5

Journaled Filesystems

ReiserFS in SuSE 6.3 ext3 dtfs:

www.complang.tuwien.ac.at/czezatke/lfs.html

SGI’s xfs being ported


LLC

LinuxFilesystems

forReal-World

Needs

6

Network Filesystems

NFS AFS smbfs ncpfs


LLC

LinuxFilesystems

forReal-World

Needs

7

Linux NFS

Longtime weak spot Unreliable Poor performance

nfsstones benchmark

Unable to communicate with non-Linux clients knfsd is what you want to use

Need to upgrade all current distributions as delivered to latest available version Kernel patches New daemon New utilities


LLC

LinuxFilesystems

forReal-World

Needs

8

Production Filesystem Features

Bigger than one disk partition (flexibility) Faster I/O Fault tolerant Distributed

Load balancing

Enhanced Security Access Control Lists Encryption


LLC

LinuxFilesystems

forReal-World

Needs

9

Logical Volumes

Dynamically-resizable filesystem consisting of multiple, independent disk partitions (physical volumes), upon which a virtual structure is imposed: Volume groups--virtual disks-- divisible into … Logical volume--virtual partitions--which hold … Filesystems

Common in modern OS’s: AIX, HP, True64

Logical Volume Manager (lvm): alpha code Veritas Volume Manager: $$$$


LLC

LinuxFilesystems

forReal-World

Needs

10

lvm

linux.msede.com/lvm/ Basics seem to work, but still Alpha code

(version 0.7) Maximum (theoretical) filesystem size:

> 1000 TB


LLC

LinuxFilesystems

forReal-World

Needs

11

lvm Commands

pv: create, change, display vg: create, change, display, ck,

cfgbackup/restore, export, extend, reduce, remove, split

lv: create, change, display, extend, reduce, remove e2fsadm (needs PartitionMagic’s resize2fs)


LLC

LinuxFilesystems

forReal-World

Needs

12

lvm Example

Run fdisk to set partition type to 0xFE pvcreate /dev/sdb1 /dev/sdc1 vgcreate new_vg /dev/sd{b,c}1

/dev/new_vg/group /etc/lvmconf/new_vg.conf

lvcreate –L 2g –n new_lv –r 4 –C y mke2fs /dev/new_vg/new_lv mount /dev/new_vg/new_lv /somewhere

e2fsadm –L +500m /dev/new_vg/new_lv vs. lvextend


LLC

LinuxFilesystems

forReal-World

Needs

13

RAID

Redundant Arrays of Inexpensive Disks

Choices Software Hardware

Controller RAID device


LLC

LinuxFilesystems

forReal-World

Needs

14

RAID Levels

0: Disk striping—performance onlyBest large transfer I/O bandwidthNo loss of storage capacity

1: Disk mirroringBest data redundancyGood performance on small transfers

3: Disk striping with parity diskOptimizes I/O bandwidth for large transfers Low impact of single disk failure

Obsolete!

5: Parity block rotates among component disksOptimizes I/O operations/secNot vulnerable to single disk failure


LLC

LinuxFilesystems

forReal-World

Needs

15

Linux RAID

0, 1, 4(!), 5 Linear

Kernel support must be enabled Patches needed (still):

ftp.*.kernel.org/pub/linux/daemons/raid/alpha RedHat installs for you

Special files: /dev/mdx Configuration file: /etc/raidtab

mkraid [device] Persistent superblock

Automatic detection raidstart and raidstop to control manually


LLC

LinuxFilesystems

forReal-World

Needs

16

Kernel Support


LLC

LinuxFilesystems

forReal-World

Needs

17

Sample /etc/raidtab Entries

Raid 0: Striping

raiddev /dev/md0raid-level 0nr-raid-disks 2chunk-size 64

persistent-superblock 1device /dev/sdc1

raid-disk 0device /dev/sdd1

raid-disk 1

Raid 1: Mirroring

raiddev /dev/md0raid-level 1nr-raid-disks 2

persistent-superblock 1device /dev/sdc1


raid-disk 1


LLC

LinuxFilesystems

forReal-World

Needs

18

Mirroring a Striped Disk: “RAID 10”

raiddev /dev/md0raid-level 0nr-raid-disks 2chunk-size 64k

…device /dev/sdc1


raid-disk 1

raiddev /dev/md1raid-level 0nr-raid-disks 2chunk-size 64k

device /dev/sde1raid-disk 0

device /dev/sdf1raid-disk 1

raiddev /dev/md2raid-level 1nr-raid-disks 2

device /dev/md0raid-disk 0

device /dev/md1raid-disk 1

But why?


LLC

LinuxFilesystems

forReal-World

Needs

19

RAID 5

raiddev /dev/md0raid-level 5nr-raid-disks 3nr-spare-disks 1persistent-superblock 1

device /dev/sdc1raid-disk 0

device /dev/sdd1raid-disk 1

device /dev/sde1raid-disk 2

device /dev/sdf1spare-disk 0


LLC

LinuxFilesystems

forReal-World

Needs

20

General RAID Considerations

Stripe size matters! Best value depends on typical I/O transfer size Testing: trial and error

Underlying filesystem block size = 4K mke2fs –b 4 …

Don’t overload controllers


LLC

LinuxFilesystems

forReal-World

Needs

21

Coda

Follow-on to the Andrew File System (AFS) www.coda.cs.cmu.edu

Distributed filesystem: /coda on client can have components (“volumes”)

from all over the network (transparent to user)

Lots of metadata, stored in a database, using logging and other transaction-based procedure to ensure integrity

Complex configuration task


LLC

LinuxFilesystems

forReal-World

Needs

22

Coda Features

Read-write data replication Automatic modification propogation and

synchronization/repair

Caching (performance, network failures, mobile computing) Client-requested files

Automatic update requests Client-modifications when server is unavailable; attempts

automatic synchronization of conflicts

Kerberos-like authentication tokens Access control lists


LLC

LinuxFilesystems

forReal-World

Needs

23

Intermezzo

www.inter-mezzo.com Distributed file system

Designed for high availability Palimpsest on Coda


LLC

LinuxFilesystems

forReal-World

Needs

24

Global File System (GFS)

www.globalfilesystem.org Network-shared storage


LLC

LinuxFilesystems

forReal-World

Needs

25

Access Control Lists

POSIX ACLs Alpha code

Recent call for testing …

Specifies file access on per-user/group basis ACLs inherited from parent directory Deny>Allow (null=>deny)


LLC

LinuxFilesystems

forReal-World

Needs

26

Enabling ACLs

Patch and build kernel Prompt for development option

Get/build utilities Patch and build ext2fs utilities

mount … -o acl setfacl and getfact Do frequent backups (files and ACLs)


LLC

LinuxFilesystems

forReal-World

Needs

27

Encryption

Encryption Steganography

Filesystem hidden in the low bits of each byte of an audio file

EFS ftp.csua.berkeley.edu/pub/cypherpunks/

filesystems/linux

PPDD linux01.gwdg.de/~alatham/ppdd.html


LLC

LinuxFilesystems

forReal-World

Needs

28

Setup

Patch/configure/build kernel Get/build utilities Patch/rebuild standard tools

dd to create file losetup to set algorithm and pass phrase mke2fs


LLC

LinuxFilesystems

forReal-World

Needs

29

Using an Encrypted filesystem

fstab: device mount-pt ext2 defaults,noauto,loop,encryption=type 0 0

Prompted for pass phrase when filesystem is mounted


LLC

LinuxFilesystems

forReal-World

Needs

30

Filesystem Futures

Documents

OReilly Open Source Convention 2000 Linux Filesystems for Real-World Needs Æleen Frisch [email protected] e x ponential Consulting,