A Fast File System for UNIX

CS533 - Concepts of Operating Systems 1

A Fast File System for UNIX

Marshall Kirk McKusick, William N. Joy, Samuel J. Leffler and Robert S. Fabry

University of California, Berkeley

Presented by Catherine Vilhauer


Introduction

Problems in old UNIX file systemo Low data throughput rates not sufficient

• E.g. VLSI design and image processing - little processing on large amount of data

• Old UNIX provided only 2% of the maximum disk bandwidth or 20 kilobytes per second per arm.

Berkeley carried out modifications which are basis of today’s UNIX NFS


Review of Disk Structure

* Taken from Modern Operating Systems, Tanenbaum


Limits to Disk Bandwidth

Seeking and settling time Rotational delay

o Disk controller may be reading data into its cache

Reading/writing time.


Review of Disk Structure

* Taken from www.storagereview.com


The Old Unix File System

Each disk drive divided into one or more partitions

Each disk partition may contain one file system File system never spans multiple partitions File system described by superblock which

contains basic parameters of the file systemo Data blockso Count of the maximum number of fileso Pointer to the free list

Partitions set up by Master Boot Record


Old UNIX File System (cont’d)

Within file system are files (obviously) Some are directories

o Contain pointers to files that may also be directories Every file has a descriptor called an i-node

o Ownership of fileo Time stamps marking last modification and access

timeso Array of indices that point to data blocks for the fileo May contain references to indirect blocks containing

further data block indices


Review of What I-Nodes Are


Disk Layout in the Old UNIX File System


Disk Layout in the Old UNIX File System

Problemso Segregation of i-node info from datao Long seek timeo Files in a single directory not normally allocated

consecutive blocks - many non-consecutive blocks of i-nodes accessed when executing operations on the i-nodes of several files in a directory


Problem with old file system

i-node

data

More data


Other problems

Never transfers more than 512 bytes Often finds that the next sequential data block is

not on the same cylinder, forcing seeks between 512 byte transfers

Small block size, limited read-ahead in the system and many seeks severely limits file system throughput


Initial Improvements Made at Berkeley

Basic block size increased to 1024 byteso Improved performance by more than 2x

BUTo Still only using 4% of disk bandwidth

Why?o Free list (initially ordered) became scrambled after useo Forced a seek every block access


New File System

Cylinder Groups added Max block size - 4096 bytes

o Now possible to create files up to 232 byteso Size can be any power of 2 up to 4096 byteso Size is stored in super-blocko Allows file systems with different block sizes on same

system New allocation algorithms to improve locality


Cylinder Groups

Disk partition divided into Cylinder Groupso One or more consecutive cylinders on a disk

Cylinder group book-keeping info o Redundant copy of super-block kept at varying offset

so it spirals through the disko Space for i-nodeso Bit map describing available blocks (replaces free list)o Summary info describing usage of data blockso Begins at varying offset (so that all super-blocks not

lost)


Fragments and Fragmentation

New file system uses 4096 byte block size Allows 4x more information to be transferred per

disk transaction Problem: Small Files

o Uniformly large block wastes space Solution: Fragments

o Can divide single file system block into one or more fragments

o 2, 4 or 8 fragments specified at file system creation o Lower bound 512 bytes - disk sector size


Fragment Example

Bits in map XXXX XXOO OOXX OOOOFragment Numbers 0-3 4-7 8-11 12-15Block Numbers 0 1 2 3


Space Allocation

Space allocated when program does write system call

Each time the system checks to see if the size of the file has increased


Space Allocation - Three Types

There is enough space - easy No fragmented blocks and no space in last block

o If space in already allocated block then written hereo Otherwise a full block is allocated and written hereo Process repeated until < full block of new data remains.

File contains one or more fragments with no spaceo If size of new data plus size of data already in fragments

is greater than size of a full block, then a new block allocated and contents of the fragment are copied.

o Then continues as in second bullet point


Space Allocation (cont’d)

Problemso Data copied many times as a fragmented block expandso But fragment reallocation can be minimized if the user

program writes a full block at a time, except for a partial block at the end of the file.

Resultso Wasted space the same as old file systemo Savings in space utilization offset by need to keep track

of free blocks Notes

o File system should not be completely full - around 90% full is optimal


File System Parameterization

New file system tries to parameterize the hardware for optimum configuration

o Allocate new blocks on same cylinder as previous blocko Cylinder group summary info keeps count of available

blocks in cylinder group at different rotational positions• 8 rotational positions• Super-block contains vector of lists called rotational layout

tableso Parameter that defines the number of milliseconds

between completion of a data transfer and the initiation of another data transfer on same cylinder can be changed at any time


Layout Policies

Global Policieso Make placement decisions for new i-nodes and data blockso Calculate rotationally optimal block layoutso Decide when to force long seek because insufficient blocks

Local Policieso Use locally optimal scheme to lay out data blocks

Aims: o Increase locality of reference to minimize seek latencyo Improve layout of data to make larger transfers possible

Too much localization = local cylinder group run out of space


Layout Policies (cont’d)

Attempts to place all i-nodes of files in directory in the same cylinder group

Directories are differento New directory placed in cylinder group that has greater

than average free i-nodes, and smallest number of directors

Data blocks o Policy tries to place all data blocks for a file in cylinder

group at rotationally optimal positionso But, large files will use up all spaceo Redirect block allocation to different cylinder group

when file size >48 kilobytes and every megabyte after


Layout Policies (cont’d.)

Global policy routines call local allocation routines

If requested block not availableo Use next available, rotationally closest block o If no blocks open on same cylinder, use block in same

cylinder groupo If cylinder group full, quadratically hash to choose

another cylinder groupo Exhaustive search


New Data Layout

i-node

data

more data


Performance


Performance

Running ‘list directory’ on a large directory, number of disk accesses for i-nodes cut by factor of 2

Containing only files - cut by factor of 8 Transfer rates no longer change over time Bandwidth

o Old file system uses only 3-5% of disk bandwidtho New file system uses up to 47%

Reads and writes fastero Due to larger block size in new file systemo Writes same speed as reads in contrast to old system


Other new features

Allowed long file names - 512 bytes Implemented file locking Symbolic links

o References across physical file systems and inter-machine linkage

Renameo Old system required 3 system callso New implementation to avoid losing file with only

temporary name if system crashes Quotas

o Restricts the file system resources a user can obtain


Conclusion

FFS is the basis for UFS (Unix File System) Nearly all UNIX machines use a variant of UFS

includingo Solariso Unix BSD, Free Unixo Mac OS offers UFS as an alternative to its HFSo Linux offers partial support for UFS


References

Andrew S. Tanenbaum, Modern Operating Systems

Entry on Unix File Systems in Wikipedia.org

Documents

A Fast File System for UNIX