File System ImplementationFile System Implementation

Chapter 12Chapter 12

File system OrganizationFile system Organization Application programsApplication programs Logical file systemLogical file system

manages directory informationmanages directory information manages file control blocksmanages file control blocks

File-organization moduleFile-organization module Knows about logical blocks and physical blocksKnows about logical blocks and physical blocks

Basic file systemBasic file system issue generic commands to the appropriate device driverissue generic commands to the appropriate device driver

I/O controlI/O control Device driversDevice drivers Interrupt handlersInterrupt handlers

devicesdevices

On disk structuresOn disk structures Boot control blockBoot control block

information needed by the system to boot the OSinformation needed by the system to boot the OS in the partition boot sectorin the partition boot sector

Partition control blockPartition control block partition detailspartition details

# of blocks in partition# of blocks in partition size of blockssize of blocks free-block countfree-block count free-block pointersfree-block pointers free FCB countfree FCB count FCB pointersFCB pointers

Unix – superblockUnix – superblock NTFS – Master File TableNTFS – Master File Table

Directory StructureDirectory Structure FCBFCB

Unix – inodeUnix – inode NTFS – stored in Master file tableNTFS – stored in Master file table

Relational structureRelational structure

In-memory structuresIn-memory structures

In-memory partition table containing info In-memory partition table containing info about each mounted partitionabout each mounted partition

In-memory directory structureIn-memory directory structuredirectory “cache”directory “cache”

system-wide open-file tablesystem-wide open-file tableFCB of each open fileFCB of each open file

per-process open file tableper-process open file tablepointer to the system-wide open-file table pointer to the system-wide open-file table

Creating a new fileCreating a new file

Application program calls the logical file Application program calls the logical file systemsystem

scan directory structurescan directory structure read appropriate directory in memoryread appropriate directory in memoryallocate a new FCBallocate a new FCBupdate directory structureupdate directory structurewrite directory structure back to disk write directory structure back to disk

Typical FCBTypical FCB

File Permissions

File owner, group, ACL

File Size

File data blocks

File dates (create access write)

Opening a FileOpening a File

User program issues an open callUser program issues an open call Directory structure searched for filenameDirectory structure searched for filename If file not already opened – If file not already opened –

FCB copied into system-wide open-file tableFCB copied into system-wide open-file table Includes a count of number of processes that possess Includes a count of number of processes that possess

the file the file

Entry made in the per-process open-file tableEntry made in the per-process open-file table File operations made via this entryFile operations made via this entry File name not necessarily in memory (why?)File name not necessarily in memory (why?)

Closing a fileClosing a file

Per-process table entry is removedPer-process table entry is removedSystem-wide entry count is reducedSystem-wide entry count is reducedWhen all processes that have opened the When all processes that have opened the

file close it, it is written back to diskfile close it, it is written back to disk

Directory ImplementationDirectory Implementation Linear ListLinear List

file pointers file pointers name name data blocks data blocks

create filecreate file search directory for duplicate namessearch directory for duplicate names add new entry at end of the directoryadd new entry at end of the directory

delete filedelete file search for named filesearch for named file release space allocated to itrelease space allocated to it

To reuse a directory entryTo reuse a directory entry mark the entry as unusedmark the entry as unused attach it to the list of free directory entriesattach it to the list of free directory entries copy last directory entry to freed location (shorter)copy last directory entry to freed location (shorter)

Directory ImplementationDirectory Implementation

Linear ListLinear ListDisadvantages:Disadvantages:

linear searchlinear search

Sorted listSorted listfaster search faster search more complex insert and deletemore complex insert and delete

B-treeB-treefast for bothfast for both

Directory ImplementationDirectory Implementation

Hash tableHash tableAdvantage: SpeedAdvantage: SpeedDisadvantage:Disadvantage:

?? ??

Allocation MethodsAllocation Methods

Contiguous AllocationContiguous AllocationEach file occupies a set of contiguous blocks Each file occupies a set of contiguous blocks

on the diskon the diskFast search timeFast search time

Block location can be computed from relative block Block location can be computed from relative block numbernumber

Directory structures need only contain the first Directory structures need only contain the first block number for the fileblock number for the file

Easy access for read and writeEasy access for read and write

Allocation MethodsAllocation Methods

Contiguous AllocationContiguous Allocation Dynamic storage allocation problemDynamic storage allocation problem first fit or best fit algorithmsfirst fit or best fit algorithms

Suffers from external fragmentationSuffers from external fragmentation How much space is needed for a file?How much space is needed for a file?

Solved by using extentsSolved by using extents another chunk of disk space linked to the end of the fileanother chunk of disk space linked to the end of the file Fixed or variable size?Fixed or variable size?

Allocation MethodsAllocation Methods Linked AllocationLinked Allocation

Each file is a linked list of disk blocksEach file is a linked list of disk blocks Directory entry is pointer to first linkDirectory entry is pointer to first link AdvantagesAdvantages

No external fragmentationNo external fragmentation Size of file need not be declared when file is createdSize of file need not be declared when file is created File can grow as long as free blocks are availableFile can grow as long as free blocks are available

DisadvantagesDisadvantages Effective only for sequential accessEffective only for sequential access space required for pointersspace required for pointers

can be mitigated by using clusterscan be mitigated by using clusters reliabilityreliability

what happens if a link is lost?what happens if a link is lost?

Allocation MethodsAllocation Methods

FATFAT file allocation tablefile allocation tablekeeps the block table in one place on diskkeeps the block table in one place on diskOne entry for each disk block, indexed by One entry for each disk block, indexed by

block numberblock numberEntry for a block contains the block number Entry for a block contains the block number

for the next block in the filefor the next block in the fileFat needs to be in memory to avoid large Fat needs to be in memory to avoid large

number of disk seeksnumber of disk seeks

Allocation MethodsAllocation Methods

Linked allocation solvesLinked allocation solvesExternal FragmentationExternal FragmentationSize-declaration problemsSize-declaration problems

But has problems withBut has problems withEfficient access (with no FAT)Efficient access (with no FAT)

Indexed AllocationIndexed Allocation

jeep 19

Directory Entry

9

16

1

10

25

-1

-1

-1

Block 19

Allocation MethodsAllocation Methods

Indexed allocationIndexed allocationAll pointers are in an index blockAll pointers are in an index blockEach file has its own index blockEach file has its own index blockDirectory contains address of index blockDirectory contains address of index blockTo read ith block, use pointer in ith block entry To read ith block, use pointer in ith block entry

to find the blockto find the block

File Allocation TableFile Allocation Table

test … 217

618

end of file

339

0

217

339

618

FAT

Directory Entry

no. of disk blocks

Indexed AllocationIndexed Allocation

Supports direct access without suffering from Supports direct access without suffering from external fragmentationexternal fragmentation

Suffers from wasted spaceSuffers from wasted space pointer overhead is greater than with linked allocationpointer overhead is greater than with linked allocation all nil index entries are wasted.all nil index entries are wasted.

What size should index block be?What size should index block be? one block may be either too big or too small.one block may be either too big or too small.

Index Block VariantsIndex Block Variants Linked SchemeLinked Scheme

An index block is normally one disk blockAn index block is normally one disk block For large files the last word in the file is a link to an additional For large files the last word in the file is a link to an additional

index blockindex block Multilevel indexMultilevel index

First index block contains pointers to second level index blocksFirst index block contains pointers to second level index blocks with 4096-byte blocks this approach would support 4gb fileswith 4096-byte blocks this approach would support 4gb files

Combined approachCombined approach First 15 or so pointers as direct index blocksFirst 15 or so pointers as direct index blocks next pointer is to a single indirect indexnext pointer is to a single indirect index following pointer is to double indirectfollowing pointer is to double indirect last pointer is to triple indirectlast pointer is to triple indirect

Unix inodeUnix inode

mode

owners(2)

timestamps(3)

size block

count

single indirect

double indirect

triple indirect

direct blocks

data

data

data

data

data

datadata

data

data

data

.

.

....

.

.

.

.

.

.

.

.

.

.

.

.

Indexed AllocationIndexed Allocation

DisadvantagesDisadvantagesSince blocks are not localized on disk, disk Since blocks are not localized on disk, disk

access can be slower because of excessive access can be slower because of excessive seeks.seeks.

Allocation PerformanceAllocation Performance

Which allocation method to use depends Which allocation method to use depends on how files will be used in the systemon how files will be used in the systemSystems with mostly sequential access should Systems with mostly sequential access should

use a different system than systems with use a different system than systems with mostly random accessmostly random access

For both types of access, contiguous For both types of access, contiguous allocation requires only one access per blockallocation requires only one access per block

Linked allocation requires up to Linked allocation requires up to ii disk reads disk reads for the for the ithith block block

PerformancePerformance

Some OS’s Some OS’s support direct-access files by contiguous support direct-access files by contiguous

allocationallocationMaximum length must be declared on creationMaximum length must be declared on creation

support sequential access by linked allocationsupport sequential access by linked allocationsequential access must be declared on creationsequential access must be declared on creation

OS must support both allocation systemsOS must support both allocation systems

Allocation PerformanceAllocation Performance

Other OS’sOther OS’sCombine contiguous allocation with indexed Combine contiguous allocation with indexed

allocationallocation

contiguous allocation for small files (3-4 contiguous allocation for small files (3-4 blocks)blocks)

switch to indexed allocation if file grows largeswitch to indexed allocation if file grows large

Allocation PerformanceAllocation Performance

Study of disk performance can lead to Study of disk performance can lead to optimizationsoptimizationsSun changed cluster size to 56k in order to Sun changed cluster size to 56k in order to

improve performanceimprove performancealso implemented read ahead and free behindalso implemented read ahead and free behindwent from 50% of CPU for 1.5 Mb per secondwent from 50% of CPU for 1.5 Mb per second to a substantial increase in throughput with to a substantial increase in throughput with

less CPUless CPU

Free Space managementFree Space management

Bit vectorBit vector each unallocated block is set to 1 in a bit vectoreach unallocated block is set to 1 in a bit vector advantage is efficiency in finding first free block or advantage is efficiency in finding first free block or nn

consecutive free blocksconsecutive free blocks disadvantage is that the bit vector can be large: disadvantage is that the bit vector can be large:

1.3GB disk needs 332kb for vector1.3GB disk needs 332kb for vector Linked List Linked List

free blocks make a linked listfree blocks make a linked list inefficient. why?inefficient. why?

FatFat automatically has free blocksautomatically has free blocks

Free Space ManagementFree Space Management

GroupingGroupingstore address of first n free blocks in first free store address of first n free blocks in first free

blockblocknth pointer actually pointer to next groupnth pointer actually pointer to next group

CountingCounting like grouping but also store a count of number like grouping but also store a count of number

of contiguous free blocksof contiguous free blocksoverall list is shorteroverall list is shorter

EfficiencyEfficiency

Unix pre-allocates inodes across partitions in Unix pre-allocates inodes across partitions in order to improve performanceorder to improve performance

Unix also tries to keep a file’s data near its Unix also tries to keep a file’s data near its inodesinodes

BSD Unix varies the cluster size as a file grows BSD Unix varies the cluster size as a file grows in order to reduce internal fragmentationin order to reduce internal fragmentation

Many OS’s have supported various sized Many OS’s have supported various sized pointers in order to increase the addressable pointers in order to increase the addressable disk spacedisk space

PerformancePerformance

Disk CacheDisk CachePage CachePage CacheUnified virtual memoryUnified virtual memory

both disk cache and virtual memory use the both disk cache and virtual memory use the same virtual memory spacesame virtual memory space

Unified buffer cacheUnified buffer cacheBoth memory-mapped IO and regular system Both memory-mapped IO and regular system

calls use memory-mapped addressescalls use memory-mapped addressesThe regular disk buffer is not used in this caseThe regular disk buffer is not used in this case