Upload
samantha-rollins
View
233
Download
3
Tags:
Embed Size (px)
Citation preview
Advanced Operating Advanced Operating SystemsSystems
Prof. Muhammad Saeed
File Systems-IFile Systems-I
Advanced Operating Systems 2
File SystemsFile Systems
FilesDirectories & namingFile system implementationExample file systems
Advanced Operating Systems 3
Long-Term Information StorageLong-Term Information StorageMust store large amounts of data
Gigabytes -> terabytes -> petabytes
Stored information must survive the termination of the process using it (persistence)
Lifetime can be seconds to yearsMust have some way of finding it!
Multiple processes must be able to access the information concurrently
Advanced Operating Systems 4
Naming filesNaming files
Important to be able to find files after they’re createdEvery file has at least one nameName can be
Human-accessible: “solve.c”, “my drawing”, “Air Routes!”, “Speech Final Draft”Machine-usable: 4502, 33481
Case may or may not matterDepends on the file system
Name may include information about the file’s contentsCertainly does for the user (the name should make it easy to figure out what’s in it!)Computer may use part of the name to determine the file type
Advanced Operating Systems 5
Typical file extensionsTypical file extensions
Advanced Operating Systems 6
File structuresFile structures
Sequence of bytes Sequence of records
1 byte1 record
12A 101 111
sab wm cm avg ejw sab elm br
S02 F01 W02
Tree
Advanced Operating Systems 7
File typesFile types
Executablefile
Archive
Advanced Operating Systems 8
Accessing a fileAccessing a file
Sequential accessRead all bytes/records from the beginningCannot jump around
May rewind or back up, however
Convenient when medium was magnetic tapeOften useful when whole file is needed
Random accessBytes (or records) read in any orderEssential for database systemsRead can be …
Move file marker (seek), then read or …Read and then move file marker
Advanced Operating Systems 9
File attributesFile attributes
Advanced Operating Systems 10
File operationsFile operationsCreate: make a new fileDelete: remove an existing fileOpen: prepare a file to be accessedClose: indicate that a file is no longer being accessedRead: get data from a fileWrite: put data to a file
Append: like write, but only at the end of the fileSeek: move the “current” pointer elsewhere in the fileGet attributes: retrieve attribute informationSet attributes: modify attribute informationRename: change a file’s name
Advanced Operating Systems 13
Memory-mapped filesMemory-mapped files
Programtext
Data
Before mapping
Programtext
Data
After mapping
abc
xyz
Segmented process before mapping files into its address spaceProcess after mapping
Existing file abc into one segment Creating new segment for xyz
Advanced Operating Systems 14
More on memory-mapped filesMore on memory-mapped files
Memory-mapped files are a convenient abstractionExample: string search in a large file can be done just as with memory!Let the OS do the buffering (reads & writes) in the virtual memory system
Some issues come up…How long is the file?
Easy if read-onlyDifficult if writes allowed: what if a write is past the end of file?
What happens if the file is shared: when do changes appear to other processes?When are writes flushed out to disk?
Clearly, easier to memory map read-only files…
Advanced Operating Systems 15
Virtual File SystemVirtual File System
Advanced Operating Systems 16
DirectoriesDirectoriesNaming is nice, but limitedHumans like to group things together for convenienceFile systems allow this to be done with directories (sometimes called folders)Grouping makes it easier to
Find files in the first place: remember the enclosing directories for the fileLocate related files (or just determine which files are related)
Advanced Operating Systems 17
Single-level directory systemsSingle-level directory systems
One directory in the file systemExample directory
Contains 4 files (abc, def, ghi, jkl)owned by 3 different people: A, B, and C (owners shown in red)
Problem: what if user B wants to create a file called abc?
Rootdirectory
Aabc
Adef
Bghi
Cjkl
Advanced Operating Systems 18
Two-level directory systemTwo-level directory system
Solves naming problem: each user has her own directoryMultiple users can use the same file nameBy default, users access files in their own directoriesExtension: allow users to access files in others’ directories
Rootdirectory
Afoo
Abar
Bfoo
Bbaz
A B C
Cbar
Cfoo
Cblah
Advanced Operating Systems 19
Hierarchical directory systemHierarchical directory systemRoot
directory
Afoo
AMom
Bfoo
Bfoo.tex
A B C
Cbar
Cfoo
Cblah
APapers
APhotos
AFamily
Asunset
Asunset
Aos.tex
Akids
BPapers
Bfoo.ps
Advanced Operating Systems 20
Unix directory treeUnix directory tree
Advanced Operating Systems 21
Operations on directoriesOperations on directories
Create: make a new directoryDelete: remove a directory (usually must be empty)Opendir: open a directory to allow searching itClosedir: close a directory (done searching)
Readdir: read a directory entryRename: change the name of a directory
Similar to renaming a file
Link: create a new entry in a directory to link to an existing fileUnlink: remove an entry in a directory
Remove the file if this is the last link to this file
Advanced Operating Systems 22
File system implementation issuesFile system implementation issues
How are disks divided up into file systems?How does the file system allocate blocks to files?How does the file system manage free space?How are directories handled?How can the file system improve…
Performance?Reliability?
Advanced Operating Systems 23
Carving up the diskCarving up the disk
Masterboot record
Partition table
Partition 1 Partition 2 Partition 3 Partition 4
Entire disk
Bootblock
Superblock
Free spacemanagement
Indexnodes Files & directories
Advanced Operating Systems 24
Contiguous allocation for file blocksContiguous allocation for file blocks
Contiguous allocation requires all blocks of a file to be consecutive on diskProblem: deleting files leaves “holes”
Similar to memory allocation issuesCompacting the disk can be a very slow procedure…
A B C D E F
A Free C Free E F
Advanced Operating Systems 25
Contiguous allocationContiguous allocation
Data in each file is stored in consecutive blocks on diskSimple & efficient indexing
Starting location (block #) on disk (start)Length of the file in blocks (length)
Random access well-supportedDifficult to grow files
Must pre-allocate all needed spaceWasteful of storage if file isn’t using all of the spaceLogical to physical mapping
is easyblocknum = (pos / 1024) + start;offset_in_block = pos % 1024;
Start=5Length=2902
0 1 2 3
4 5 6 7
8 9 10 11
Advanced Operating Systems 26
Linked allocationLinked allocation
File is a linked list of disk blocks
Blocks may be scattered around the disk driveBlock contains both pointer to next block and dataFiles may be as long as needed
New blocks are allocated as needed
Linked into list of blocks in fileRemoved from list (bitmap) of free blocks
0 1 2 3
4 5 6 7
8 9 10 11
Start=9End=4Length=2902
Start=3End=6Length=1500
0
x
4 6
x
Advanced Operating Systems 27
Finding blocks with linked allocationFinding blocks with linked allocation
Directory structure is simpleStarting address looked up from directoryDirectory only keeps track of first block (not others)
No wasted space - all blocks can be usedRandom access is difficult: must always start at first block!Logical to physical mapping is done byblock = start;offset_in_block = pos % 1024;for (j = 0; j < pos / 1024; j++) { block = block->next;}
Assumes that next pointer is stored at end of blockMay require a long time for seek to random location in file
Advanced Operating Systems 28
Linked allocation using a Linked allocation using a RAM-based tableRAM-based table
Links on disk are slowKeep linked list in memoryAdvantage: fasterDisadvantages
Have to copy it to disk at some pointHave to keep in-memory and on-disk copy consistent
Advanced Operating Systems 29
Using a block index Using a block index for allocationfor allocation
Store file block addresses in an array
Array itself is stored in a disk blockDirectory has a pointer to this disk blockNon-existent blocks indicated by -1
Random access easyLimit on file size?
0 1 2 3
4 5 6 7
8 9 10 11
grades 4 4802Name index size
69708
Advanced Operating Systems 30
Finding blocks with indexed allocationFinding blocks with indexed allocation
Need location of index table: look up in directoryRandom & sequential access both well-supported: look up block number in index tableSpace utilization is good
No wasted disk blocks (allocate individually)Files can grow and shrink easilyOverhead of a single disk block per file
Logical to physical mapping is done byblock = index[block % 1024];offset_in_block = pos % 1024;
Limited file size: 256 pointers per index block, 1 KB per file block -> 256 KB per file limit
Advanced Operating Systems 31
Larger files with indexed allocationLarger files with indexed allocation
How can indexed allocation allow files larger than a single index block?Linked index blocks: similar to linked file blocks, but using index blocks insteadLogical to physical mapping is done byindex = start;blocknum = pos / 1024;for (j = 0; j < blocknum /255); j++) { index = index->next;}block = index[blocknum % 255];offset_in_block = pos % 1024;
File size is now unlimitedRandom access slow, but only for very large files
Advanced Operating Systems 32
Two-level indexed allocationTwo-level indexed allocation
Allow larger files by creating an index of index blocksFile size still limited, but much largerLimit for 1 KB blocks = 1 KB * 256 * 256 = 226 bytes = 64 MB
Logical to physical mapping is done byblocknum = pos / 1024;index = start[blocknum / 256)];block = index[blocknum % 256]offset_in_block = pos % 1024;
Start is the only pointer kept in the directoryOverhead is now at least two blocks per file
This can be extended to more than two levels if larger files are needed...
Advanced Operating Systems 33
Block allocation with extentsBlock allocation with extentsReduce space consumed by index pointers
Often, consecutive blocks in file are sequential on diskStore <block,count> instead of just <block> in indexAt each level, keep total count for the index for efficiency
Lookup procedure is:Find correct index block by checking the starting file offset for each index blockFind correct <block,count> entry by running through index block, keeping track of how far into file the entry isFind correct block in <block,count> pair
More efficient if file blocks tend to be consecutive on disk
Allocating blocks like this allows faster reads & writesLookup is somewhat more complex
Advanced Operating Systems 34
The ext4 filesystem can support volumes with sizes up to 1 exbibyte (EiB) and files with sizes up to 16 tebibytes (TiB). Stable system in 2008.
Extents replace the traditional block mapping scheme used by ext2/3 filesystems. An extent is a range of contiguous physical blocks, improving large file performance and reducing fragmentation. A single extent in ext4 can map up to 128 MiB of contiguous space with a 4 KiB block size.1 There can be 4 extents stored in the inode. When there are more than 4 extents to a file, the rest of the extents are indexed in an Htree.
ExtentsExtents
Advanced Operating Systems 35
The Relationships Among Segments, The Relationships Among Segments, Extents, and Data Blocks as implemented by Extents, and Data Blocks as implemented by OracleOracle
Advanced Operating Systems 36
FS Name Year Introduced
Original OS Max File Size Max FS Size Journaling
FAT16 1983 MSDOS V2 4GB 16MB to 8GB N
FAT32 1997 Windows 95 4GB 8GB to 2TB NHPFS 1988 OS/2 4GB 2TB NNTFS 1993 Windows NT 16EB 16EB YHFS+ 1998 Mac OS 8EB ? NUFS2 2002 FreeBSD 512GB to
32PB1YB N
ext2 1993 Linux 16GB to 2TB4 2TB to 32TB N
ext3 1999 Linux 16GB to 2TB4 2TB to 32TB Y
ReiserFS3 2001 Linux 8TB8 16TB YReiserFS4 2005 Linux ? ? YXFS 1994 IRIX 9EB 9EB YJFS ? AIX 8EB 512TB to 4PB Y
VxFS 1991 SVR4.0 16EB ? YZFS 2004 Solaris 10 1YB 16EB N
Advanced Operating Systems 37
ENDEND
Courtesy of University of PITTSBURGH