Long-term Information Storage

1

Long-term Information Storage

1. Must store large amounts of data

2. Information stored must survive the termination of the process using it

3. Multiple processes must be able to access the information concurrently. In short:

2

Long-term Information Storage

Files: Good!

3

Long-term Information Storage

Files: Good! No Files: Bad!

4

File System

Operating system determines how files are: Structured Named Accessed Used Protected Implemented

Most important aspect to users is how files appear to them: naming convention, available operations, protection, etc. (Not implementation!!).

5

File Naming

Unix: Case sensitive. Allows, but does not require, extensions (e.g., prog.c).

Assigns no meaning to extensions. Add as many extensions as desired (e.g.,

prog.back.stupid.c). Does not allow spaces in name (unless “\ “) ;

Windows: Not case sensitive. Allows 1-3 character extensions. Extensions have meaning (to other application codes,

not to the OS) Allows spaces in file name. (ever tried to copy “my

work” to Unix?)

6

File Naming

Typical file extensions.

7

File Structure

None - sequence of words, bytes Simple record structure

Lines Fixed length Variable length

Complex Structures Formatted document Relocatable load file

8

File Structure

Three kinds of files byte sequence (i.e., no structure). record sequence Tree (e.g., data base)

9

File Structure

Can simulate last two with first method by inserting appropriate control characters

Who decides: Operating system Program (i.e., programs can support any

model they want) Unix and Windows support only the

sequence of bytes functionality.

10

An executable file (Unix)

11

File Access

Sequential access read all bytes/records from the beginning cannot jump around, could rewind or back up convenient when medium was mag tape

Random access bytes/records read in any order essential for data base systems read can be …

move file marker (seek), then read or … read and then move file marker

12

File Attributes

Name – only information kept in human-readable form Identifier (file descriptor) – unique tag (number)

identifies file within file system Type – needed for systems that support different types Location – pointer to file location on device Size – current file size

13

File Attributes

Protection – controls who can do reading, writing, executing

Time, date, and user identification – data for protection, security, and usage monitoring

Information about files are kept in the directory structure, which is maintained on the disk (although generally cached).

14

File Operations Create Write Read Reposition within file Delete Truncate

15

File Operations in Unix

int fd = open(Fi) – search the directory structure on disk for entry Fi, and move the content of entry to memory

fd is a file descriptor (integer).

close (fd) – move the content of entry Fi in memory to directory structure on disk

seek() // change pointer to current location in file. read(fd, buf, num_bytes)

16

Open Files

Several pieces of data are needed to manage open files:

File pointer: pointer to last read/write location, per process that has the file open

Open-file count: counter of number of times a file is open – to allow removal of data from open-file table when last processes closes it

Disk location of the file: cache of data access information

Access rights: per-process access mode information

17

Open Files

Unix maintains an open-file table for each process and for the whole system.

File descriptor is used as an index into the process open-file table. Entries are items that have to do with that particular process (e.g., file pointer, access rights, etc.).

A pointer to the system-wide open-file table is also in the process open-file table.

System-wide open-file table holds process-independent information (e.g., location on disk, last access time, file size, count of the number of processes using the file).

18

Open File Locking

Provided by some operating systems and file systems

Mediates access to a file Mandatory or advisory:

Mandatory – access is denied depending on locks held and requested

Advisory – processes can find status of locks and decide what to do

19

Directory A collection of data structures containing information about files

F 1 F 2F 3

F 4

F n

Directory

Files

Both the directory structure and the files reside on diskBackups of these two structures are kept on tapes

20

Operations Performed on Directory

Search for a file Create a file Delete a file List a directory Rename a file Traverse the file system

21

Organize the Directory (Logically) to Obtain

Efficiency – locating a file quickly Naming – convenient to users

Two users can have same name for different files

The same file can have several different names

Grouping – logical grouping of files by properties, (e.g., all Java programs, all games, …)

22

Single-Level Directory

Naming problem

Grouping problem

23

Two-Level Directory Separate directory for each user

Path name

Can have the same file name for different user

Efficient searching

No grouping capability

24

Tree-Structured Directories

25

Tree-Structured Directories (Cont)

Efficient searching

Grouping Capability

In Unix, a directory is a file that contains meta-data about the files it contains.

26

Tree-Structured Directories (Cont)

Most OS support absolute and relative path names.

Unix has two pre-defined relative path names: . Represents current directory .. Represents parent directory

Current directory (working directory) cd /spell/mail/prog or cd .. (relative to CD)

27A UNIX directory tree

Path Names

28

To Open dict, the absolute path name is: /usr/lib/dict.

29

Assume Current Directory is /usr/jim. Then .. is /usr, . is /usr/jimTo access dict: ../lib/dict.

Relative Path Name

30

Unix: mkdir creates a new sub-directory below the current working directory.

rmdir removes an entire directory (and all sub-directories).

rm deletes a file If someone suggests that you try out a cool

command calledrm –r * don’t do it!!

31

Shared Files (1)

File system containing a shared file

32

Links

This is termed a hard link. Both directory entry pointing to the same inode.

(a) Situation prior to linking(b) After the link is created(c) After the original owner removes the file

33

Symbolic Links

Provide the path name of the target file in the linked file.

Other processes do not have access to the inode (i.e., directory structure).

What happens when file deleted by owner?

34

Implementing Directories (1)

(a) A simple directory fixed size entriesdisk addresses and attributes in directory entry

(b) Directory in which each entry just refers to an i-node

35

File Control Block

36

Accessing a File

37

Allocation of File Blocks

Contiguous allocation Linked-list allocation FAT Indexed (inodes).

38

Directory Structure with Contiguous Allocation of File Blocks

39

Implementing Files: Contiguous Allocation

(a) Contiguous allocation of disk space for 7 files(b) State of the disk after files D and E have been removed

40

Linked-list Allocation

41

File Allocation Table

42

Entry 4 bytes. Blocks 1K. 20 Million Entries (not files!) == 80 MB for table.

43

Indexed Allocation

44

Unix inode

45

Unix Directory Entry

Tester 15

File Name Inode

46

Unix File System

Unix File System: 1 inode for each file/directory.

B S Inode list Data blocks

Boot area superblock

47

File Attributes

Direct blocks

Single Indirect

Pointing to file data blocks

Points to data block whose data is the addresses of data blocks belonging to the file

Pointing to file data blocks

Double and Triple Indirect blocks not shown

48

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

100

400

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct block 1

File Attributes

253

Direct block 1

Problem: Open file /usr/pmd

Root inode

49

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

100

400

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct block 1

File Attributes

253

Direct block 1

Open file /usr/pmd

Root inode

Step 1: Fetch inode for / (root, always inode 0)

50

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

100

400

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct block 1

File Attributes

253

Direct block 1

Open file /usr/pmd

Root inode

Inode 0

51

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

100

400

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct block 1

File Attributes

253

Direct block 1

Root inode

Step 2: Grab data block 100 (it had better be a directory!!)

52

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct block 1

File Attributes

253

Direct block 1

Root inode

data block 100 (directory listing for /)

bin 8

include 6

usr 2

File Attributes

100

400

53

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct block 1

File Attributes

253

Direct block 1

Root inode

data block 100 (directory listing for /)

bin 8

include 6

usr 2

File Attributes

100

400

Step 3: Grab inode 2.

54

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

180

Direct block 1

File Attributes

Direct block 0

Direct block 1

File Attributes

253

Direct block 1

Root inode

File Attributes

100

400

inode 2

Step 4: Grab Data block 180

55

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

180

Direct block 1

File Attributes

Direct block 0

Direct block 1

File Attributes

253

Direct block 1

Root inode

File Attributes

100

400

Step 4: Grab Data block 180

56

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct block 1

File Attributes

253

Direct block 1

Root inode

data block 180. Directory listing for /usr

weew 8

Pops 6

pmd 3

File Attributes

100

400

57

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct block 1

File Attributes

253

Direct block 1

Root inode

Data block 180. Directory listing for /usr

weew 8

Pops 6

pmd 3

File Attributes

100

400

Step 5: Grab inode 3

58

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct block 1

Root inode

File Attributes

253

Direct block 1

File Attributes

100

400

inode 3

59

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct block 1

Root inode

File Attributes

253

Direct block 1

File Attributes

100

400

inode 3

Step 6. Fetch Data block 253.

60

Inode 0 Inode 1 Inode 2 Inode 3

File Attributes

Direct block 0

Direct block 1

File Attributes

180

Direct blbock 1

File Attributes

253

Direct block 1

Root inode

File Attributes

100

400

Step 6. Fetch Data block 253. Orc 8

Poo9 6

wimp 3

Data block 253. Directory listing for /usr/pmd

61

The UNIX File System

The steps in looking up /usr/ast/mbox

62

Exam

Comprehensive Will cover essentially the same concepts from

previous tests. Will cover virtual memory and Unix file system

organization and searching. VM:

Understand page replacement algorithms. What they are based on. Work through optimal, LRU, clock, counter as done in

class.

63

Understand demand paging. Page faults. What causes a page fault and what

happens!

When exam asks to solve a problem using approach shown in class, please do so.