Memory ManagementMemory ManagementVirtual Memory

Background; key issuesMemory allocation schemesyVirtual memoryMemory management design and implementation issues y g g p

1

Background

• Program– must be brought into memory (must be made a

process) to be executed.– process might need to wait on the disk, in input

queue before execution starts• Memory

– can be subdivided to accommodate multiple pprocesses

– needs to be allocated efficiently to pack as many processes into memory as possible

2

Memory Management

• Ideally programmers want memory that isy p g y– large– fast– non volatile

M hi h• Memory hierarchy– small amount of fast, expensive memory – cache

s m m di m sp d m di m p i m in m m– some medium-speed, medium price main memory– gigabytes of slow, cheap disk storage

• Memory manager handles the memory hierarchy

3

The position and function of the MMU(A T MOS 2/e)(A.T. MOS 2/e)

4

Relocation and Protection

• Cannot be sure where Cannot be sure where program will be loaded in memory– address locations of variables,

code routines cannot be absoluteabsolute

– must keep a program out of other processes’ partitions

5

Hardware support for relocation and protection

Base, bounds registers: set when the process is executing

6

Swapping (Suspending) a processA process can be swapped out of memory to a backing store (swap device) and later brought back (swap-in) into memory for continued device) and later brought back (swap in) into memory for continued execution.

7

Contiguous Allocation of Memory:Fixed PartitioningF xed art t on ng

any program no • any program, no matter how small, occupies an entire occupies an entire partition.

• this causes internal fragmentation.

8

Contiguous Allocation:Dynamic PartitioningDynam c art t on ng

• Process is allocated exactly as much memory as required• Eventually holes in memory: external fragmentation• Must use compaction to shift processes

(d f )(defragmentation)

9

Dynamic Partitioning: Placement algorithmslacement algor thms

• First-fit: use the first block that is big enough – fast

• Next-fit: use the next block that is big enough t d t t th l bl k t th d f th – tends to eat-up the large block at the end of the memory

• Best-fit: use the smallest block that is big enough must search entire list (unless free blocks are ordered by size) – must search entire list (unless free blocks are ordered by size)

– produces the smallest leftover hole.• Worst-fit: use the largest blockWorst fit: use the largest block

– must also search entire list – produces the largest leftover hole…. p g– … but eats-up big blocks

10

To avoid external fragmentation: Paging

• Partition memory into small equal size chunks (frames) equal-size chunks (frames) and divide each processinto the same size chunks (pages)

• OS maintains a page tablefor each processfor each process– contains the frame

location for each page in location for each page in the process

– memory address = (page b ff i hi number, offset within

page)

11

Paging Example

Question: do we avoid fragmentation completely?

12

Typical page table entry

(Fig From A Tanenbaum Modern OS 2/e)(Fig. From A. Tanenbaum, Modern OS 2/e)

13

Implementation of Page Table?1. Main memory:1. Main memory:

• page-table base, length registers• each program reference to memory => 2 memory accesses

14

Implementation of Page Table? 2: Associative Registers ssoc at ve eg sters

a.k.a Translation Lookaside Buffers (TLBs): special fast-lookup hardware cache; parallel search (cache for page table)hardware cache; parallel search (cache for page table)Address translation (P, O): if P is in associative register (hit), get frame # from TLB; else get frame # from page table in memory

Page # Frame #

Effective Access Time• Associative Lookup = ε time units (fraction of microsecond)• Associative Lookup = ε time units (fraction of microsecond)• Assume memory cycle time is 1 microsecond• Hit ratio (= α): percentage of times a page number is found in the

15

( ) p g f m p g m fassociative registers

• Effective Access Time = (1 + ε) α + (2 + ε)(1 – α) = 2 + ε – α

Two-Level Page-Table Scheme

2-level paging Address-Translation SchemeScheme

Three-level Paging Scheme

Hashed Page Tables

Th i t l b i • The virtual page number is hashed into a page table– This page table contains a p

chain of elements hashing to the same location

• Virtual page numbers are compared in this chain searching for a matchsearching for a match– If a match is found, the

corresponding physical f i t t dframe is extracted

Inverted Page Table• One entry for each real

page of memorypage of memory• Entry consists of the

virtual address of the page stored in that real page stored in that real memory location, with information about the process that owns that process that owns that page

• Decreases memory needed to store each page needed to store each page table, but increases time needed to search the table when a page table when a page reference occurs

• Use hash table to limit the search to one — or at the search to one or at most a few — page-table entries

Shared Pages

Shared code: one copy of read-only (reentrant) code shared (i t t dit il i d t among processes (i.e., text editors, compilers, window systems,

library-code, ...).watch for page numbers though; or use indirect referencingwatch for page numbers, though; or use indirect referencing

21

SegmentationSegmentation

• Memory management scheme that 1

1

4

• Memory-management scheme that supports user view of memory/program, i.e. a collection f t

2of segments.

• segment = logical unit such as:main program

32

main program,procedure, function,

43

local, global variables, common block,t k

user space physical memory space

stack,symbol table, arrays

22

space

Segmentation (A.T. MOS 2/e)

• One dimensional address space with growing tables23

• One-dimensional address space with growing tables• One table may bump into another

Segmentation Architecture

• Protection: each entry in segment table:lid ti bit 0 ill l t– validation bit = 0 ⇒ illegal segment

– read/write/execute privileges– ...

• Code sharing at segment level (watch for segment numbers, though; or use indirect referencing).

• Segments vary in length => need dynamic partitioning for memory allocation.

24

Sharing of segments

25

Comparison of paging and segmentation(A T MOS 2/e)(A.T. MOS 2/e)

26

Combined Paging and Segmentation• Paging

– transparent to the programmer– transparent to the programmer– eliminates external fragmentation

• Segmentation• Segmentation– visible to the programmer– allows for growing data structures modularity allows for growing data structures, modularity,

support for sharing and protection– But: memory allocation?y

• Hybrid solution: page the segments (each segment is b k f d )broken into fixed-size pages)– E.g. MULTICS, Pentium

27

Combined Address Translation Scheme

28

Example: The Intel Pentium

• Supports both segmentation and segmentation ith iwith paging

• CPU generates logical address– Given to segmentation unit

• Which produces linear addresses – Linear address given to paging unit

• Which generates physical address in main memoryP i it f i l t f MMU• Paging units form equivalent of MMU

Logical to Physical Address Translation in Pentium

Intel Pentium Segmentation

Pentium Paging Architecture

Linear Address in Linux

Broken into four parts:Broken into four parts:

Three-level Paging in Linux

Segmentation with Paging: MULTICS (A T MOS 2/e)(A.T. MOS 2/e)

Si lifi d i f th MULTICS TLB35

• Simplified version of the MULTICS TLB• Existence of 2 page sizes makes actual TLB more complicated

Virtual memory

36

Execution of a program:Virtual memory conceptV rtual memory concept

Main memory = cache of the disk space

• Operating system brings into main memory a few pieces of the programof the program

• Resident set - portion of process that is in main memorywhen an address is needed that is not in main memory a • when an address is needed that is not in main memory a page-fault interrupt is generated: – OS places the process in blocking state and issues a disk IO OS places the process in blocking state and issues a disk IO

request– another process is dispatched

37

Valid-Invalid Bit

• With each page table entry a valid–invalid bit is associated (initially 0)associated (initially 0)1 ⇒ in-memory0 ⇒ not-in-memoryy

11

Frame # valid-invalid bit

1110

M

• During address translation if valid invalid bit in

00

page table

38

• During address translation, if valid–invalid bit in page table entry is 0 ⇒ page fault interrupt to OS

Page Fault and (almost) complete address-translation scheme

In response to page-fault interrupt, OS must:

• get empty frame (swap outthat page?).that page?).

• swap in page into frame.

bl • reset tables, validation bit

• restartr startinstruction

39

if there is no free frame?Page replacement –want an algorithm which will result in minimum

number of page faults.p g

• Page fault forces choice – which page must be removedwhich page must be removed– make room for incoming page

Modified pa e must first be saved • Modified page must first be saved – unmodified just overwritten(use dirty bit to optimize

writes to disk)

• Better not to choose an often used pageBetter not to choose an often used page– will probably need to be brought back in soon

40

First-In-First-Out (FIFO) Replacement Algorithm

Can be implemented using a circular bufferEx.:Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5g• 3 frames

11 4 5

2

3

2

3

4

1

2

5

3

4

9 page faults

• 4 frames33 2 4

11 5 4

2

3

2

3

5

1

2

4

5 10 page faults

• Belady’s Anomaly: more frames , sometimes more page faults

33 2

44 3

41Problem: replaces pages that will be needed soon

Optimal Replacement Algorithm

• Replace page that will not be used for longest period of time.• 4 frames example4 frames example

1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5

1 4

2

3

6 page faults

H d k thi i f ?

4 5

• How do we know this info?• Algo used for measuring how well other algorithms perform.

42

Least Recently Used (LRU) Replacement Algorithm

Idea: Replace the page that has not been referenced for the longest time the longest time.

• By the principle of locality, this should be the page least likely to be referenced in the near future

Implementation:• tag each page with the time of last reference • tag each page with the time of last reference • use a stackProblem: high overhead (OS kernel involvement at every Problem: high overhead (OS kernel involvement at every

memory reference!!!) if HW support not available

43

LRU Algo (cont)

1 5

2

3 45

4 3

Example: Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5

44

LRU Approximations: Clock/Second Chance -Clock/Second Chance

• uses use (reference) bit : – initially 0

h i f d t t – when page is referenced, set to 1 by HW

• to replace a page:to replace a page:– the first frame encountered

with use bit 0 is replaced.– during the search for

replacement, each use bit set to 1 is changed to 0 by OSg y

• note: if all bits set => FIFO

45

Simulating LRU:the aging algorithm( B MO 2/ )the aging algorithm(A.B. MOS 2/e)

• The aging algorithm simulates LRU in software

46

LRU Approximations: Not Recently Used Page Replacement AlgorithmNot Recently Used Page Replacement Algorithm

• Each page has Reference (use) bit, Modified p g f ( ) , f(dirty) bit

– bits are set when page is referenced, modified• Pages are classified

1. not referenced, not modified f d difi d2. not referenced, modified

3. referenced, not modified4 referenced modified4. referenced, modified

• NRU removes page at random– from lowest numbered non empty classfrom lowest numbered non empty class

47

48

Design Issues for Paging Systems

• Global vs local allocation policies– Of relevance: Thrashing, working set

• Cleaning Policy• Fetch Policy• Page sizePage size

49

Local versus Global Allocation Policies (A.T. MOS2/e)

• Original configuration• Local page replacement

50

p g p• Global page replacement

Local versus Global Allocation Policies (A.T. MOS2/e)

Page fault rate as a function of the number 51

gof page frames assigned

Thrashing

• If a process does not have “enough” pages, the page fault rate is very high This leads to:page-fault rate is very high. This leads to:– low CPU utilization.

operating system may think that it needs to – operating system may think that it needs to increase the degree of multiprogramming.

– another process added to the systemanother process added to the system…– and the cycle continues …

• Thrashing ≡ the system is busy serving page faults (swapping pages in and out)faults (swapping pages in and out).

52

Thrashing Diagram

Wh d i k?Why does paging work?Locality model– Process migrates from one locality to another.– Localities may overlap.

Why does thrashing occur?

53

Why does thrash ng occur?Σ size of locality > total memory size

Page-Fault Frequency Scheme and Frame Allocation for Thrashing Avoidancefor Thrashing Avoidance

• Establish “acceptable” per-process page-fault rate.Esta sh acc pta p r proc ss pag fau t rat .– If actual rate too low, process loses frame.– If actual rate too high, process gains frame.

54

Working-Set Model for Thrashing Avoidance

• Δ ≡ working-set window ≡ a fixed number of page references Example: 10 000 instructionsExample: 10,000 instructions

• WSSi (working set of Process Pi) =total number of pages referenced in the most recent Δ(varies in time)– if Δ too small will not encompass entire locality.– if Δ too large will encompass several localities– if Δ too large will encompass several localities.– if Δ = ∞ ⇒ will encompass entire program.

• D = Σ WSSi ≡ total demand for frames

• D > m ⇒ Thrashing

• Policy: if D > m then suspend some process(es)55

• Policy: if D > m, then suspend some process(es).

Process Suspension for Thrashing Avoidance

• Lowest priority processp y p• Faulting process

– does not have its working set in main memory so will be blocked anywayblocked anyway

• Last process activated– this process is least likely to have its working set residentthis process is least likely to have its working set resident

• Process with smallest resident set– this process requires the least future effort to reload

• Largest process– obtains the most free frames

Process with the lar est remainin execution • Process with the largest remaining execution window

56

Keeping Track of the Working Set

• Approximate with interval timer f bit ( ll LRU + reference bit (recall LRU

approximation in software /aging algo)/aging algo)

• Example: Δ = 10,000– Timer interrupts after every 5000 Timer interrupts after every 5000

time units.– Keep in memory 2 bits for each

page.– Whenever a timer interrupts: copy

each page’s ref-bit to one of the p gmemory bits and reset each of them

– If one of the bits in memory = 1 ⇒57

– If one of the bits in memory = 1 ⇒page in working set.

Cleaning PolicyDetermines when dirty pages are written to disk:• Need for a background process, paging daemon: periodically

f g p p g g p y

inspects state of memory

Precleaning: pages are written out in batches, off-line, periodically: Wh f f f i d

g p g p yWhen too few frames are free, paging daemon– selects pages to evict using a replacement algorithm– can use same circular list (clock)

• as regular page replacement algorithm but with different pointers

Page buffering: use modified, unmodified lists of replaced pages (freed in advance)– A page in the unmodified list may be:p y

• reclaimed if referenced again• lost when its frame is assigned to another page

– Pages in the modified list are

58

g m f• periodically written out in batches• can also be reclaimed

Fetch Policy

Determines when a page should be brought into memory:

Demand paging only brings pages into main memory when a reference is made to itf m– Many page faults when process first started

P i b i i th d dPrepaging brings in more pages than needed– More efficient to bring in pages that reside contiguously on the

disk

59

Page Size: Trade-off

• Small page size: – less internal fragmentationless internal fragmentation– more pages required per process– larger page tables (may not be always in main memory)

• Small page size:– large number of pages in main memory; as time goes on during

execution the pages in memory will contain portions of the execution, the pages in memory will contain portions of the process near recent references. Page faults low.

– Increased page size causes pages to contain locations further from recent reference Page faults risefrom recent reference. Page faults rise.

– Page size approaching the size of the program: Page faults low again.

S d d si d t ffi i tl t sf l bl ks • Secondary memory designed to efficiently transfer large blocks => favours large page size

60

Page Size: managing space-overhead trade-off(A T MOS2/e)(A.T. MOS2/e)

• Overhead due to page table and internal f t tifragmentation

s e p⋅page table space

Wh2

s e poverheadp⋅

= + internal fragmentation

• Where– s = average process size– p = page size Optimized whenp = page size– e = page entry size

Optimized when

2p se=

62

Page Size (cont)

• Multiple page sizes provide the flexibility needed (to l TLB ffi i tl )also use TLBs efficiently):– Large pages can be used for program instructions

ll b d f h d– Small pages can be used for threads

• Multiple page-sizes available by microprocessors: MIPS R4000, UltraSparc, Alpha, Pentium. M t t ti t t l • Most current operating systems support only one page size, though.

63

Implementation IssuesOperating System Involvement with Paging (A.T. MOS2/e)Operating System Involvement with Paging (A.T. MOS2/e)

Four times when OS involved with paging1. Process creation

− determine program sizet t bl− create page table

2. Process execution− MMU reset for new processMMU reset for new process− TLB flushed

3. Page fault time− determine virtual address causing fault− swap target page out, needed page in

4 Process termination time4. Process termination time− release page table, pages

64

Memory Management with Bit Maps and linked lists (AT MOS 2e)Bit Maps and linked lists (AT MOS 2e)

• Part of memory with 5 processes 3 holes• Part of memory with 5 processes, 3 holes– tick marks show allocation units– shaded regions are freeshaded reg ons are free

• Corresponding bit map• Same information as a list

65

Same information as a list

Memory Management with Linked Lists: need of merge operations (AT MOS 2e)need of merge operations (AT MOS 2e)

Four neighbor combinations for the terminating process X

66

Implementation IssuesLocking Pages in Memory (A T MOS2/e)Locking Pages in Memory (A.T. MOS2/e)

Virtual memory and I/O occasionally interact• Virtual memory and I/O occasionally interact• Proc issues call for read from device into buffer

hil iti f I/O th t t – while waiting for I/O, another processes starts up– has a page fault

buffer for the first proc may be chosen to be pa ed – buffer for the first proc may be chosen to be paged out

• Need to specify some pages lockedNeed to specify some pages locked– exempted from being target pages– Recall lock-bitRecall lock bit

67

Implementation IssuesBacking Store (A.T. MOS2/e)

(a) Paging to static swap area68

(b) Backing up pages dynamically

Performance of Demand Paging

• Page Fault Rate 0 ≤ p ≤ 1.0– if p = 0 no page faults if p = 0 no page faults – if p = 1, every reference is a fault

• Effective Access Time (EAT)EAT = (1 – p) x memory access

+ p (page fault overhead+ [swap page out ]+ [swap page out ]+ swap page in+ restart overhead))

69

Other Considerations

• Program structure– Array A[1024 1024] of integerArray A[1024, 1024] of integer– Each row is stored in one page– Program 1 for j := 1 to 1024 do

for i := 1 to 1024 dofor i := 1 to 1024 doA[i,j] := 0;

1024 x 1024 page faults P 2 f i 1 t 1024 d– Program 2 for i := 1 to 1024 do

for j := 1 to 1024 doA[i,j] := 0;

1024 page faults1024 page faults

70

Memory-Mapped Files

• Memory-mapped file I/O allows file I/O to be t t d ti b i treated as routine memory access by mapping a disk block to a page in memory

• A file is initially read using demand paging. A y g p g gpage-sized portion of the file is read from the file system into a physical page. Subsequent y p y p g qreads/writes to/from the file are treated as ordinary memory accesses.y y

• Simplifies file access by treating file I/O • Simplifies file access by treating file I/O through memory rather than read() write()system calls

Memory Mapped Files

Memory-Mapped Shared Memory in Windows

Operating System Examples

• Windows XP• Windows XP

l • Solaris

Windows XP

• Uses demand paging with clustering. Clustering b i i di th f lti brings in pages surrounding the faulting page

• Processes are assigned working set minimum and working set maximum

• Working set minimum is the minimum number of gpages the process is guaranteed to have in memoryy

• A process may be assigned as many pages up to its working set maximumits working set maximum

• When the amount of free memory in the system falls below a threshold automatic working set falls below a threshold, automatic working set trimming is performed to restore the amount of free memory

Solaris

• Maintains a list of free pages to assign faulting processes

• Lotsfree – threshold parameter (amount of free memory) to begin paging

• Desfree – threshold parameter to increasing p gpaging

• Minfree – threshold parameter to being Minfree threshold parameter to being swapping

• Paging is performed by pageout process• Paging is performed by pageout process• Pageout scans pages using modified clock

l ithalgorithm• Scanrate is the rate at which pages are scanned.

Solaris 2 Page Scanner