VMLihu Rappoport, 12/2004 1 MAMAS – Computer Architecture Virtual Memory Dr. Lihu Rappoport

VMLihu Rappoport, 12/2004 1

MAMAS – Computer Architecture

Virtual MemoryDr. Lihu Rappoport


Memory System Problems Different Programs have different memory requirements

– How to manage program placement? Different machines have different amount of memory

– How to run the same program on many different machines? At any given time each machine runs a different set of

programs– How to fit the program mix into memory? Reclaiming unused memory?

Moving code around? The amount of memory consumed by each program is dynamic

(changes over time)– How to effect changes in memory location: add or subtract space?

Program bugs can cause a program to generate reads and writes outside the program address space

– How to protect one program from another?


Virtual Memory Provides illusion of very large memory

– sum of the memory of many jobs greater than physical memory address space of each job larger than physical memory

Allows available (fast and expensive) physical memory to be very well utilized

Simplifies memory management: code and data movement, and protection

Exploits memory hierarchy to keep average access time low Involves at least two storage levels: main and secondary Virtual Address: address used by the programmer Virtual Address Space: collection of such addresses Memory Address: address in physical memory also known as

“physical address” or “real address”


Virtual Memory: Basic Idea Divide memory (virtual and physical) into fixed size blocks

– Pages in Virtual space, Frames in Physical space– Page size = Frame size– Page size is a power of 2: page size = 2k

All pages in the virtual address space are contiguous Pages can be mapped into physical Frames in any order Some of the pages are in main memory (DRAM),

some of the pages are on disk. All programs are written using Virtual Memory Address Space The hardware does on-the-fly translation between virtual and

physical address spaces.– Use a Page Table to translate between Virtual and Physical addresses


Main memory can act as a cache for the secondary storage (disk)

Advantages:– illusion of having more physical memory– program relocation – protection

Virtual Memory

Virtual Addresses Physical AddressesAddress

Translation

Disk Addresses


Virtual to Physical Address translation

3 2 1 011 10 9 815 14 13 1231 30 29 28 27

3 2 1 011 10 9 815 14 13 1229 28 27

Virtual page number

Physical page number

Page offset

Page offset

Virtual Address

Physical Addresses

Page size: 212 byte =4K byte

Page Table


The Page Table

31

Page offset011

Virtual Page Number

Page offset11 0

Physical Frame Number29

Virtual Address

Physical Address

V D Frame number

1

Page table basereg

0

Valid bit

Dirty bit

12

AC

Access Control

12


If V = 1 then page is in main memory at frame address stored in table Fetch data

else (page fault)need to fetch page from disk causes a trap, usually accompanied by a context switch: current process suspended while page is fetched from disk

Access Control (R = Read-only, R/W = read/write, X = execute only)If kind of access not compatible with specified access rights then

protection_violation_fault causes trap to hardware, or software fault handler

Missing item fetched from secondary memory only on the occurrence of a fault demand load policy

Address Mapping Algorithm


Page Tables

Valid

1

Physical Memory

Disk

Page TablePhysical Page

Or Disk Address

111

11

11

1

0

0

0

Virtual page number


Page Replacement Algorithm Not Recently Used (NRU)

– Associated with each page is a reference flag such that ref flag = 1 if the page has been referenced in recent past

If replacement is needed, choose any page frame such that its reference bit is 0. – This is a page that has not been referenced in the recent past

Clock implementation of NRU:

last replaced pointer (lrp)if replacement is to take place,advance lrp to next entry (modtable size) until one with a 0 bitis found; this is the target forreplacement; As a side effect,all examined PTE's have theirreference bits set to zero.

1 01 000

page table entry

Ref bit

1 0

Possible optimization: search for a page that is both not recently referenced AND not dirty


Page Faults Page faults: the data is not in memory retrieve it from disk

– The CPU must detect situation– The CPU cannot remedy the situation (has no knowledge of the disk) CPU must trap to the operating system so that it can remedy the situation– Pick a page to discard (possibly writing it to disk)– Load the page in from disk– Update the page table– Resume to program so HW will retry and succeed!

Page fault incurs a huge miss penalty – Pages should be fairly large (e.g., 4KB)– Can handle the faults in software instead of hardware– Page fault causes a context switch– Using write-through is too expensive so we use write-back


Optimal Page Size Minimize wasted storage

– Small page minimizes internal fragmentation

– Small page increase size of page table Minimize transfer time

– Large pages (multiple disk sectors) amortize access cost

– Sometimes transfer unnecessary info

– Sometimes prefetch useful data

– Sometimes discards useless data early General trend toward larger pages because

– Big cheap RAM

– Increasing memory / disk performance gap

– Larger address spaces


Translation Lookaside Buffer (TLB) Page table resides in memory

each translation requires a memory access

A way to speed up translation is to use a special cache of recently used page table entries

TLBs have typically 128 to 256 entries

TLBs are usually highly associative

TLB access time is comparable to L1 cache access time

Yes

No

TLB Access

TLB Hit ?AccessPage Table

Virtual Address

Physical Addresses


TLB (Translation Lookaside Buffer) is a cache for recent address translations:

Valid

111101101101

111101

Making Address Translation Fast

Physical Memory

Disk

Virtual page number

Page Table

Valid Tag Physical PageTLB

Physical PageOr

Disk Address


Virtual Memory And Cache

Yes

No

TLB Access

TLB Hit ?AccessPage Table

Access Cache

Virtual Address

Cache Hit ?

Yes

No AccessMemory

Physical Addresses Data

TLB access is serial with cache access


Overlapped TLB & Cache Access

Physical page number Page offset

Virtual Memory view of a Physical Address

Disp

Cache view of a Physical Address

# SetTag

In the above example #Set is not contained within the Page Offset The #Set is not known Until the Physical page number is known Cache can be accessed only after address translation done

3 2 1 011 10 9 815 14 13 1229 28 27

3 2 1 011 10 9 815 14 13 1229 28 27


Overlapped TLB & Cache Access (cont)

Physical page number Page offset

Virtual Memory view of a Physical Address

Disp

Cache view of a Physical Address

# SetTag

In the above example #Set is contained within the Page Offset The #Set is known immediately Cache can be accessed in parallel with address translation First the tags from the appropriate set are brought Tag compare takes place only after the Physical page number is known (after address translation done)

3 2 1 011 10 9 815 14 13 1229 28 27

3 2 1 011 10 9 815 14 13 1229 28 27


Overlapped TLB & Cache Access (cont) First Stage

– Virtual Page Number goes to TLB for translation– Page offset goes to cache to get all tags from appropriate set

Second Stage– The Physical Page Number, obtained from the TLB, goes to the cache for tag compare.

Limitation: Cache < (page size × associativity) How can we overcome this limitation ?

– Fetch 2 sets and mux after translation– Increase associativity …


Overlapped TLB & Cache AccessThe K6 solution

In x86 the page size is 4K byte bits [11:0] are not translated

K6 L1 caches are 32K Byte, 2 way set-associative, 32 byte/line Index is comprised of bits [13:6] of the virtual address Since bits [13:12] are translated, the line may reside in one of 4 sets

(the physical translation of these bits may be one of 00, 01, 10 or 11). Tag is comprised of bits [31:1212] of the physical address (not just [31:14]) The tags from all 4 sets (8 tags) are then compared with bits [31:12] of the

physical address simultaneously If one of the 2 tags of the indexed set matches bits [31:12] of the physical

address, we have a cache hit Notice: bits [13:12] of the tag may differ from bits [13:12] of the indexed set

If the line is found in one of the 4 sets, but not in the indexed set, the line is invalidated (possibly written back if dirty), and then re-fetched from the L2 cache of from memory and put into the indexed set


Virtually-Addressed Cache Cache uses virtual addresses (tags are virtual)

Only require address translation on cache miss– TLB not in path to cache hit

Aliasing: two different virtual addresses map to same physical address– Two different cache entries holding data for the same physical address! – must update all cache entries with same physical address

Cache must be flushed at task switch

data

Trans-lation

Cache

MainMemory

VA

hit

PACPU


Virtually-Addressed Cache (cont).

Cache must be flushed at task switch– Solution: include process ID (PID) in tag

New problem: memory sharing among processes– Solution: permit multiple virtual pages to refer to same memory

Problem: incoherence if they point to different physical pages– Solution: require sufficiently many common virtual LSB. – With direct mapped cache, guarantied that they all point to same

physical page


More On Page Swap-out DMA copies the page to the disk controller:

– Reads each byte: Executes snoop-invalidate for each byte in the cache (both L1 and L2) If the byte resides in the cache:

if it is modified reads its line from the cache into memory invalidates the line

– Writes the byte to the disk controller– This means that when a page is swapped-out of memory:

All data in the caches which belongs to that page is invalidated The page in the disk is up-to-date

The TLB is snooped, and there is a TLB hit for the swapped-out page, its TLB entry is invalidated

In the page table – The valid bit in the PTE entry of the swapped-out pages set to 0 – All the rest of the bits in the PTE entry may be used by the operating

system for keeping the location of the page in the disk.


Large Address Spaces For a large virtual address space, we may

get a large page table, for example: For a 2 GB virtual address space, page

size of 4KB, we get 231/212=219 entries Assuming each entry is 4 bytes wide,

page table alone will occupy 2MB in main memory

Solution: use two-level Page Tables Master page table resides in main

memory Secondary page tables reside in virtual

address space Virtual address format

P1 index P2 index page offest10 10 12

4 bytes

4 bytes

4KB1KPTEs


VM in VAX: Address Format

Page size: 29 byte = 512 bytes

31

Page offset08

Virtual Page Number

Virtual Address930 29

0 0 - P0 process space (code and data)0 1 - P1 process space (stack)1 0 - S0 system space1 1 - S1

Page offset8 0

Physical Frame Number29

Physical Address9


VM in VAX: Virtual Address SpacesProcess0 Process1 Process2 Process3

P0 process code & global vars grows upward

P1 process stack & local vars grows downward

S0 system space grows upward, generally static

0

7FFFFFFF80000000


Page Table Entry (PTE)

V PROT M Z OWN S S0

Physical Frame Number31 20

Valid bit =1 if page mapped to main memory, otherwise page fault: • Page on the disk swap area • Address indicates the page location on the disk

4 Protection bits

Modified bit

3 ownership bits Indicate if the line was cleaned (zero)


System Space Address Translation

00

10 Page offset8 029 9

VPN

SBR (System page table base physical address)+

VPN0

PTE physical address=

00

00

PFN (from PTE)

Page offset8 029 9

PFN

Get PTE


System Space Address Translation (cont)

SBR

VPN*4

PFN

10 offset8 029 9

VPN

offset8 029 9

PFN

31


P0 Space Address Translation

00

00 Page offset8 029 9

VPN

P0BR (P0 page table base virtual address)+

VPN0

PTE S0 space virtual address=

00

00

PFN (from PTE)

Page offset8 029 9

PFN

Get PTE using system space translation algorithm


P0 Space Address Translation (cont)

SBR

P0BR+VPN*4

Offset’8 029 9

PFN’

00 offset8 029 9

VPN31

10 Offset’8 029 9

VPN’31

PFN’

VPN’*4

Physical addrof PTE

PFN

Offset8 029 9

PFN


P0 space Address translation Using TLB

Yes

No

ProcessTLB Access

ProcessTLB hit?

00 VPN offset

NoSystemTLB hit?

Yes

Get PTE of req page from the proc. TLB

Calculate PTE virtual addr (in S0): P0BR+4*VPN

System TLB Access

Get PTE from system TLB

Get PTE of req page from the process Page table

Access Sys Page Table inSBR+4*VPN(PTE)

Memory Access

Calculate physical address

PFN

PFN

Access Memory


Paging in x86 2-level hierarchical mapping

Page directory and page tables All pages and page tables are 4K

Linear address divided to: Dir 10 bits Table 10 bits Offset 12 bits Dir/Table serves as index

into a page table Offset serves ptr into a

data page Page entry points to a page

table or page Performance issues: TLB!

031

DIR TABLE OFFSET

Page Table

PG Tbl Entry

Page Directory

4K Dir Entry

4K Page Frame

Operand

Linear Address Space (4K Page)1121

CR3


x86 Page Translation Mechanism CR3 points to current page directory (may be changed per process) Usually, a page directory entry (covers 4MB) will point to a page

table that covers data of the same type/usage Can allocate different physical for same Linear (e.g. 2 copies of same code) Sharing can alias pages from diff. processes to same physical (e.g., OS)

DIR TABLE OFFSET

Page Dir Code

Data

StackOS

Phys Mem

4K page

CR3

Page Tables


x86 Page Entry Format 20 bit pointer to a 4K

Aligned address 12 bits flags Virtual memory

Present Accessed, Dirt

Protection Writable (R#/W) User (U/S#)– 2 levels/type only!

Caching Page WT Page Cache Disabled

3 bit for OS usage

Figure 11-14. Format of Page Directory and Page Table Entries for 4K Pages

0

0 0

Page Frame Address 31:12 AVAIL 0 0 APCD

PWT

U W P

PresentWritableUserWrite-ThroughCache DisableAccessedPage Size (0: 4 Kbyte)Available for OS Use

Page DirEntry

04 12357911 681231

Page Frame Address 31:12 AVAIL D APCD

PWT

U W P

PresentWritableUserWrite-ThroughCache DisableAccessedDirtyAvailable for OS Use

Page TableEntry

04 12357911 681231

Reserved by Intel for future use (should be zero)

-

-


x86 Paging - Virtual memory A page can be

– Not yet loaded– Loaded– On disk

A loaded page can be– Dirty– Clean

When a page is not loaded (P bit clear) => Page fault occurs– It may require throwing a loaded page to insert the new one

OS prioritize throwing by LRU and dirty/clean/avail bits Dirty page should be written to Disk. Clean need not.

– New page is either loaded from disk or “initialized”– CPU will set page “access” flag when accessed, “dirty” when written


Backup


Inverted Page Tables

V.Page P. FramehashVirtualPage

=

IBM System 38 (AS400) implements 64-bit addresses.

48 bits translated

start of object contains a 12-bit tag

=> TLBs or virtually addressed caches are critical


Hardware / Software Boundary What aspects of the Virtual → Physical Translation is determined in

hardware? TLB Format Type of Page Table Page Table Entry Format Disk Placement Paging Policy


Why virtual memory? Generality

– ability to run programs larger than size of physical memory Storage management

– allocation/deallocation of variable sized blocks is costly and leads to (external) fragmentation

Protection– regions of the address space can be R/O, Ex, . . .

Flexibility– portions of a program can be placed anywhere, without relocation

Storage efficiency– retain only most important portions of the program in memory

Concurrent I/O– execute other processes while loading/dumping page

Expandability– can leave room in virtual address space for objects to grow.

Performance

Documents

VMLihu Rappoport, 12/2004 1 MAMAS – Computer Architecture Virtual Memory Dr. Lihu Rappoport