Virtual Memory Modern Operating systems can run programs that

require more memory than the system has If your CPU is 32-bit, meaning that it has registers

that are 32-bits, you can access up to 4G Which means you would need 4Gb of RAM in order

to take advantage of this Although many systems are currently available

with 512MB Usually memory requirements of the programs

you are running, reach far beyond the physical memory you have.

Usually we don't notice any performance problems.

So, how is this possible?

1Virtual Memory

Virtual Memory

• To solve this problem OS uses something called virtual memory

• It is virtual because it can use more that you actually have.

• In fact, with virtual memory you can use the whole 232 bytes.

• Basically, what this means is that you can run more programs at once without the need for buying more memory.

• E.g, in Linux OS if you have more data than physical memory, the system store it temporarily on the hard disk if not needed at the moment.

• Process of moving data to and from the disk is called swapping.

2Virtual Memory

Main Idea

• All memory transfers are only between consecutive levels (e.g. VM to main memory, main memory to cache).

Virtual Memory

(residing on disk)

Main Memory

System Cache

Is cached by

Is cached by

3Virtual Memory

Cache vs. VM

• Concept behind VM is almost identical to concept behind cache.

• But different terminology!– Cache: Block VM: Page

– Cache: Cache Miss VM: Page Fault

• Caches implemented completely in hardware.• VM implemented in software, with hardware support from

CPU.• Cache speeds up main memory access, while main memory

speeds up VM access.

4Virtual Memory

Virtual Memory Design

• Main Memory at Virtual Memory are both divided into fixed size pages.– Page size is typically about 16KB to 32KB.

– Large page sizes are needed as these can be more efficiently transferred between main memory and virtual memory.

– Size of physical pagephysical page ALWAYS equal to size of virtual pagevirtual page.

• Pages in main memory are given physical page numbersphysical page numbers, while pages in virtual memory are given virtual page numbersvirtual page numbers. – I.e. First 32KB of main memory is physical page 0, 2nd 32KB is

physical page 1 etc.

– First 32KB of virtual memory is virtual page 0, etc.

5Virtual Memory

Virtual Memory Design

• In cache, we can search through all the blocks until we find the data for the address we want.– This is because the number of blocks is small.

• This is extremely impractical for virtual memory!– The number of VM pages is in the tens of thousands!

6Virtual Memory

Solution

• Use a look up table.• The addresses generated by the CPU is called the virtual

address.

• The virtual address is divided into a page offset and a virtual page number:

Virtual Page Number Page Offset

• The virtual page number indicates which page of virtual memory the data that the CPU needs is in.

7Virtual Memory

Solution…cont

• The data must also be in physical memory before it can be used by the CPU!

• Need a way to translate between the virtual page number where the data is in VM, to the page number of the physical page where the data is in physical memory.

• To do this, use Virtual Page Table.– Page TablePage Table resides in main memory.

– One entry per virtual page. Can get VERY large as the number of virtual pages can be in the tens of thousands.

8Virtual Memory

Virtual Page Table

1. Gives the physical page (frame) # of a virtual page, if that page is in memory.

2. Gives location on disk if virtual page is not yet in main memory.

VM (on Disk Space)

page0

page1

page2

page3

page4

page5

Virtual Memory Table

frame0

frame1

frame2

frame3

Physical Memory

9Virtual Memory

Page Table Contents

• The page table also contains a Valid Bit (V) to indicate if the

virtual page is in main memory (V=1) or still on disk (V=0).

1

(2,1,7)0

2

(7,2,9)1

01

1

031

page0page1

page2page3page4

page5

• If a page is in physical memory (V=1), then the page table gives the Frame #.

• Otherwise it gives the location of the page on disk , in

the form (side#, track#, block#). 10Virtual Memory

Accessing Data

• To retrieve data:1. Extract the Virtual Page Number from the Virtual

Address

Virtual Page Number (e.g. 02) Page Offset

Virtual Page Number (e.g. 02) Page Offset

11Virtual Memory

Accessing Data

2. Use the page to look up the page table. If V=1, get the frame from the page table:

1

(2,1,7)0

2

(7,2,9)1

01

1

031

page0page1

page2page3page4

page5

page = 2frame=0

Here virtual page#2 mapped to frame#0.

12Virtual Memory

Accessing Data

3. Combine the frame found with the page offset to form the physical memory address:

Physical Page Number 0 Page Offset

Phyiscal Page Number 0 Page Offset

Physical Address

13Virtual Memory

Accessing Data

4. Access main memory using the physical address.

– A page consists of many bytes (e.g. 32KB)

– The page offset tells us exactly which byte of these 32KB we are accessing.

• Similar to the idea of block offset and byte offset in caches

14Virtual Memory

Page Fault

• What if the page we want is not in main memory yet?What if the page we want is not in main memory yet?

1. In this case, V=0, and the page table contains the disk address of the page (e.g. page1 in the previous example is still at side 2, track 1, block 7 (2,1,7) of the disk.

2. Find a free physical page

- if none are available, apply a replacement policy (e.g. LRU) to find one.

3. Load the virtual page into the physical page.

- Set the V flag, and update the page table to show which physical page the virtual page has gone to.

15Virtual Memory

Behavior of Page Replacement Algorithms

16Virtual Memory

DiskDisk

Example: 2 blocks cache, 4bytes/block

VA space=8 pages &32 Byte/page VA= 3bits(page#) + 5-bit(offset)

load 32

load 40load 00101 0 00C-MissC-Miss

C-MissC-Miss

Index Valid Tag Data

0 N

1 NY 00100 Memory[000100 0 00]

load 00100 0 00

Page Index Valid Frame number

0 N

1 N

... NY 111

001 00000

Virtual Page Number Page offset

MemoryMemory

Y 00101 Memory[000101 0 00]

P-FaultP-Fault

P- Hit!P- Hit!

001 01000

17Virtual Memory

Example

Suppose we have

32 bit virtual address (232 bytes) 4096 bytes per page (212 bytes)4 bytes per page table entry (22 bytes)

What is the total page table size?

entries22

2entriestablepageofNumber 20

12

32

MB4bytes2bytes2entries2tablepageofSize 22220

4 Megabytes just for page tables!! Too Big 18Virtual Memory

Writing to VM

• Writes to Virtual Memory is always done on a write-back basis.

• To support write-back, the page-table must be augmented with a dirty-bit (D).

• This bit is set if the page is updated in physical memory.

19Virtual Memory

Writing to VM

• Here virtual page#2page#2 was updated in physical frame#0frame#0.

• If frame#0frame#0 is ever replaced, its contents must be written back to disk to update page#2page#2.

1

(2,1,7)0

2

(7,2,9)1

01

1

031

page0page1

page2page3page4

page5

10

01

0

00

1

frame or disk locationVD

20Virtual Memory

Translation Look-aside Buffer• An access to virtual memory requires 2 main memory accesses at

best.– One access to read the page table, another to read the data.

• Remember from the Cache section that main memory is slow

• Fortunately, page table accesses themselves tend to display both temporal and spatial locality!– Temporal Locality: Accesses to the different words in the same page will cause

access to same entry in page table!

– Spatial Locality: Sequential access of data from one virtual page into the next will cause consecutive accesses to page table entries.

• Initially I am at page0, and I access Page Table entry for page0. As I move into page1, I will access Page Table entry for page1, which is next to page table entry for page0!

21Virtual Memory

Translation Look-aside Buffer

• Solution:– Implement a cache for the page table! This cache is called the

translation look-aside buffer, or TLB.

– The TLB is separate from the caches we were looking at earlier.• Those caches cached data from main memory.

• The TLB caches page table entries! Different!

– TLB is small (about 8 to 10 blocks), and is implemented as a fully associative cache.

22Virtual Memory

Translation Look-aside Buffer

• Fully Associative– New page table entries go into the next free TLB block,

or a block is replaced if there are none.

• Note that only page table entries with V=1 are written to the TLB!

• The page table entries already in the TLB are not usually updated, so no need to consider write-through or write-back– Exceptional cases: page aliasing, where more than 1

page can refer to the same Physical Page.

23Virtual Memory

Translation Look-aside Buffer

• The tags used in the TLB is the virtual page number of a virtual address.

• All TLB blocks are searched for the page. If found, we have a TLB hit and the physical page number is read from the TLB. This is joined with the page offset to form the physical address.

• If not found, we have a TLB miss. Then we must go to the page table in main memory to get the page table entry there. Write this entry to TLB.

24Virtual Memory

Translation Look-aside Buffer

• Complication– If we have a TLB miss and go to main memory to get the page

table entry, it is possible that this entry has a V of 0 - page fault.

– In this case we must remedy the page fault first, update the page table entry in main memory, and then copy the page table entry into TLB. The tag portion of TLB is updated to the page of the virtual address.

• Note that the TLB must also have a valid bit V to indicate if the TLB entry is valid (see cache section for more details on the V bit.)

25Virtual Memory

Integration Cache, Main Memory and Virtual Memory

• Suppose a Virtual Address V is generated by the CPU (either from PC for instructions, or from ALU for lw and sw instructions).1. Perform address translation from Virtual Address to Physical

Address

(a) Look up TLB or page table (see previous slides). Remedy page fault if necessary (again, see previous slides).

2. Use the physical address to access the cache (see cache notes).

3. If cache hit, read the data (or instruction) from the cache.

4. If cache miss, read the data from main memory.

26Virtual Memory

Use of a Translation Lookaside Buffer

27Virtual Memory

Integration Cache, Main Memory and Virtual Memory

• Note that a page-fault in VM will necessarily cause a cache miss later on (since the data wasn’t in physical memory, it cannot possibly be in cache!)

• Can optimize algorithm in event of page fault:1. Remedy the page fault.

2. Copy the data being accessed directly to cache.

3. Restart previous algorithm at step 3.

• This optimization eliminates 1 unnecessary cache access that would definitely miss.

28Virtual Memory

Page Table Size

• A Virtual Memory System was implemented for a MIPS workstation with 128MB of main memory. The Virtual Memory size is 1GB, and each page is 32KB. Calculate the size of the page table.

29Virtual Memory

Page Table Size

• Previous calculation shows that page tables are huge!

• These are sitting in precious main memory space.• Solutions:

– Use inverted page tables• Instead of indexing virtual pages, index physical pages.

• Page table will provide virtual page numbers instead.

• Search page table for the page of address virtual address V. If the page is found in entry 25, then the data can be found in physical page 25.

– Have portions of page table in virtual memory.• Slow, complex

30Virtual Memory

Finer Points of VM

• VM is a collaboration between hardware and OS– Hardware:

• TLB

• Page Table Register– Indicates where the page table is in main memory

• Memory Protection– Certain virtual pages are allocated to processes running in memory.

– If one process tries to access the virtual page of another process without permission, hardware will generate exception.

– This gives the famous “General Protection Fault” of windoze and the “Segmentation Fault” of Unix.

31Virtual Memory

Finer Points of VM

– Hardware• Does address translations etc.

– Operating System• Actually implements the virtual memory system.

– Does reads and writes to/from disk

– Creates the page table in memory, sets the Page Table Register to point to the start of the page table.

– Remedies page faults,updates the page table.

– Remedies VM violations

» Windows: Pops up blue screen of death, dies messily. Sometimes thrashes your hard-disk.

» Unix: Gives “Segmentation Fault”. Kills offending process and continues working.

32Virtual Memory

Finer Points of VM

• Where is the Virtual Memory located on disk?– Virtual memory is normally implemented as a very large file,

created by the OS. E.g. in Windows NT, the virtual memory file is called swapfile.sys

• Insecure. Sometimes sensitive info gets written to swapfile.sys, and you can later retrieve the sensitive info.

• In Unix, implemented as a partition on the disk that cannot be read except by the OS. Unix good. Windows bad.

– Whenever virtual memory is read or written to, the OS actually reads or writes from/to this file.

• Virtual Memory is NOT the other files on your disk (e.g. your JAVA assignment)

33Virtual Memory

Page Tables

Page offsetVirtual page number

Virtual address

Page offsetPhysical page number

Physical address

Physical page numberValid

If 0 then page is notpresent in memory

Page table register

Page table

20 12

18

31 30 29 28 27 15 14 13 12 11 10 9 8 3 2 1 0

29 28 27 15 14 13 12 11 10 9 8 3 2 1 0

34Virtual Memory

Making Address Translation Fast• A cache for address translations: translation lookaside buffer

Valid

1

1

1

1

0

1

1

0

1

1

0

1

Page table

Physical pageaddressValid

TLB

1

1

1

1

0

1

TagVirtual page

number

Physical pageor disk address

Physical memory

Disk storage

35Virtual Memory

TLBs and caches

Yes

Deliver datato the CPU

Write?

Try to read datafrom cache

Write data into cache,update the tag, and put

the data and the addressinto the write buffer

Cache hit?Cache miss stall

TLB hit?

TLB access

Virtual address

TLB missexception

No

YesNo

YesNo

Write accessbit on?

YesNo

Write protectionexception

Physical address

36Virtual Memory