Computer Architecture and Engineering Lecture 15 Virtual

UC Regents Fall 2005 © UCBCS 152 L15: Virtual Memory

2005-10-20John Lazzaro

(www.cs.berkeley.edu/~lazzaro)

CS 152 Computer Architecture and Engineering

Lecture 15 – Virtual Memory

www-inst.eecs.berkeley.edu/~cs152/

TAs: David Marquardt and Udam Saini


Last Time: Practical Cache Design

ToCPU

ToLowerLevelMemory

ToCPU

ToLowerLevelMemory

TagsBlocks

Addr

Din

Dout

Addr

Din

Dout

State Machine

Control

Control Control

Cache design control is done by many loosely coupled state machines, including ...


State machines for bus control ....Lower Level

MemoryUpper Level

MemoryTo Processor

From Processor

Blk X

Blk Y

Small, fast Large, slow

FromCPU

To CPU

For reads,your state machine must: (1) sense REQ(2) latch Addr(3) create Wait(4) put Data Out on the bus.

An example interface ... there are other possibilities.


State machines for block fetch from DRAM

20256Mb: x4, x8, x16 SDRAM Micron Technology, Inc., reserves the right to change products or specif icat ions without not ice.

256MSDRAM_G.p65 – Rev. G; Pub. 9/03 ©2003, Micron Technology, Inc.

256M b: x4, x8, x16SDRAM

va lid , wh ere x eq u a ls th e CAS la ten cy m in u s on e .

Th is is shown in Figure 13 for CAS laten cies of two an d

th ree; data elem en t n + 3 is either the last of a burst of

four or the last desired of a lon ger burst. The 256Mb

SDRAM u ses a p ip elin ed arch itectu re an d th erefore

does n ot requ ire the 2n ru le associated with a p refetch

arch itectu re. A READ com m an d can be in itiated on an y

clock cycle followin g a p revious READ com m an d. Fu ll-

speed ran dom read accesses can be perform ed to the

sam e ban k, as shown in Figure 14, or each subsequen t

READ m ay be perform ed to a d ifferen t ban k.

Figure 13: Consecut ive READ Bursts

DON’T CARE

NOTE: Each READ command may be to any bank. DQM is LOW.

CLK

DQDOUT

n

T2T1 T4T3 T6T5T0

COMMAND

ADDRESS

READ NOP NOP NOP NOP

BANK,COL n

NOP

BANK,COL b

DOUT

n + 1DOUT

n + 2DOUT

n + 3DOUT

b

READ

X = 1 cycle

CAS Latency = 2

CLK

DQDOUT

n

T2T1 T4T3 T6T5T0

COMMAND

ADDRESS

READ NOP NOP NOP NOP

BANK,COL n

NOP

BANK,COL b

DOUT

n + 1DOUT

n + 2DOUT

n + 3DOUT

b

READ NOP

T7

X = 2 cycles

CAS Latency = 3

TRANSITIONING DATA

One request ...

Many returns ...

DRAM can be set up to request an N byte region starting at an arbitrary N+k within region

State machine challenges: (1) setting up correct block read mode (2) delivering correct word direct to CPU (3) putting all words in cache in right place.


53256Mb: x4, x8, x16 SDRAM Micron Technology, Inc., reserves the right to change products or specif icat ions without not ice.

256MSDRAM_G.p65 – Rev. G; Pub. 9/03 ©2003, Micron Technology, Inc.

256M b: x4, x8, x16SDRAM

Figure 47: Write – With Auto Precharge1

NOTE: 1. For this example, the burst length = 4.

2. x16: A9, A11, and A12 = “ Don’t Care”

x8: A11 and A12 = “ Don’t Care”

x4: A12 = “ Don’t Care”

* CAS latency indicated in parentheses.

-7E -75

SYM BOL* M IN M AX M IN M AX UNITStCMS 1.5 1.5 nstDH 0.8 0.8 nstDS 1.5 1.5 nstRAS 37 120,000 44 120,000 nstRC 60 66 nstRCD 15 20 nstRP 15 20 nstWR 1 CLK + 1 CLK + –

7ns 7.5ns

TIM ING PARAM ETERS

-7E -75

SYM BOL* M IN M AX M IN M AX UNITStAH 0.8 0.8 nstAS 1.5 1.5 nstCH 2.5 2.5 nstCL 2.5 2.5 nstCK (3) 7 7.5 nstCK (2) 7.5 10 nstCKH 0.8 0.8 nstCKS 1.5 1.5 nstCMH 0.8 0.8 ns

ENABLE AUTO PRECHARGE

tCH

tCLtCK

tRP

tRAS

tRCD

tRC

DQM/DQML, DQMU

CKE

CLK

A0-A9, A11, A12

DQ

BA0, BA1

A10

tCMHtCMS

tAHtAS

ROW

ROW

BANK BANK

ROW

ROW

BANK

tWR

DON’T CARE

DIN m

tDHtDS

DIN m + 1 DIN m + 2 DIN m + 3

COMMAND

tCMHtCMS

NOPNOP NOPACTIVE NOP WRITE NOP ACTIVE

tAHtAS

tAHtAS

tDHtDS tDHtDS tDHtDS

tCKHtCKS

NOP NOP

COLUMN m2

T0 T1 T2 T4T3 T5 T6 T7 T8 T9

State machine for writeback to DRAM

One command ...

Many bytes

written

State machine challenges: (1) putting cache block into correct location (2) what if a read or write wants to use DRAM before the burst is complete? Must stall ...


State machines to manage write buffer

Q. Why a write buffer ?

ProcessorCache

Write Buffer

Lower Level

Memory

Holds data awaiting write-through to lower level memory

A. So CPU doesn’t stall Q. Why a buffer, why not just one register ?

A. Bursts of writes arecommon.

Q. Are Read After Write (RAW) hazards an issue for write buffer?

A. Yes! Drain buffer before next read, or check write buffers.

Solution: add a “write buffer” to cache datapath

On reads, state machine checks cache and write buffer -- what if word was removed from cache before lower-level

write? On writes, state machine stalls for full write buffer, handles write buffer duplicates.


Don’t design one big state machine!!!

ToCPU

ToLowerLevelMemory

ToCPU

ToLowerLevelMemory

TagsBlocks

Addr

Din

Dout

Addr

Din

Dout

State Machine

Control

Control Control

Focus on the high-level state machine structure early!


Today’s Lecture - Virtual Memory

Virtual address spaces

Page table layout

TLB design options


The Limits of Physical Addressing

CPU Memory

A0-A31 A0-A31

D0-D31 D0-D31

“Physical addresses” of memory locations

Data

All programs share one address space: The physical address space

No way to prevent a program from accessing any machine resource

Machine language programs must beaware of the machine organization

Where we are in CS 152 ...

UC Regents Fall 2005 © UCBCS 152 L13: Cache I

Apple II: A physically addressed machine

Apple ][ (1977)

Steve WozniakSteve

Jobs

CPU: 1000 ns DRAM: 400 ns


The Limits of Physical Addressing

CPU Memory

A0-A31 A0-A31

D0-D31 D0-D31

“Physical addresses” of memory locations

Data

All programs share one address space: The physical address space

No way to prevent a program from accessing any machine resource

Machine language programs must beaware of the machine organization

Programming the Apple ][ ...


Solution: Add a Layer of Indirection

CPU Memory

A0-A31 A0-A31

D0-D31 D0-D31

Data

User programs run in an standardizedvirtual address space

Address Translation hardware managed by the operating system (OS)

maps virtual address to physical memory

“Physical Addresses”

AddressTranslation

Virtual Physical

“Virtual Addresses”

Hardware supports “modern” OS features:Protection, Translation, Sharing


MIPS R4000: Address Space ModelProcess A

AddressError

2 GB

0

231

232

- 1

Process B

AddressError

2 GB

0

231

232

- 1

Process A and B have independent address spaces

ASID = 12 ASID = 13

ASID = Address Space Identifier

When Process A writes its address 9, it writes to a different physical memory location than

Process B’s address 9

May only be accessed by kernel/supervisor

To let Process A and B share memory, OS maps parts of

ASID 12 and ASID 13 to the same physical memory locations.

All address spacesuse a standard memory map

Still works (slowly!) if a process accesses more virtual memory than the machine has physical memory


MIPS R4000: Who’s Running on the CPU?System Control Registers

Chapter 4

80 MIPS R4000 Microprocessor User' s Manual

4.3 System Control Coprocessor

The System Control Coprocessor (CP0) is implemented as an integral part

of the CPU, and supports memory management, address translation,

exception handling, and other privileged operations. CP0 contains the

registers shown in Figure 4-7 plus a 48-entry TLB. The sections that follow

describe how the processor uses the memory management-related

registers†.

Each CP0 register has a unique number that identifies it; this number is

referred to as the register number. For instance, the Page Mask register is

register number 5.

Figure 4-7 CP0 Registers and the TLB

† For a description of CP0 data dependencies and hazards, please see Appendix F.

EntryLo02*

EntryHi

Page Mask

Index

Random

Wired

Count

47

0

BadVAddr

TLB

(“Safe” entries)

(See Random Register,

PRId

0127

8*

15*

Compare

11*

Config

16*

LLAddr

17*

WatchLo

18*

WatchHi

19*

TagLo

28*TagHi

29*

contents of TLB Wired)ECC

26*

*Register number

Used with exception

processing. SeeUsed with memory

Chapter 5 for details.

EntryLo0

2*

3*

EntryLo1

EntryHi

10*

5*Page Mask

Index

0*

Random1*

Wired6*

ErrorEPC

30*

Context

4*

Status

12*Cause

13*

EPC14*

management system.

CacheErr

27*

XContext

20*

9*

User cannot write supervisor/kernel bits. Supervisor cannot write kernel bit.

User cannot change address translation configuration

Status (12): Indicatesuser, supervisor, or

kernel mode

EntryLo0 (2): 8-bit ASID field codes virtualaddress space ID.


MIPS Address Translation: How it works“Physical Addresses”

CPU Memory

A0-A31 A0-A31

D0-D31 D0-D31

Data

TLB also contains ASID andkernel/supervisor bits for virtual address

Virtual Physical


TranslationLook-Aside

Buffer(TLB)

Translation Look-Aside Buffer (TLB)A small fully-associative cache of

mappings from virtual to physical addresses

Fast common case: Virtual address is in TLB, process has permission to read/write it.

What is the table

ofmappings

that it caches?


Page tables encode virtual address spaces

A machine usually supports

pages of a few sizes

(MIPS R4000):

MIPS R4000 Microprocessor User' s Manual 87

Memory Management

EntryLo0 (2), and EntryLo1 (3) Registers

The EntryLo register consists of two registers that have identical formats:

• EntryLo0 is used for even virtual pages.

• EntryLo1 is used for odd virtual pages.

The EntryLo0 and EntryLo1 registers are read / write registers. They hold

the physical page frame number (PFN) of the TLB entry for even and odd

pages, respectively, when performing TLB read and write operations.

Figure 4-10 shows the format of these registers.

PageMask Register (5)

The PageMask register is a read / write register used for read ing from or

writing to the TLB; it holds a comparison mask that sets the variable page

size for each TLB entry, as shown in Table 4-9.

TLB read and write operations use this register as either a source or a

destination; when virtual addresses are presented for translation into

physical address, the corresponding bits in the TLB identify which virtual

address bits among bits 24:13 are used in the comparison. When the Mask

field is not one of the values shown in Table 4-9, the operation of the TLB

is undefined .

Table 4-9 Mask Field Values for Page Sizes

Page SizeBi t

24 23 22 21 20 19 18 17 16 15 14 13

4 Kbytes 0 0 0 0 0 0 0 0 0 0 0 0

16 Kbytes 0 0 0 0 0 0 0 0 0 0 1 1

64 Kbytes 0 0 0 0 0 0 0 0 1 1 1 1

256 Kbytes 0 0 0 0 0 0 1 1 1 1 1 1

1 Mbyte 0 0 0 0 1 1 1 1 1 1 1 1

4 Mbytes 0 0 1 1 1 1 1 1 1 1 1 1

16 Mbytes 1 1 1 1 1 1 1 1 1 1 1 1

PhysicalMemory Space

A valid page table entry codes physical memory “frame” address for the page

A virtual address spaceis divided into blocks

of memory called pagesframeframe

frame

frame

A page table is indexed by a virtual address

virtual address

Page Table(One per ASID)

OS manages the page table for each ASID


4/19/04 ©UCB Spring 2004CS152 / Kubiatowicz

Lec21.21

CPU Registers100s Bytes<10s ns

CacheK Bytes10-100 ns$.01-.001/bit

Main MemoryM Bytes100ns-1us$.01-.001

DiskG Bytesms10 - 10 cents

-3 -4

CapacityAccess TimeCost

Tapeinfinitesec-min10-6

Registers

Cache

Memory

Disk

Tape

Instr. Operands

Blocks

Pages

Files

StagingXfer Unit

prog./compiler1-8 bytes

cache cntl8-128 bytes

OS512-4K bytes

user/operatorMbytes

Upper Level

Lower Level

faster

Larger

Recall: Levels of the Memory Hierarchy


Lec21.22

° Virtual memory => treat memory as a cache for the disk° Terminology: blocks in this cache are called “Pages”

• Typical size of a page: 1K — 8K

° Page table maps virtual page numbers to physical frames• “PTE” = Page Table Entry

Physical Address Space

Virtual Address Space

What is virtual memory?

Virtual Address

Page Table

indexintopagetable

Page TableBase Reg

VAccessRights PA

V page no. offset10

table locatedin physicalmemory

P page no. offset

10

Physical Address


Lec21.23

Three Advantages of Virtual Memory

° Translation:• Program can be given consistent view of memory, even

though physical memory is scrambled

• Makes multithreading reasonable (now used a lot!)

• Only the most important part of program (“Working Set”) must be in physical memory.

• Contiguous structures (like stacks) use only as much physical memory as necessary yet still grow later.

° Protection:• Different threads (or processes) protected from each other.

• Different pages can be given special behavior- (Read Only, Invisible to user programs, etc).

• Kernel data protected from User programs

• Very important for protection from malicious programs=> Far more “viruses” under Microsoft Windows

° Sharing:• Can map same physical page to multiple users

(“Shared memory”) 4/19/04 ©UCB Spring 2004CS152 / Kubiatowicz

Lec21.24

What is the size of information blocks that are transferred from secondary to main storage (M)? ⇒ page size(Contrast with physical block size on disk, I.e. sector size)

Which region of M is to hold the new block ⇒ placement policy

How do we find a page when we look for it? ⇒ block identification

Block of information brought into M, and M is full, then some region of M must be released to make room for the new block ⇒ replacement policy

What do we do on a write? ⇒ write policy

Missing item fetched from secondary memory only on the occurrence of a fault ⇒ demand load policy

pages

reg

cachemem disk

frame

Issues in Virtual Memory System Design

V=0 pages either reside on disk or

have not yet been allocated.

OS handles V=0“Page fault”

In this example,physical and virtual

pages must be the same size!

The TLB caches page table entries

TLB

Page Table

2

0

1

3

virtual address

page off

2frame page

250

physical address

page off

TLB caches page table

entries.

MIPS handles TLB misses in software (random replacement). Other

machines use hardware.

for ASID

Physicalframe

address


Page tables may not fit in memory!A table for 4KB pages for a 32-bit

address space has 1M entries Each process needs its own address space!

P1 index P2 index Page Offset

31 12 11 02122

32 bit virtual address

Top-level table wired in main memory

Subset of 1024 second-level tables in main memory; rest are on disk or

unallocated

Two-level Page Tables


Lec21.25

How big is the translation (page) table?

° Simplest way to implement “fully associative” lookup policy is with large lookup table.

° Each entry in table is some number of bytes, say 4

° With 4K pages, 32- bit address space, need:232/4K = 220 = 1 Meg entries x 4 bytes = 4MB

° With 4K pages, 64-bit address space, need:264/4K = 252 entries = BIG!

° Can’t keep whole page table in memory!

Virtual Page Number Page Offset


Lec21.26

Large Address Spaces

Two-level Page Tables

32-bit address:

P1 index P2 index page offest

4 bytes

4 bytes

4KB

10 10 12

1KPTEs

° 2 GB virtual address space

° 4 MB of PTE2

– paged, holes

° 4 KB of PTE1

What about a 48-64 bit address space?


Lec21.27

Inverted Page Tables

V.Page P. FramehashVirtual

Page

=

IBM System 38 (AS400) implements 64-bit addresses.

48 bits translated

start of object contains a 12-bit tag

=> TLBs or virtually addressed caches are critical


Lec21.28

Virtual Address and a Cache: Step backward???

° Virtual memory seems to be really slow:

• Must access memory on load/store -- even cache hits!

• Worse, if translation not completely in memory, may need to go to disk before hitting in cache!

° Solution: Caching! (surprise!)

• Keep track of most common translations and place them in a “Translation Lookaside Buffer” (TLB)

CPUTrans-lation

CacheMain

Memory

VA PA miss

hit

data



Lec21.21

CPU Registers100s Bytes<10s ns

CacheK Bytes10-100 ns$.01-.001/bit

Main MemoryM Bytes100ns-1us$.01-.001

DiskG Bytesms10 - 10 cents

-3 -4

CapacityAccess TimeCost

Tapeinfinitesec-min10-6

Registers

Cache

Memory

Disk

Tape

Instr. Operands

Blocks

Pages

Files

StagingXfer Unit

prog./compiler1-8 bytes

cache cntl8-128 bytes

OS512-4K bytes

user/operatorMbytes

Upper Level

Lower Level

faster

Larger

Recall: Levels of the Memory Hierarchy


Lec21.22

° Virtual memory => treat memory as a cache for the disk° Terminology: blocks in this cache are called “Pages”

• Typical size of a page: 1K — 8K

° Page table maps virtual page numbers to physical frames• “PTE” = Page Table Entry

Physical Address Space

Virtual Address Space

What is virtual memory?

Virtual Address

Page Table

indexintopagetable

Page TableBase Reg

VAccessRights PA

V page no. offset10

table locatedin physicalmemory

P page no. offset

10

Physical Address


Lec21.23

Three Advantages of Virtual Memory

° Translation:• Program can be given consistent view of memory, even

though physical memory is scrambled

• Makes multithreading reasonable (now used a lot!)

• Only the most important part of program (“Working Set”) must be in physical memory.

• Contiguous structures (like stacks) use only as much physical memory as necessary yet still grow later.

° Protection:• Different threads (or processes) protected from each other.

• Different pages can be given special behavior- (Read Only, Invisible to user programs, etc).

• Kernel data protected from User programs

• Very important for protection from malicious programs=> Far more “viruses” under Microsoft Windows

° Sharing:• Can map same physical page to multiple users

(“Shared memory”) 4/19/04 ©UCB Spring 2004CS152 / Kubiatowicz

Lec21.24

What is the size of information blocks that are transferred from secondary to main storage (M)? ⇒ page size(Contrast with physical block size on disk, I.e. sector size)

Which region of M is to hold the new block ⇒ placement policy

How do we find a page when we look for it? ⇒ block identification

Block of information brought into M, and M is full, then some region of M must be released to make room for the new block ⇒ replacement policy

What do we do on a write? ⇒ write policy

Missing item fetched from secondary memory only on the occurrence of a fault ⇒ demand load policy

pages

reg

cachemem disk

frame

Issues in Virtual Memory System Design

V=0 pages either reside on disk or

have not yet been allocated.

OS handles V=0“Page fault”

What if a page resides on disk?

TLB

Page Table

2

0

1

3

virtual address

page off

2frame page

250

physical address

page off

TLB caches page table

entries.

Question: What to do when a TLB miss causes an access to a

page table entry with V=0?

for ASID

Physicalframe

address


VM and Disk: Page replacement policy

...

Page Table

1 0useddirty

1 00 11 10 0

Set of all pagesin Memory Tail pointer:

Clear the usedbit in thepage table

Head pointerPlace pages on free list if used bitis still clear.Schedule pages with dirty bit set tobe written to disk.

Freelist

Free Pages

Dirty bit: page written.

Used bit: set to1 on any reference

Architect’s role: support setting dirty and used bits

On page fault: deallocate page table entry of a page

on the free list.

UC Regents Fall 2005 © UCBCS 152 L16: Error Correcting Codes

Friday: Design document deadlines

UC Regents Spring 2005 © UCBCS 152 L8: Pipelining I

Instruction Cache

Data Cache

DRAM

D

R

A

M

C

o

n

t

r

o

l

l

e

r

P

i

p

e

l

i

n

e

d

C

P

U

IC Bus IM Bus

DC Bus DM Bus

Define the timing diagrams and signal names for the IM, DM, IC, DC buses.

List the bugs you will target in test benches.

Other items ...Also: Lab 3 Peer Evaluations ...


TLB Design Concepts


MIPS R4000 TLB: A closer look ...“Physical Addresses”

CPU MemorySystem

A0-A31 A0-A31

D0-D31 D0-D31

Data


Buffer(TLB)

Virtual Physical


MIPS R4000 Microprocessor User' s Manual 65

Memory Management

32-bit Mode Address Translation

Figure 4-2 shows the virtual-to-physical-address translation of a 32-bit

mode address.

• The top portion of Figure 4-2 shows a virtual add ress w ith a

12-bit, or 4-Kbyte, page size, labelled Offset. The remaining 20

bits of the add ress represent the VPN, and index the 1M-entry

page table.

• The bottom portion of Figure 4-2 shows a virtual add ress w ith

a 24-bit, or 16-Mbyte, page size, labelled Offset. The remaining

8 bits of the add ress represent the VPN, and index the 256-

entry page table.

Figure 4-2 32-bit Mode Virtual Address Translation

28 11 0

20 12

2931

VPN Offset

3239

ASID

8

Virtual Address with 1M (220) 4-Kbyte pages

23 0

8 24

Offset

39

Virtual Address with 256 (28)16-Mbyte pages

8 bits = 256 pages

20 bits = 1M pages 12

ASID

8

28 293132

VPN

24

Virtual-to-physicaltranslation in TLB

Bits 31, 30 and 29 of the virtualaddress select user, supervisor,or kernel address spaces.

Offset passedunchanged tophysicalmemory

Virtual-to-physicaltranslation in TLB

TLB

TLB

35 0

PFN Offset

Offset passedunchanged tophysicalmemory

36-bit Physical Address

Physical space larger than virtual space!Checked

againstCPO ASID


Can TLB and caching be overlapped?

Index Byte Select

Valid

Cache Block

Cache Block

Cache Tags Cache Data

Data out

Virtual Page Number Page Offset


Buffer(TLB)

Virtual

Physical

=

Hit

Cache Tag

This works, but ...

Q. What is the downside?

A. Inflexibility. VPN size locked to cache tag size.


Can we cache virtual addresses?

“Physical Addresses”

CPU Main Memory

A0-A31 A0-A31

D0-D31 D0-D31

Only use TLB on a cache miss !


Buffer(TLB)

Virtual Physical


A. Synonym problem. If two address spaces share a physical frame, data may be in cache twice.

Maintaining consistency is a nightmare.

CacheVirtual

D0-D31

Downside: a subtle, fatal problem. What is it?


Conclusions

VM: Uniform memory models,protection, sharing.

Operating systems manage the page table and (often) the TLB

A TLB acts as a fast cache forrecent address translations.

UC Regents Fall 2005 © UCBCS 152 L16: Error Correcting Codes

Detecting and correcting RAM bit errors

Replacing lost network packets, recovering from disk drive failure

Detecting arbitrary bit errors in network packets

Tuesday: How ECC memory works ...

Documents

Computer Architecture and Engineering Lecture 15 Virtual