FlashVM: Virtual Memory Management on FlashMohit Saxena and Michael M. Swift

IntroductionFlash storage is the largest change to memory and storage systems in decades. In addition to serving a fast, low-power disk replacement or accelerator, flash devices can also be used for virtual memory to handle large workloads cheaply. FlashVM is a system architecture and extension to the Linux VM subsystem pursuing this challenge.

Performance Reliability

Flash Benefits for VM FlashVMThe FlashVM architecture consists of dedicated flash for paging and the FlashVM manager to control Linux’s use of flash.

References[1] Saxena M., and Swift, M. M. FlashVM: Revisiting the Virtual Memory Hierarchy. In HotOS-XII (2009).[2] Agrawal N., Prabhakaran V., Wobber T., Davis J., Manasse M., and Panigraph R. Design Tradeoffs for SSD performance. In USENIX (2008).[3] Park S., Jung D., Kang J., Kim J., and Lee J., CFLRU: A replacement algorithm for flash memory, In CASES (2006).[4] Mogul J. C., Argollo E., Shah M., and Farboschi P., Operating System support for NVM+DRAM hybrid main memory. In HotOS (2009).

FlashVM EnhancementsPerformance Parameter Tuning,

Prefetching

Reliability Page sampling, Page Sharing

Garbage Collection Dummy discard, Merged discard

DiskVM

FlashVM

Memory Size M

Exec

ution

Tim

e T M

Reduced DRAM Same performance Lower system price

T

Faster and scalable No additional DRAM Similar system price

Block Device Driver

DRAMPage Swapping

FlashVM Manager:Performance, Reliability and Garbage Collection

Dedicated FlashCheaper than disk for VMReduced FS interference

Disk

Disk Scheduler

Block Layer

VM Memory Manager

Dedicated MLC NAND

Flash

Unused memory

No locality

FlashVM benefits system operating with moderate paging: FlashVM can improve performance with the same amount of DRAM, by speeding page faults or achieve the same performance with a cheaper mix of Flash and less DRAM leading to more page faults.

Application Performance and Memory Savings

0

10

20

30

40

50

60

70

ImageMagick Spin SpecJBB memcached-store

memcached-lookup

Applications

Per

form

ance

/Mem

ory

Sav

ing

s

Runtime Memory Use

84% memory savings 94% less

execution time

Write Reduction Performance

0

20

40

60

80

100

120

Uniform PageSampling

Adaptive PageSampling

Page Sharing

Write Reduction Technique

Pe

rfo

rma

nc

e/W

rite

s

Performance Writes

14% reduction

93% reduction

Garbage Collection

1

10

100

1000

10000

ImageMagick Spin memcached

Application

Ela

pse

d T

ime

(s)

Linux/Discard FlashVM Linux/No Discard

10X faster15% slower

Write Reduction

Dirty?Dirty

Clean

Inactive LRU Page List

free_pages

Flash

FlashVMSample dirty pages to elide writes

Classify page contentsWrite-back non-zero dirty pages

Free Page List

classify

Non-Zero

Zero

sample

evict

leavein memory

Page Prefetching• Disk VM assumption

- Seek and rotational delays are longer than the transfer cost of extra blocks

• Linux sequential prefetching- Minimize costly disk seeks by

reading only blocks adjacent to request

• FlashVM prefetching- Exploit fast flash random reads

and spatial/temporal locality in reference pattern

- Seek over free and bad blocks

• Current VM systems are optimized for disk performance– Slow random reads– High access and seek costs– Symmetrical read/write performance

• FlashVM retunes VM parameters for Flash:– Page write back: unit of writing– Page scanning: congestion timeouts– Disk scheduling: work-conserving FIFO

• FlashVM aggressively prefetches pages

Discard

Discard

Discard

Discard

swap map

FREE

FREE

BAD

FlashVM

Request

Linux

• Flash chips lose durability after 10,000—100,000 writes– Challenge: reduce the number of writes

• Page sampling: probabilistically leave old, dirty pages in memory instead of writing back– Measure page dirtying rate– Increase sampling probability with low rate,– Decrease with high rate

• Page sharing: avoid storing multiple copies of same data– Marks zero-filled pages in swap map and

avoid writing back– Avoids read/write cost

Garbage Collection• All writes to flash go to new location

– SSD aware of freed page clusters when the blocks are overwritten

• Discard/Trim: notifies SSD that blocks are unused– More free blocks for writing– Avoids copying data for partial overwrites

• Observation: Overwriting a block– notifies SSD it is empty– after discarding it, uses the free space made

available by discard• FlashVM dummy discard

– Monitors rate of allocation– Virtualize discard by reusing blocks likely to

be overwritten soon

• Native Linux discard once per page cluster– Result: 55 ms latency for

freeing 32 pages• FlashVM merged discard

– Defer and batch discard until 100MB of free pages available

– Pages discarded may be non-contiguous

Dummy Discard

Merged Discard