Why 4k?

Why 4K?

October 2, 2012

George Wilson

[email protected]

Delphix Proprietary and Confidential

Why 4K? (Not Y4K)

● This is not the next Millennium bug!


Storage History

● 1998 IBM publishes a paper proposing an increase of disk

sector size from 512B to 4K

● 2000 4K IDEMA (International Disk Drive Equipment and

Materials Association) committee was formed

● 2005 ZFS released in OpenSolaris with support for block sizes

ranging from 512B to 128K

● 2005 512B emulation mode proposed, later known as AF

512e

● 2006 ZFS adds large sector support

● 2009 Advanced Format is approved as naming convention

for 4K sectors

● 2011 All hard drive manufactures start to ship AF 512e drives


● Two flavors of Advanced Format Drives

○ AF 512e - Advanced Format 512B Emulation

○ AF 4Kn - Advanced Format 4K Native

Future(2012?)

Today

Advanced Format Drives


● Maps 8 512B logical blocks

into 1 physical 4K block

● Provides an emulation layer

for compatibility

Advanced Format 512e (AF 512e)

0 1 2 3 4 5 6 7

4K Physical Block #0 4K Physical Blocks

512B Logical Blocks

"Access on a 512-byte basis would continue to be

supported, but performance would be inferior to

that in which access is done on a 4096-byte basis,

and might well be inferior to that of previous

drives with 512-byte native block size." -- Large

Block Size by Paul Hodges and David Cheng, 1998

Predicting Future Problems


Problems in a 4K World

● Lies

○ AF 512e Drive lie about their physical block size

○ LUNs from storage vendors lie about their physical block

size

● Misaligned I/O

○ Proper partitioning

○ Some AF 512e drives provide an XP jumper (XP partition

starts on sector 63, not 4K aligned)

● Read-modify-write


Sub-block Reads

0 1 2 3 4 5 6 7


512B Logical Blocks

0 1 2 3 4 5 6 7


512B Logical Blocks

Read 512B Block

Must Read 4K Block


Sub-block Writes (Read-modify-write)

0 1 2 3 4 5 6 7


512B Logical Blocks

0 1 2 3 4 5 6 7


512B Logical Blocks

0 1 2 3 4 5 6 7


512B Logical Blocks

Rea

d 51

2B

Logi

cal B

lock

Rea

d 4K

P

hysi

cal B

lock

Writ

e 4K

P

hysi

cal B

lock


Misaligned 4K Writes

0 1 2 3 4 5 6 7

4K Physical Block #0 4K Physical Block #1

0 1 2 3 4 5 6 7


0 1 2 3 4 5 6 7


Rea

d 4K

Lo

gica

l Blo

ckR

ead

2 4K

P

hysi

cal B

lock

Writ

e 2

4K

Phy

sica

l Blo

ck

8 9

8

8 9

9


Solutions (sort of)

● Override the lies from the device

○ FreeBSD, Illumos, and Linux have all implemented a way

to override the discovered sector size

○ FreeBSD

■ using gnop to create 4k device

○ Illumos

■ add an override into sd.conf:sd-config-list = "VENDOR PRODUCT", physical-block-size:4096;

○ Linux■ zpool create -o ashift=12 tank <device>


Drawbacks of 4K and ZFS

● Reduced compression ratio

○ Blocks less than 4K mean 0% compression

○ 8K block can only achieve 50% compression

● Migrating drives from 512B to 4K

● Inefficient metadata allocation

○ Some metadata is allocated in 4K chunks and will no

longer get compressed

● Improper accounting of compressed sizes in datasets

● RAID-Z and 4k -- not recommended

● Configuring root pools to use 4K

○ Grub support?

● Fewer uberblocks


Q&A / Beer?

ZFS Day

October 2, 2012

George Wilson

[email protected]

Technology

Why 4k?