Storage CachingTim Fletcher

Brighter Connections

Who are we?

15+ year Linux experience13 years work in educationRHCE since 1999 (RedHat 6.1)

Joined Brighter Connections in OctoberIT Practise in HuddersfieldSpecialise in innovative solutions

Why your SAN is slow

Centralised storage, good for cost bad for performance

Lots of different competing workloads mean disks "thrash"

Making your SAN faster

HardwareSpindlesRAM

SoftwareCopy on WriteQuality of Service

Hard Drives

Technology basically the same for over 30 years, known as "Spinning Rust".

Good for cheap bulk storageGood at linear access

Bad at random accessPower hungryFragile

Memory

Very fast accessVery good random accessDirectly connected to the CPU

ExpensiveVolatilePower Hungry

Flash

Very fast, especially at random accessCan be connected directly to the PCI-e busNon volatileLow powerMassive drops in price, but still expensiveCells can wear outIncludes firmware

Storage Speeds

PCIe vs Hard disk emulation

Hard disk emulationEasy to deployWell understoodCheapMany layers of translation

PCIeExpensive, or is it?Higher power drawDirectly attached to the CPU bus

Caching - Why does it matter

Flash is expensive and wears outThe closer the data is to the CPU the betterKeep your SAN arrayLeverages the best of both worlds

Caching - Where do your blocks go?

Flashcache

Fully open source, released by FacebookKernel module, based in the device mapper layerStarting to appear in distrosError resistantCache can be added and removed from storageUsed by Facebook in production

Bcache

Fully Open SourceDesigned from the ground up to cacheDesigned with an understanding of flashHigher performance

Requires devices to be rebuilt as "bcache" devicesRequires full kernel rebuild

ZFS

Not just a filesystemIncludes device management, RAID and filesystem management.Open Source but license incompatible with GPLWritten by Sun, available in FreeBSD, Solaris derivatives, and LinuxCan use SSDs as L2ARC and as SLOGCache devices can be live added and removedCache does not persist over reboots

Closed Source Options

Hook in to VMware

FusionIO - ioTurbineSanDisk - FlashSoft

Questions?tim@brighter-connections.comwww.brighter-connections.com

@timjdfletcher