22
Erasure Code in Ceph Loic Dachary @ Red Hat

Ceph Day 2015 - Erasure Coding

Embed Size (px)

Citation preview

Page 1: Ceph Day 2015 - Erasure Coding

Erasure Code in CephLoic Dachary @ Red Hat

Page 2: Ceph Day 2015 - Erasure Coding

Save Space

Page 3: Ceph Day 2015 - Erasure Coding

5 minutes role playing game

Page 4: Ceph Day 2015 - Erasure Coding

XOR

Page 5: Ceph Day 2015 - Erasure Coding

3 peta => 1.3 peta

Page 6: Ceph Day 2015 - Erasure Coding

Harder object mutations / recovery

Page 7: Ceph Day 2015 - Erasure Coding

Simple operations and tiering

Replicated Erasure Coded

Page 8: Ceph Day 2015 - Erasure Coding

Promoted to replica on read

Replicated Erasure Coded

Page 9: Ceph Day 2015 - Erasure Coding

Sam & David : internals

Page 10: Ceph Day 2015 - Erasure Coding

Janne & Andreas & Loic & Takeshierasure code

Page 11: Ceph Day 2015 - Erasure Coding

Released May 2014 : Firefly

Page 12: Ceph Day 2015 - Erasure Coding

Why save space before shortage ?

Page 13: Ceph Day 2015 - Erasure Coding

Reliability Model

● After an OSD is lost

● Recovery

● Backfilling a new OSD

Page 14: Ceph Day 2015 - Erasure Coding

April 2015 : Hammer

Page 15: Ceph Day 2015 - Erasure Coding

Repair K=10, M=4

Page 16: Ceph Day 2015 - Erasure Coding

Locally Recoverable CodesLRC @ Red Hat

Page 17: Ceph Day 2015 - Erasure Coding

SHEC Takeshi @ Fujitsu

Page 18: Ceph Day 2015 - Erasure Coding

ISA plugin Yuan @ Intel

SIMD aka SSE2, SSE3, SSE4

Only for Intel processors

~50% Faster

Page 19: Ceph Day 2015 - Erasure Coding

jerasure ARM Janne @ ARM

Page 20: Ceph Day 2015 - Erasure Coding

Infernalis

Page 21: Ceph Day 2015 - Erasure Coding

Hitchhiker Rashmi @ U.C. Berkeley

Page 22: Ceph Day 2015 - Erasure Coding

[email protected]

Artwork GPLv3+ Tartaruga Feliz