Embed Size (px)
Citation preview
Cloudius Systems presents:
1,000,000 CQL operationsper server
Dor Laor @ ScyllaDB
Capable of 1,000,000 operations per secondPER NODE
With predictable, low latenciesCompatible with Apache Cassandra
Scylla: A new NoSQL Database
LATENCY
FULLY COMPATIBLE❏ Uses Cassandra SSTables❏ Use your existing drivers❏ Use your existing CQL queries❏ Use your existing cassandra.yaml❏ Manage with nodetool or other JMX console❏ Use your existing code with no change❏ Copy over a complete Cassandra database❏ Works with the Cassandra ecosystem (Spark etc.)
FULLY COMPATIBLE
WHAT WOULD YOU DO WITH 1 MILLION TPS?Shrink your cluster by a factor of 10XHandle 10X traffic spikes on Black FridayModel your data instead of using Cassandra as K/VGet the most out of your data - Run more queriesMaintain your clusters while servingStop using caches in front of the database
TECHNOLOGY:HOW IT WORKS
SCYLLA IS QUITE DIFFERENTShard-per-core, no locks, no threads, zero-copyBased on the Seastar C++ application frameworkEfficient, unified DB cache (not using Linux page cache)CQL-oriented storage engineExploit all hardware resources - NUMA, multiqueue NICs, etc
SCYLLA DB: ARCHITECTURE COMPARISON
● KVM was invented by Avi in 2006, development was managed by Dor● It was a new hypervisor after VMW, Xen had dominated the market● By smart design choices and leveraging Linux and the hardware it became the most
performing hypervisor.○ KVM holds SPECvirt performance record○ KVM holds max IOPS record
● The Open Virtualization Alliance includes hundreds of companies, including HP, IBM, Intel, AMD, Red Hat, etc
● KVM is the engine behind many clouds such as OpenStack, IBM, NTT, Fujitsu, HP, Google, DigitalOcean, etc.
Kernel
Cassandra
TCP/IPScheduler
queuequeuequeuequeuequeuethreads
NICQueues
Kernel
Traditional stack Scylla sharded stack
Memory
Lock contentionCache contentionNUMA unfriendly
Application
TCP/IP
Task Schedulerqueuequeuequeuequeuequeuesmp queue
NICQueue
DPDK
Kernel (isn’t
involved)
Userspace
Application
TCP/IP
Task Schedulerqueuequeuequeuequeuequeuesmp queue
NICQueue
DPDK
Kernel (isn’t
involved)
Userspace
Application
TCP/IP
Task Schedulerqueuequeuequeuequeuequeuesmp queue
NICQueue
DPDK
Kernel (isn’t
involved)
Userspace
CoreDatabase
TCP/IP
Task Schedulerqueuequeuequeuequeuequeuesmp queue
NICQueue
DPDK
Kernel (isn’t
involved)
Userspace
No contentionLinear scalingNUMA friendly
Scylla has its own task schedulerTraditional stack Scylla stack
Promise
Task
Promise
Task
Promise
Task
Promise
Task
CPU
Promise
Task
Promise
Task
Promise
Task
Promise
Task
CPU
Promise
Task
Promise
Task
Promise
Task
Promise
Task
CPU
Promise
Task
Promise
Task
Promise
Task
Promise
Task
CPU
Promise
Task
Promise
Task
Promise
Task
Promise
Task
CPU
Promise is a pointer to eventually computed value
Task is a pointer to a lambda function
Scheduler
CPU
Scheduler
CPU
Scheduler
CPU
Scheduler
CPU
Scheduler
CPU
Thread
Stack
Thread
Stack
Thread
Stack
Thread
Stack
Thread
Stack
Thread
Stack
Thread
Stack
Thread
Stack
Thread is a function pointer
Stack is a byte array from 64k to megabytes
Context switch cost is
high. Large stacks pollutes
the caches No sharing, millions of
parallel events
Unified cacheCassandra Scylla
Key cache
Row cache
On-heap /Off-heap
Linux page cache
SSTables
Unified cache
SSTables
TuningCaching unparsed dataParasitic rowsPage faults
Scylla’s graphs
Cassandra’s graphs
❏ Implement missing CQL features❏ Complete nodetool support❏ Spark integration, other connectors❏ Thrift support ❏ Authentication and encryption
Work in progress
❏ Multi-tenancy❏ New protocols❏ Vertical integration
FUTURE
SCYLLA, NoSQL GOES NATIVEThank you.
http://scylladb.com
