Embed Size (px)
Citation preview
ScyllaDB, Present and FutureAvi Kivity (@AviKivity)
CTO @ScyllaDB
Agenda
• The Three ScyllaDB Commitments
• ScyllaDB Road Map
Agenda
▸The Three ScyllaDB Commitments
• ScyllaDB Road Map
The Three ScyllaDB Commitments
• High Throughput and Low Latency
•Compatibility with the Apache Cassandra Ecosystem
• Workload Conditioning
Agenda
• The Three ScyllaDB Commitments▸Throughput and latency
• ScyllaDB Road Map
Throughput and Latency
Throughput and Latency
• Reduces Capital and/or Cloud Expenses
• Reduces the need to manage large clusters
• Reduces support costs
• Reduces failure rate
Not Just Equipment Cost!
• Fewer nodes = fewer emergencies
• Reduce risk of double failure
Not Just Equipment Cost!
• Lose fewer customers due to page-load time
• Win more real-time bids
Agenda
• The Three ScyllaDB Commitments▸Cassandra Compatibility
• ScyllaDB Road Map
Cassandra Ecosystem Compatibility
Cassandra Ecosystem Compatibility
• Reuse existing investments and knowledge
• Leverage existing software
• Reduce dev effort, time to market
Agenda
• The Three ScyllaDB Commitments▸Workload Conditioning
• ScyllaDB Road Map
Workload Conditioning
• Internal feedback loops to balance competing loads
Memtable
Seastar SchedulerCompaction
Query
Repair
Commitlog
SSD
Compaction Backlog Monitor
Memory Monitor
Adjust priorityAdjust priority
WAN
CPU
Workload Conditioning Examples
• Prevent compaction from falling behind
• Ensure repair makes forward progress
• Prevent memtable memory from filling up
• Isolate read loads from write loads
• Ramp up load to a newly started node until its cache
is warm
ScyllaDB Commitments RecapSELECT * FROM ScyllaDB.Commitments;
Commitment | Value------------------------+-------------------------------------
Performance & Latency | Reduced CapEx/CloudExEcosystem Compatibility | Reduced time-to-market and dev cost
Workload Conditioning | Simplified operations
RoadMap
Near Term (4Q16)
• Materialized Views / Secondary Indexes
• Counters
• Lightweight Transactions
• Management stack phase 1
• Formal support for REST API
• Container Orchestration Integration▪ For less critical throughput/latency
Why REST API?
• JMX slow, somewhat clumsy▪ Hard to operate from non-Java applications
• Need standard, documented, simple approach to
automating ScyllaDB cluster operations
Medium-long Term (1 / 2)
• New storage format
• Multitenancy
• Analytics
• Search
• Additional protocol support
Medium-Long Term (2 / 2)
• Filesystem bypass
• NVDIMM / 3DXpoint
New Filesystem Format
• C* 2.x format metadata intensive
• C* 3.x format improves, but large partition support
remains slower
• Scylla will provide first-class large partition performance
Multitenancy
• Many orgs run multiple small-ish clusters▪ Wish to isolate performance considerations
• Problems▪ Underutilized hardware▪ Duplication of ops effort
Multitenancy
• Run several “virtual ScyllaDB clusters” on top of one
physical ScyllaDB cluster
▪ Share resources, ops efforts
▪ Workload Conditioning isolates distinct workloads
▪ Each virtual cluster receives an SLA
Roadmap Recap
• Bridging the gap with Cassandra
• ScyllaDB features
• ScyllaDB meta-features
