Embed Size (px)
DESCRIPTION
Дмитрий Каляда (Senior Java Developer в Altoros)
Citation preview
NoSQL benchmarking 2.0Dmitriy Kalyada
People produce
• More than 144.8 billion Email a day.
• 340 million tweets a day.
• 2 million Google search queries a minute.
• Apple receives around 47000 app downloads a minute.
• Instagram photographers share 3,600 new photos a minute.
• People perform over 2,000 Foursquare check-ins a minute.
RDBMS issues
• Hardware price: hi-end solutions required.
• Scalability: significant development&administrative effort.
NoSQL solutions
• Document oriented: Couchbase, MongoDB
• BigTable(Column family): Hadoop/HBase, Cassandra
• Key value: Riak, Membase
• Graph database: Neo4j, AllegroGraph
• XML database: BaseX, eXist
• Object database management systems: Cache, Twig, Db4o
Benchmark participants
YCSB
• Yahoo! Cloud Serving Benchmark
• Framework: Platform
• Workloads: Generators + Scenarios
• Supported DBs: PNUTS, BigTable, HBase, Hypertable, Azure, Cassandra, CouchDB, Voldemort, MongoDb, Infinispan, Dynomite, Redis, GemFire, GigaSpaces XAP, DynamoDB
https://github.com/brianfrankcooper/YCSB
YCSB client architecture
YCSB Client
Workload Executor
Client treads
Stats
DB Interface
Layer
DB Cluster
Workload file• Read/write mix• Record size• Popularity distribution
Command line properties• DB to use•Workload to use• Target throughput• Number of threads
Data
• Record: 1KB = 10 fields x 100 Bytes
• Amount: 100 million records
• Total: ~130GB (~35 per node)
Nodes configuration
CPU Cores 8
CPU frequency 1500 MHz
Disk Space 160 GB
Bandwidth 102400 Kbit/sec
RAM 14848 MB
OS Сentos-6
Cluster configurations
Database Configuration Access Points
Cassandra 1.2.5
Couchbase 2.1.0
HBase 0.94.6
MongoDB 2.4.4
Riak 1.3.2
MySQL 7.3.2
4 x 4
4 x 4
1 name + 4 data 1
2 m-s + 4 m-d 2
4 x 4
4 x 1
Load structure
Type Read Insert Update RMW Scan
A
B
C
D
E
F
G
50% 50%
95% 5%
100%
95% 5%
5% 95%
50% 50%
5% 95%
Workload structure
A
B
C
D
E
F
G
0% 25% 50% 75% 100%
95%
50%
95%
50%
5%
50%
5%
5%
5%
95%
100%
95%
50%
Read Insert Update Read-Modify-Write Scan
A: Update 50%, Read 50%
0"
1"
2"
3"
4"
5"
6"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'m
s'
Target'Througthput,'ops/sec'
Update'50%'
Couchbase"2.1.0"
Cassandra"1.2.5"
HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
A: Update 50%, Read 50%
0"
0.5"
1"
1.5"
2"
2.5"
3"
3.5"
4"
4.5"
5"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'m
s'
Target'Througthput,'ops/sec'
Read'50%'
Couchbase"2.1.0"
Cassandra"1.2.5"
HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
A: Update 50%, Read 50%
0"
5000"
10000"
15000"
20000"
25000"
30000"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Actual'Throu
gthp
ut,'o
ps/sec'
Target'Througthput,'ops/sec'
Througthput'Limits'
Couchbase"2.1.0"
Cassandra"1.2.5"
HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
B: Update 5%, Read 95%
0"
1"
2"
3"
4"
5"
6"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'m
s'
Target'Througthput,'ops/sec'
Update'5%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
B: Update 5%, Read 95%
0"
0.5"
1"
1.5"
2"
2.5"
3"
3.5"
4"
4.5"
5"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'m
s'
Target'Througthput,'ops/sec'
Read'95%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
B: Update 5%, Read 95%
0"
5000"
10000"
15000"
20000"
25000"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Actual'Throu
gthp
ut,'o
p/sec'
Target'Througthput,'ops/sec'
Througthput'Limits'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
C: Read 100%
0"
1"
2"
3"
4"
5"
6"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'m
s'
Target'Througthput,'ops/sec'
Read'100%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
C: Read 100%
0"
5000"
10000"
15000"
20000"
25000"
30000"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Actual'Throu
gthp
ut,'o
p/sec'
Target'Througthput,'ops/sec'
Througthput'Limits'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
F: Read-Modify-Write 50%, Update 50%
!1#
0#
1#
2#
3#
4#
5#
6#
0# 5000# 10000# 15000# 20000# 25000# 30000#
Avarage'latency,'m
s'
Target'Througthput,'ops/sec'
Update,'50%'
Couchbase#2.1.0#
#Cassandra#1.2.5#
#HBase#0.94.6#
#Riak#1.3.2#
#MySQL#Cluster#7.3.2#
#Monngodb#2.4.4#
F: Read-Modify-Write 50%, Update 50%
0"
1"
2"
3"
4"
5"
6"
7"
8"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'m
s'
Target'Througthput,'ops/sec'
Read8Modify8Write,'50%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
F: Read-Modify-Write 50%, Update 50%
0"
2000"
4000"
6000"
8000"
10000"
12000"
14000"
16000"
18000"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Actual'Throu
gthp
ut,'o
ps/sec
'
Target'Througthput,'ops/sec'
Througthput'Limits'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
G: Insert 95%, Read 5%
0"
0.5"
1"
1.5"
2"
2.5"
3"
3.5"
0" 5000" 10000" 15000" 20000" 25000" 30000" 35000" 40000" 45000" 50000"
Avarage'latency,'m
s'
Target'Througthput,'ops/sec'
Insert,'95%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
G: Insert 95%, Read 5%
0"
2"
4"
6"
8"
10"
12"
14"
0" 5000" 10000" 15000" 20000" 25000" 30000" 35000" 40000" 45000" 50000"
Avarage'latency,'m
s'
Target'Througthput,'ops/sec'
Read,'5%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
G: Insert 95%, Read 5%
0"
5000"
10000"
15000"
20000"
25000"
30000"
35000"
40000"
0" 5000" 10000" 15000" 20000" 25000" 30000" 35000" 40000" 45000" 50000"
Actual'Throu
gthp
ut,'o
ps/sec
'
Target'Througthput,'ops/sec'
Througthput'Limits'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"
Benchmark participants
More than 20 000 new websites were launched from our start
