Upload
wilfred-springer
View
1.924
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Slides used during my JFall 2010 talk
Citation preview
A NoSQL Rollercoaster
Saturday, November 6, 2010
Saturday, November 6, 2010
IMSThe Hierarchical
Database
(1966)
Vern WattsSaturday, November 6, 2010
“A Relational Model for Large
Shared Databanks”
(1970)
Ted CoddSaturday, November 6, 2010
“"SEQUEL: A Structured
English Query Language"
Don Chamberlin & Ray Boyce
(1974)
Don ChamberlinSaturday, November 6, 2010
ACID
(late 1970’s)
Jim GraySaturday, November 6, 2010
“NoSQL” coined
(1998)
Carlo StrozziSaturday, November 6, 2010
“NoSQL” Reintroduced(2008)
Eric Evans
Saturday, November 6, 2010
select fun, profit from real_world where relational=false?
Saturday, November 6, 2010
“In striving to make every user happy, a technology an actually leave the majority
unhappy.”
“Every good idea is generalized to its level of
inapplicability.”
(Peter Principle)
Saturday, November 6, 2010
Saturday, November 6, 2010
Why?
Saturday, November 6, 2010
Internet Scale
• Massive data collections
• Huge number of requests
• Coming from geographic areas across the globe
• 24/7
Saturday, November 6, 2010
Total Cost of Ownership
• The price of a license
• The price of support
• The price of hardware
Saturday, November 6, 2010
Availability
Saturday, November 6, 2010
Data Models
Saturday, November 6, 2010
Column Oriented
…
key namedcolumn
namedcolumn
namedcolumn
namedcolumn
namedcolumn
Column Family ≈ Table
Empty cells are cheap (sparse
table)
Can grow “indefinitely”
Schemaless No secundary
indexes
Saturday, November 6, 2010
Column Oriented + Super Columns
…
key namedcolumn
namedcolumn
namedcolumn
namedcolumn
namedcolumn
…
namedcolumn
namedcolumn
namedcolumn
…
Super Columns
Saturday, November 6, 2010
Key Value Store
•Schemaless•Versioning
10110110
Saturday, November 6, 2010
Graph Database
SPARQL?
Saturday, November 6, 2010
Document Store
Improved Indexing
<persons><person><name>Wilfred</name><surname>Springer</person>…</persons>
[{ "Name" : "Wilfred", "Surname" : "Springer"}, …]
JSON
XML
Serverside Processing
Saturday, November 6, 2010
Challenges
Saturday, November 6, 2010
CAP Theorem
Eric Brewer
Saturday, November 6, 2010
Availability Consistency
PartitionTolerance
Pick two
Saturday, November 6, 2010
ConsistencyConsistency
Strong Consistency
Weak Consistency
Eventual Consistency Other
Causal Consistency
Read-your-writes
ConsistencySession
ConsistencyMonotonic Read
ConsistencyMonotonic Write
Consistency
Saturday, November 6, 2010
Strong Consistency
AB
C
1
2
2
2
0 value = "foo"
value = "bar"
value = "bar" value = "bar"
value = "bar"
After the update, any subsequent access will return the updated value.
Saturday, November 6, 2010
Weak Consistency
AB
C
1 value = "bar"
value = "bar" /"foo"
value = "bar" / "foo"
value = "bar" / "foo"
0 value = "foo"
>1
>1
>1
The system does not guarantee that at any given point in the future subsequent access will return the updated value
Saturday, November 6, 2010
Eventual Consistency
If no updates are made to the object, eventually all accesses will return the last updated value.
AB
C
1 value = "bar"
value = "bar" value = "bar"
value = "bar"
0 value = "foo"
t
t
t
t ≥ 1
Saturday, November 6, 2010
Session Consistency
Within the “session”, the system guarantees read-your-writes consistency
2 value = "foo"
Session 1
Session 2
AB
C
1 value = "bar"
0 value = "foo"
2 value = "bar"
Saturday, November 6, 2010
Partition Tolerance
Awrites new value
replicates new value
reads new value
Saturday, November 6, 2010
Partition Tolerance
Awrites new value
replicates new value
reads new value
!
Saturday, November 6, 2010
Partition Tolerance
Awrites new value
fails to replicate
new value
reads old value
Saturday, November 6, 2010
Partition Tolerance
A
failing attempt to write a new
value
fails to replicate
new value
Saturday, November 6, 2010
Byzantine Failures
Saturday, November 6, 2010
Faults that might occur in a
distributed system
•Lying
•Fabrication of messages
•Collisions
•Selective non-participation
Saturday, November 6, 2010
Byzantine Generals Problem
Saturday, November 6, 2010
Solution Strategies
Saturday, November 6, 2010
Strong Consistency
1
Saturday, November 6, 2010
Παξοί
Saturday, November 6, 2010
Leader Proposer
Acceptor Learner
Four Roles
Saturday, November 6, 2010
Paxos in a Nutshell
RequestPropose
Promise
AcceptAcceptedResponse
Saturday, November 6, 2010
Processors combine various
roles
Consensus based on
majority vote
Paxos guarantees progress for 2F + 1
processors in face of failure of F processors
Saturday, November 6, 2010
“Either Paxos, Paxos with
cruft, or broken”
Mike Burrows
(Google Chubby)
Saturday, November 6, 2010
Partitioning2
Saturday, November 6, 2010
Consistent HashingKey KA
B
C
DE
F
G
H
Saturday, November 6, 2010
High Availability
3
Saturday, November 6, 2010
ReplicationA
B
C
DE
F
G
H
Saturday, November 6, 2010
W = 3
N = 4R = 2
A
Replication Factor
Saturday, November 6, 2010
W = 3
N = 4R = 2
A
Reconciliation during reads
Reconciliation
Saturday, November 6, 2010
Handling Temporary
Failures
4
Saturday, November 6, 2010
ProblemReconcilation
?
Alice Ben CathyDave
Wednesday?
Thursday
Tuesday
What's it gonna be?
Tuesday ThursdayX? ? ?
Saturday, November 6, 2010
Vector Clocksdate = Wednesdayvclock = Alice:1
date = Tuesdayvclock = Alice:1, Ben:1
date = Tuesdayvclock = Alice:1, Ben:1, Dave:1
date = Tuesdayvclock = Alice:1, Cathy:1
date = Thursdayvclock = Alice:1, Ben:1, Cathy:1, Dave:2
Saturday, November 6, 2010
Conflict Resolution
date = Tuesdayvclock = Alice:1, Ben:1, Dave:1
date = Thursdayvclock = Alice:1, Ben:1, Cathy:1, Dave:2
6
Saturday, November 6, 2010
Recovering from permanent
failures
5
Saturday, November 6, 2010
What if nodes die permantly?A
B
C
DE
F
G
H
Saturday, November 6, 2010
Merkle Trees
##
#
#
##
##
#
#
#
##
#
Saturday, November 6, 2010
Enough is enough!!!
Saturday, November 6, 2010
Conclusions
Saturday, November 6, 2010
Conclusions
Scale!!!
Saturday, November 6, 2010
Conclusions
Scale!!!No free ride
Saturday, November 6, 2010
Conclusions
Scale!!!No free ride
You gain some
Saturday, November 6, 2010
Conclusions
Scale!!!No free ride
You gain some
You lose some
Saturday, November 6, 2010
Conclusions
Scale!!!No free ride
You gain some
You lose someChoice
Saturday, November 6, 2010
Conclusions
Scale!!!No free ride
You gain some
You lose someChoice
Control
Saturday, November 6, 2010
Conclusions
Scale!!!No free ride
You gain some
You lose someChoice
ControlTake control
Saturday, November 6, 2010
Conclusions
Scale!!!No free ride
You gain some
You lose someChoice
ControlTake control
Understand your options
Saturday, November 6, 2010
Conclusions
Scale!!!No free ride
You gain some
You lose someChoice
ControlTake control
Understand your options
Fun!!!
Saturday, November 6, 2010
Conclusions
Scale!!!No free ride
You gain some
You lose someChoice
ControlTake control
Understand your options
Fun!!!
Changes your way of thinking
Saturday, November 6, 2010
Conclusions
Scale!!!No free ride
You gain some
You lose someChoice
ControlTake control
Understand your options
Fun!!!
Changes your way of thinking
Saturday, November 6, 2010
No SQL!!!
Saturday, November 6, 2010
BASE
• Basically Available
• Soft State
• Eventually Consistent
Saturday, November 6, 2010