View
1.304
Download
0
Category
Preview:
DESCRIPTION
My session at NoSQL Now! conference
Citation preview
1
Go Simple, Fast, Elas1c with Couchbase Server and
Document-‐oriented Data Management
Dip1 Borkar Director, Product Management
2
WHY TRANSITION TO NOSQL?
3
Changes in interac1ve soDware – NoSQL driver
4
11%
12%
16%
29%
35%
49%
Other
All of these
Costs
High latency/low performance
Inability to scale out data
Lack of flexibility/rigid schemas
Source: Couchbase NoSQL Survey, December 2011, n=1351
What is the biggest data management problem driving your use of NoSQL in the coming year?
Survey: Two big drivers for NoSQL adop1on
5
COUCHBASE SERVER
6
Couchbase Server: Simple. Fast. Elas1c. NoSQL.
• Easy scalability • Scale your the database layer without any interruption or change to your application
• Grow your cluster from 5 to 25 to 100 nodes and evenly distributed workloads across your cluster
• Consistent High Performance • Consistent sub-‐millisecond response times • High throughput means fewer servers required to serve growing numbers of users
• Always on • With high availability features applica1ons are always
online • Whether upgrading the database, system soDware or
hardware -‐ zero down1me
7
Flexible Data Model
• Not need to worry about the database when changing your applica1on
• Records can have different structures, there is no fixed schema • Record structure can simply be changed at any 1me, without affec1ng remaining stored data
• The JSON based document model also maps extremely well to how applica1ons represent data, including nested sturctures
{ “ID”: 1, “FIRST”: “DipR”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA” }
JSON JSON
JSON JSON
8
Rela1onal vs Document data model
RelaRonal data model Document data model Collec1on of complex documents with arbitrary, nested data formats and
varying “record” format.
Highly-‐structured table organiza1on with rigidly-‐defined data formats and
record structure.
JSON JSON
JSON
C1 C2 C3 C4
{ }
9
Example: User Profile
Address Info
1 DEN 30303 CO
2 MV 94040 CA
3 CHI 60609 IL
User Info
KEY First ZIP_id Last
4 NY 10010 NY
1 DipR 2 Borkar
2 Joe
2 Smith
3 Baxter 2 Dodson
4 Lari 3 Gorin
ZIP_id CITY ZIP STATE
1 2
2 MV 94040 CA
To get informaRon about specific user, you perform a join across two tables
10
Represen1ng rows as documents
Geo Info
1 DEN 30303 CO
2 MV 94040 CA
3 CHI 60609 IL
User Info
KEY First ZIP_id Last
4 NY 10010 NY
1 DipR 2 Borkar
2 Joe
2 Smith
3 Baxter 2 Dodson
4 Lari 3 Gorin
ZIP_id CITY ZIP STATE
= { “ID”: 1, “FIRST”: “DipR”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA” } JSON
All data in a single document
11
Couchbase Server 2.0 – Coming soon
• Next major release of Couchbase Server
What’s new: • Indexing and Querying • Incremental Map Reduce • Cross Data Center Replica1on
12
Paid Produc1on Deployments (par1al list)
13
COUCHBASE SERVER ARCHITECTURE
14
Couchbase Server 2.0 Architecture
Heartbeat
Process m
onito
r
Glob
al singleton supe
rviso
r
Confi
gura1o
n manager
on each node
Rebalance orchestrator
Nod
e he
alth m
onito
r
one per cluster
vBucket state and
replica1
on m
anager
hdp RE
ST m
anagem
ent A
PI/W
eb UI
HTTP 8091
Erlang port mapper 4369
Distributed Erlang 21100 -‐ 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210 Memcapable 2.0
Moxi
11211 Memcapable 1.0
Memcached
New Persistence Layer
8092 Query API
Que
ry Engine
Data Manager Cluster Manager
15
Couchbase Server 2.0 Architecture
Heartbeat
Process m
onito
r
Glob
al singleton supe
rviso
r
Confi
gura1o
n manager
on each node
Rebalance orchestrator
Nod
e he
alth m
onito
r
one per cluster
vBucket state and
replica1
on m
anager
hdp RE
ST m
anagem
ent A
PI/W
eb UI
HTTP 8091
Erlang port mapper 4369
Distributed Erlang 21100 -‐ 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210 Memcapable 2.0
Moxi
11211 Memcapable 1.0
Memcached
New Persistence Layer
8092 Query API
Que
ry Engine
16
Couchbase deployment
Data Flow
Cluster Management
Web Applica1on
Couchbase Client Library
17
Couchbase Server Admin Console
18
3 3 2
Clustering With Couchbase
SET request arrives at KEY’s master server
Listener-‐Sender
Master server for KEY Replica Server 2 for KEY Replica Server 1 for KEY
1 1 SET acknowledgement returned to applica1on
2
Disk Disk Disk
RAM
Couchb
ase storage en
gine
Disk Disk Disk
4
19
COUCHBASE CLIENT LIBRARY
Basic Opera1on
§ Docs distributed evenly across servers in the cluster
§ Each server stores both ac2ve & replica docs § Only one server ac1ve at a 1me
§ Client library provides app with simple interface to database
§ Cluster map provides map to which server doc is on § App never needs to know
§ App reads, writes, updates docs
§ Mul1ple App Servers can access same document at same 1me
Doc 4
Doc 2
Doc 5
SERVER 1
Doc 6
Doc 4
SERVER 2
Doc 7
Doc 1
SERVER 3
Doc 3
User Configured Replica Count = 1
Read/Write/Update
COUCHBASE CLIENT LIBRARY
Read/Write/Update
Doc 9
Doc 7
Doc 8 Doc 6
Doc 3
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
Doc 9
Doc 5
DOC
DOC
DOC
Doc 1
Doc 8 Doc 2
Replica Docs Replica Docs Replica Docs
Ac1ve Docs Ac1ve Docs Ac1ve Docs
CLUSTER MAP
CLUSTER MAP
APP SERVER 1 APP SERVER 2
COUCHBASE SERVER CLUSTER
20
Add Nodes
§ Two servers added to cluster § One-‐click opera1on
§ Docs automa1cally rebalanced across cluster § Even distribu1on of
docs § Minimum doc movement
§ Cluster map updated
§ App database calls now distributed over larger # of servers
User Configured Replica Count = 1
Read/Write/Update Read/Write/Update
Doc 7
Doc 9
Doc 3
Ac1ve Docs
Replica Docs
Doc 6
COUCHBASE CLIENT LIBRARY CLUSTER MAP
APP SERVER 1
COUCHBASE CLIENT LIBRARY CLUSTER MAP
APP SERVER 2
Doc 4
Doc 2
Doc 5
SERVER 1
Doc 6
Doc 4
SERVER 2
Doc 7
Doc 1
SERVER 3
Doc 3
Doc 9
Doc 7
Doc 8 Doc 6
Doc 3
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
Doc 9
Doc 5
DOC
DOC
DOC
Doc 1
Doc 8 Doc 2
Replica Docs Replica Docs Replica Docs
Ac1ve Docs Ac1ve Docs Ac1ve Docs
SERVER 4 SERVER 5
Ac1ve Docs Ac1ve Docs
Replica Docs Replica Docs
COUCHBASE SERVER CLUSTER
21
Fail Over Node
§ App servers happily accessing docs on Server 3
§ Server fails § App server requests to server 3 fail § Cluster detects server has failed
§ Promotes replicas of docs to ac2ve § Updates cluster map
§ App server requests for docs now go to appropriate server
§ Typically rebalance would follow
User Configured Replica Count = 1
Doc 7
Doc 9
Doc 3
Ac1ve Docs
Replica Docs
Doc 6
COUCHBASE CLIENT LIBRARY CLUSTER MAP
APP SERVER 1
COUCHBASE CLIENT LIBRARY CLUSTER MAP
APP SERVER 2
Doc 4
Doc 2
Doc 5
SERVER 1
Doc 6
Doc 4
SERVER 2
Doc 7
Doc 1
SERVER 3
Doc 3
Doc 9
Doc 7 Doc 8
Doc 6
Doc 3
DOC
DOC
DOC DOC
DOC
DOC
DOC DOC
DOC
DOC
DOC DOC
DOC
DOC
DOC
Doc 9
Doc 5 DOC
DOC
DOC
Doc 1
Doc 8
Doc 2
Replica Docs Replica Docs Replica Docs
Ac1ve Docs Ac1ve Docs Ac1ve Docs
SERVER 4 SERVER 5
Ac1ve Docs Ac1ve Docs
Replica Docs Replica Docs
COUCHBASE SERVER CLUSTER
22
Reading and Wri1ng
Reading Data WriRng Data
Server
Give me document A
Here is document A
Application Server
A
Server
Please store document A
OK, I stored document A
Application Server
A
RAM
DISK
A
A
RAM
DISK
A
A
23
Server
Flow of data when wri1ng
WriRng Data
Application ServerApplication Server Application Server
ApplicaRons wriRng to Couchbase
Couchbase wriRng to disk
network
Couchbase transmiing replicas
ReplicaRon queue Disk write queue
24
Indexing and Querying
APP SERVER 1
COUCHBASE CLIENT LIBRARY
§ Indexing work is distributed amongst nodes § Large data set possible § Parallelize the effort
§ Each node has index for data stored on it
§ Queries combine the results from required nodes
CLUSTER MAP
Doc 4
Doc 2
Doc 5
SERVER 1
Doc 6
Doc 4
SERVER 2
Doc 7
Doc 1
SERVER 3
Doc 3
APP SERVER 2
COUCHBASE CLIENT LIBRARY CLUSTER MAP
Doc 9
Doc 7
Doc 8 Doc 6
Doc 3
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
DOC
Doc 9
Doc 5
DOC
DOC
DOC
Doc 1
Doc 8 Doc 2
Replica Docs Replica Docs Replica Docs
Ac1ve Docs Ac1ve Docs Ac1ve Docs
COUCHBASE CLIENT LIBRARY
COUCHBASE CLIENT LIBRARY CLUSTER MAP
CLUSTER MAP
APP SERVER 1 APP SERVER 2
Query Response
25
Couchbase SDKs
Java SDK
.Net SDK
PHP SDK
Ruby SDK
…and many more
Java client API
User Code
Couchbase Server
CouchbaseClient cb = new CouchbaseClient(listURIs, "aBucket", "letmein"); // this is all the same as before cb.set("hello", 0, "world"); cb.get("hello");
htp://www.couchbase.com/develop
Couchbase Java Library (spymemcached)
26
DEMO (IF TIME PERMITS)
27
Coming September 21, 2012
SAN FRANCISCO
Featured speakers include:
Tell them DipR sent you and get 50% off. Use discount code DIPTI. hdp://www.couchbase.com/couchconf-‐san-‐francisco
28
THANK YOU
DIPTI@COUCHBASE.COM
29
30
Recommended