Couchbase Mobile

Enterprise Solutions NOT just Sync

©2015 Couchbase Inc. 2

Couchbase Mobile Solution

Couchbase Lite

Sync Gateway Couchbas

e ServerFull featured embedded

NoSQL database

Secure data synchronization over the

web

Our flagship, highly scalable

NoSQL database in the cloud

©2015 Couchbase Inc. 3

Replication Topologies

STAR

TIERED

MESHFull multi-master replicationContinuous or ad-hoc in either direction

©2015 Couchbase Inc. 4

Topics

How Channels WorksPast, Present & FutureSimplify Data Flow in an OrganizationPush vs PullScaling

©2015 Couchbase Inc. 5

Syncing a single user’s database is EASY

©2015 Couchbase Inc. 6

What about syncing x,000,000?

©2015 Couchbase Inc. 7

Read-side filtering doc.owner == 'bob'

©2015 Couchbase Inc. 8

Read-side filtering is too slow

©2015 Couchbase Inc. 9

Bob

Jim

channel(doc.owner)Partition data during writes instead

©2015 Couchbase Inc. 10

Now syncing X,000,000 is easy

©2015 Couchbase Inc. 11

Sync Gateway & Channels – Pull Feed

{“_id”:”12”,“_rev”:”3-hno..”,“channels”:[“yellow”]}

{“_id”:”78”,“_rev”:”5-u9b..”,“channels”:[“yellow”]}

78 31 20 17

©2015 Couchbase Inc. 12

Sync Gateway & Channels

• Private• Public• Share Private

©2015 Couchbase Inc. 13

Sync Gateway & Channels – Multi-channel Feed

{“_id”:”12”,“_rev”:”3-hno..”,“channels”:[“yellow”]}

{“_id”:”xzq”,“_rev”:”1-7tpb..”,“channels”:[“blue”]}

{“_id”:”81x”,“_rev”:”2-jba..”,“channels”:[“red”]}

78 172031 1963

©2015 Couchbase Inc. 14

Sync Gateway & Channels – Filtered Feed by Channel(s)

78 1720316319

Continuous

Every 2 min

Every 15 min & Only Wi-Fi

©2015 Couchbase Inc. 15

Couchbase Mobile to My Current Systems

Your App for:Your Business

Logic

24 2625

©2015 Couchbase Inc. 16

Couchbase Mobile to My Current Systems

24 2625

Get me channel feed: since=25

262728

“Failure”

©2015 Couchbase Inc. 17

Traditional - Data Flow

3rd Party

Public

Internal

MobileMobileInternalMobile

3rd Party

in 2010

©2015 Couchbase Inc. 18

Mobile the Hard Way - 1st Request / Response

Data Request App

Auth

©2015 Couchbase Inc. 19

Mobile the Hard Way – 2nd Online/Offline Data Sync

Query/Filter by User(s)/Group(s) Auth

Data Request App

Auth

Buffer / Que

Data Processing App

©2015 Couchbase Inc. 20

Mobile the Hard Way - 3rd , 4th , 5th … etc.

• Conflict Resolution ?• 2x users = 2x Reads & 2x Writes ?• New Release = New Features = New Schema =

Schema Migration

©2015 Couchbase Inc. 21

Simple Mobile - Data Flow

3rd Party

Public

Internal

MobileMobileInternalMobile

3rd Party

©2015 Couchbase Inc. 22

Who has the “Truth” ?

Big Applications

Mini Applications

I have the Truth I have the Truth

I have the

Truth

I have the

Truth

©2015 Couchbase Inc. 23

Sync Gateway – the “Truth”

“What is my value

add ?”

©2015 Couchbase Inc. 24

Sync Gateways – Value Add & Data Sourcing

{“_id”:”flight::100::sfo”,“_rev”:”2-aoru”,“seats”:[

{“a1”:”open”} ………. ],“status”:”onTime”,“channel”:”sfo::100”}{“_id”:”weather::sfo”,“_rev”:”5-8oum”,“windSpeed”:”75knots”,”windDirection”:”sw”,“channel”:”sfo”}

{“_id”:”price::100::sfo::a1”,“_rev”:”3-d90j’”,“src”:{“_rev”:”2-aoru”},“price”:450.00“cost”: {

“weather”:-55.00,

”src”:{

“_rev”:”5-8oum”},

},“channel”:”price::sfo::100”}

?

©2015 Couchbase Inc. 25

Market Place of Micro Services

Source 2

Source 1

Source 4

Source 3Different: Data , Locations and/or Sync Functions

Public Data

Private Data

©2015 Couchbase Inc. 26

Replication

SG-Replicate XDCR

• Replicate via REST/HTTP• Master/Master or Master/Slave• Fine Grain Data Replication vi

channel(s)• Replication offline for years is OK.

• Replicate via Memory to Memory

• Master/Slave• Full Data Copy• Replication Offline for few days

is OK.

or

©2015 Couchbase Inc. 27

“Chaining” Sync Gateways – Data Locality

Channels[“A”,”B”]

Channels[“A”,”T”,”Z”]

DC Failed?No Problem go to the main source

DC1 WEST

DC2 EAST

via SG-Replicate(built-into Sync Gateway

v1.3)

©2015 Couchbase Inc. 28

“Chaining” Sync Gateways - Options

Unidirectional

Bidirectional

&

& Future Changes

©2015 Couchbase Inc. 29

Push

PUSH {“data”}

Q: Who is responsible for getting the data to the BLUE app?Q: How dose the BLUE app know if the publisher is available?

A: “it depends” or “its complicated”A: BLUE app polls the publisher or publisher send empty packets

©2015 Couchbase Inc. 30

Pull – Streaming or Polling

PULL {“data”}

A: The BLUE appA: Connection timeout to publisher

Q: Who is responsible for getting the data to the BLUE app?Q: How dose the BLUE app know if the publisher is available?Q: How dose the BLUE app know what data it needs?A: Query publisher or last sequence number since=25

©2015 Couchbase Inc. 31

Couchbase Mobile - Before

Load Balance

r

Couchbase Mobile v1.1.1 & lower

20,000 MaxConcurrent

CRUD , Security &

Seq. Number

HA DataStorage &Channel

Index

©2015 Couchbase Inc. 32

Problems – Single Global Sequence Number

Updating a single key at scale causes too many problems

©2015 Couchbase Inc. 33

Solution - DCP (Database Change Protocol)Cluster data replication via sequential

streaming feed from each vBucket (shard) on each node called “DCP”

©2015 Couchbase Inc. 34

Sync Gateway Accelerator (Enterprise Only)

Couchbase Mobile v1.2+Accelerator listens to DCP to create k/v channel indexes & uses DCP sequence

numbersSame vLAN

©2015 Couchbase Inc. 35

Horizontally Scale All Three Parts in v1.2+

LBx,000,000

+Concurren

t CRUD &

Security

HA Data

Storage

Channel Index &

Seq. Number

©2015 Couchbase Inc. 36

ConsolidationKey/Value

& Document

Query&

Index

Mobile

Search

Analytics