Webinar - Getting Started with Couchbase App Development

Technical Evangelist

twi0er: @tgrall email: [email protected]

Tugdual “Tug” Grall

Ge;ng Started with Couchbase (Europe)

Technical Evangelist

twi0er: @scalabl3 email: [email protected]

Jasdeep Jaitla

Ge;ng Started with Couchbase (US)

RDBMS vs Couchbase

RDBMS vs Couchbase

• It's Different but Familiar!

• Think in New Ways!

• Simpler in many cases!

• New Pa0erns! • Faster/Lighter/Agile! • "Unlearn what you have Learned!"

Remember: Data is Data; databases are just opYmizing how to store and retrieve it. Objects are Classes with Data and Methods.

RDBMS OrganizaYon

• RDBMS organizes data as tables

-‐ Tables represent data in rows, n columns of m rows

-‐ Table rows have a specific schema, each column as a staYc type

-‐ Simple Datatypes: strings, numbers, dateYmes, booleans, can be represented by columns in a single table

-‐ Complex Datatypes: dicYonaries/hashes, arrays/lists can not be represented in a single table [Impedence Mismatch]

• All rows have idenYcal schema, schema changes require taking database offline

• Reading/WriYng/TransacYons require mutex and locking

Couchbase OrganizaYon

• Couchbase operates like a Key-‐Value Document Store

-‐ Simple Datatypes: strings, numbers, dateYme, boolean, and binary data can be stored; they are stored as Base64 encoded strings

-‐ Complex Datatypes: dicYonaries/hashes, arrays/lists, can be stored in JSON format (simple lists can be string based with delimiter)

-‐  JSON is a special class of string with a specific format for encoding simple and complex data structures

• Schema is unenforced and implicit, schema changes are programmaYc, done online, and can vary from Document to Document

Complex Datatypes

• Can represent both simple and complex data types in JSON data structures

• Can modify schema on the fly, and Documents of a specific "type" can vary in schema

• "Type" is arbitrary, it's a programming strategy, there are no actual "types", but it's typical to embed the class name as a "doctype" json key

Couchbase

{!"doctype": "User","name": "John Doe",!"email": "[email protected]",!"age": 38,!"gender_male": true,!"created_at": "2013-09-20 23:59:59",!"items_viewed": [!

"12345", "23456", 34567"!],!"preferences": {!

"email_notifications": true,!"sms_notifications": false!

},!"authored": [!

{ "title": "Couchbase Models",!"price": 49.95 }]}!

Architecture

Install Couchbase

• Install Couchbase by following InstrucYons • Ajer installaYon, open your browser to h0p://localhost:8091/ • Finish setup step prompts, now you have a running Couchbase server with a default bucket

www.couchbase.com/download

Disk Que

ue

Disk

ReplicaYon Queue

Couchbase Server Node

To other node

Managed Cache

App Server

Doc 1 Doc 1

Doc 1

Write OperaYon

Disk Que

ue

Disk

ReplicaYon Queue


To other node

Managed Cache

App Server

Doc 1’

Doc 1

Doc 1’ Doc 1

Doc 1’

.

Update OperaYon

Disk Que

ue

Disk

ReplicaYon Queue


To other node

Managed Cache

GET

Doc 1

2

App Server

Doc 1

Doc 1 Doc 1

Read OperaYon

SDK's

Setup SDK

• Each supported SDK page has instrucYons for setup • PHP, Ruby, NodeJS and Python clients are wrappers around libcouchbase C library, so libcouchbase must be installed first

• For other community clients, click on All Clients on lej nav, scroll down the page and you can see clients for Go, Erlang, Clojure, TCL and Perl.

www.couchbase.com/communiYes

Make a ConnecYon

require 'rubygems'!require 'couchbase'! !cb = Couchbase.connect(!

:bucket => "default",!:hostname => "localhost")!!

data = { jsonkey: "value", created_at: Time.now }!!cb("mydata", data)!puts cb.get("mydata")!

#!/usr/bin/env python!from couchbase import Couchbase!from pprint import pprint!from datetime import datetime!!cb = Couchbase.connect(bucket='default')!!data = { "jsonkey": "value", "created_at": datetime.now() }!!cb.add('mydata', data)!result = cb.get('mydata')!pprint(result.value, indent=4)!

RUBY

PYTHON

Make a ConnecYon

import com.couchbase.client.CouchbaseClient;!import java.net.URI;!import java.util.*;!import com.google.gson.Gson;import com.google.gson.GsonBuilder;!public class HelloWorld {! !public static void main(String[] args) throws Exception {! !List<URI> hosts = Arrays.asList(!new URI("http://127.0.0.1:8091/pools")!);!!CouchbaseClient cb = new CouchbaseClient(hosts, "default", "");!Gson json = new Gson();!!Hashtable data = new Hashtable();!data.put("jsonkey", "value");!data.put("created_at", new Date());!!cb.add("mydata", json.toJson(data))!System.out.println(cb.get("mydata"));! !cb.shutdown();!}!}!

JAVA

Make a ConnecYon

var Couchbase = require('couchbase');!var cb = new Couchbase.Connection({bucket: "default"}, function(err) { });!var data = { jsonkey: "value", created_at: new Date().toString() };!cb.add("mydata", data);!!console.log(cb.get("mydata"));!

!

NODEJS

<?php!// adjust these parameters to match your installation!$cb = new Couchbase("127.0.0.1:8091", "", "", "default");!!$data = array("jsonkey" => "value", ! "created_at" => date("Y-m-d H:i:s"));!!$cb->add("mydata",json_encode($data));!var_dump($cb->get("mydata"));!!?>!

!

PHP

Fundamentals

• Couchbase is structured as a Key-‐Value store, therefore All Data has a Key and a Value

• Keys can be any string up to 250 characters long • Keys are unique, within a database (bucket), there can only be one instance of a key

• Keys are completely in the control of the applicaYon developer, there is no internal mechanism for key generaYon

• Values can be JSON, strings, numbers, binary blobs, or a special posiYve atomic counter (unsigned integer)

• Values can be up to 20MB in size

• get (key) –  Retrieve a document

• set (key, value) –  Store a document, overwrites if exists

• add (key, value) –  Store a document, error/excepYon if exists

• replace (key, value) –  Store a document, error/excepYon if doesn’t exist

• cas (key, value, cas) –  Compare and swap, mutate document only if it hasn’t changed while execuYng this operaYon

Store & Retrieve OperaYons

Atomic Counter OperaYons These operaYons are always executed in order atomically. • incr (key) –  Increase an atomic counter value, default by 1

• cb.incr(“my_counter”) # now it’s 2

• decr (key) –  Decrease an atomic counter value, default by 1

• cb.decr(“my_counter”) # now it’s 1

Objects Serialized to JSON and Back User Object

string uid

string firstname

string lastname

int age

array favorite_colors

string email

u::[email protected] { “uid”: 123456, “firstname”: “john”, “lastname”: “doe”, “age”: 22, “favorite_colors”: [“blue”, “black”], “email”: “[email protected]” }

User Object

string uid

string firstname

string lastname

int age

array favorite_colors

string email

u::[email protected] { “uid”: 123456, “firstname”: “John”, “lastname”: “Doe”, “age”: 22, “favorite_colors”: [“blue”, “black”], “email”: “[email protected]” }

add()

get()

Basic Key Pa0erns

Basic Keying

• Use a Unique value for key (email, username, sku, isbn, etc.)

-‐ Users • u::[email protected]

• u::jdoetw -‐ Products

• p::978-‐0321573513 [isbn] • Predictable Keys can follow Key-‐Value pa0erns (Users typically can be done this way and are the most numerous items)

• Unpredictable Keys (GUID, UUID, etc.) require Views (Map-‐Reduce Indexes) to find their documents

Counter-‐ID

ApplicaYon

incr("counter-‐key")

add("key" + counter_val, data)

ApplicaYon

count = get("counter-‐key")

mulY-‐get(keys[])

Data CreaYon

Iterate Through CollecYon

Counter-‐ID

• Similar to IDENTITY column in RDBMS

• CreaYng New Document is a pair of operaYons, INCR and ADD

-‐  IniYalize one Key as an Atomic Counter (I do at App Start)

-‐  Increment Counter and save new value

✴  id = client.incr("blog::couchbase_dev::comment_count")

-‐ Use the id as component of key for new document

✴ client.add(""blog::couchbase_dev::c"::" + id, self.to_json)

Lookup Pa0ern

ApplicaYon

add("[email protected]", "u::550e8400-‐e29b-‐41d4-‐a716")

add("u::550e8400-‐e29b-‐41d4-‐a716", data)

add("jdoetw", "u::550e8400-‐e29b-‐41d4-‐a716")

ApplicaYon

key = get("[email protected]")

get(key)

Data CreaYon

Data Retrieval

Lookup Pa0ern

• Create simple document that has referenYal data (Key) to primary document

-‐ Primary Document u::a2bf2-‐23317-‐2302

-‐ Lookup Document: u::[email protected] { u::a2bf2-‐23317-‐2302 }

• Lookup Documents aren't JSON, they should just be the Key as a string so you skip JSON parsing

• Requires Two GET operaYons, first GET Lookup, then GET primary Document

-‐ key = client.get("u::[email protected]")

-‐ doc = client.get(key)

Combine Counter-‐ID and Lookup

ApplicaYon

add("[email protected]", id)

add("u::" + id, data)

add("jdoetw", id)

ApplicaYon

key = get("[email protected]")

get(key)

Data CreaYon

Data Retrieval

id = incr("user::count")

Combine Counter-‐ID and Lookup

Pro's

• Binary OperaYons, overall faster than large volume of View Queries

• EssenYally creates several ways to find a single document

• Is always consistent, just like all other Binary operaYons

Con's

• Increases Number of Documents, therefore Metadata usage in RAM

-‐ But this is generally a non-‐issue for most people

Views

Map() FuncYon => Index

function(doc, meta) { emit(doc.username, doc.email)

}

Every Document passes through View Map() functions

Map

Single Element Keys (Text Key)

function(doc, meta) { emit(doc.email, doc.points)

} text key

Map

doc.email doc.points

[email protected] 1000



Compound Keys (Array)

function(doc, meta) { emit(dateToArray(doc.timestamp), 1)

} array key

Array Based Index Keys get sorted as Strings, but can be grouped by array elements

Map

dateToArray(doc.Ymestamp) value

[2012,10,9,18,45] 1

[2012,9,26,11,15] 1

[2012,8,13,2,12] 1

Most Common Query’s Are Ranges

doc.email meta.id

[email protected] u::1







?startkey=”b1” & endkey=”zZ”

Pulls the Index-‐Keys between UTF-‐8 Range specified by the startkey and endkey.

?startkey=”bz” & endkey=”zn”

Pulls the Index-‐Keys between UTF-‐8 Range specified by the startkey and endkey.

Index-‐Key Matching

doc.email meta.id








?key=”[email protected]”

Match a Single Index-‐Key

Index-‐Key Set Matches

doc.email meta.id








?keys=[“[email protected]”, “[email protected]”]

Query MulYple in the Set (Array NotaYon)

Key-‐Value Pa0erns vs Views

• Key Pa0erns only use Binary OperaYons • Documents are consistent, CAS operaYons can be used for race condiYons on parYcular documents

• Do not have a Yme gap, therefore can be faster

• Even as data size gets larger, doesn't increase latency

Key-‐Value Pa0erns

Key-‐Value Pa0erns vs Views

• As Documents are created/mutated, they are persisted

• Views (Indexes) do batch indexing for persisted Documents, incrementally from last batch process

• The process of persistence + indexing == Yme gap between CRUD OperaYons and showing in View query ==> Eventually Consistent

Views

Quick Demo

Q&A

Technology

Webinar - Getting Started with Couchbase App Development