CouchConf London Developing with Couchbase I: Getting Started

1  

Developing  with  Couchbase  Part  I:  Ge:ng  Started  

Raghavan  “Rags”  Srinivas  Developer  Advocate  rags@couchbase.com  

2  

•  Architect  and  Evangelist  working  with  developers  •  Speaker  at  JavaOne,  RSA  conferences,  Sun  Tech  Days,  JUGs  and  other  developer  conferences  

•  Taught  undergrad  and  grad  courses  •  Technology  Evangelist  at  Sun  Microsystems  for  10+  years  

•  SQll  trying  to  understand  and  work  more  effecQvely  on  Java  and  distributed  systems  

•  Couchbase  Developer  Advocate  working  with  Java  and  Ruby  developers  

•  Philosophy:  “Be#er  to  have  an  unanswered  ques1on  than  a  unques1oned  answer”  

Speaker  IntroducQon  

3  

•  Ge:ng  Started  •  An  introducQon  to  Documents  with  Couchbase  •  Couchbase  server  2.0  compaQble  client  libraries  •  Resources  and  Summary  •  Q/A  

Agenda  

4  

GETTING  STARTED:  DEVELOPMENT  ENVIRONMENT  

5  

Development  Environment:  Development  DB  

•  Downloads  at  couchbase.com/download  

5  

•  Provision  via  wizard  in  Web  Console  –  Or  provision  via  REST  interface:  operaQons  folks  to  automate  provisioning    

(i.e  Chef,  puppet,  Rightscale  rightscript)  

•  Linux:  Ubuntu  and  Red  Hat/CentOS  –  Packages  for  most  common  

distribuQons.  •  dpkg  -­‐i  ,  rpm  -­‐i,  etc.  

•  Mac  –  Download  a  .zip,  open,  drag  to  

ApplicaQons,  

•  Windows  –  Download  a  setup.exe,  double  click  

6  

Development  Environment:  Obtaining  a  Client  

•  High  performance,  official  client  libraries  for  Java,  .NET,  PHP,  Ruby,  C,  Python*  

•  Head  to  couchbase.com/develop  for  SDKs  where  you  will  find  –  Client  libraries  –  Screencasts  on  Ge:ng  Started  –  Ge:ng  started  guides  –  Tutorial  with  a  sample  applicaQon  –  A  complete  API  reference  

*Current  version  0.6  as  of  2012-­‐03-­‐13  

7  

Client  Setup:  Ge:ng  Cluster  ConfiguraQon  

7  

Couchbase  Server  Node  

Couchbase  Client  

h2p://myserver:8091/  

{ … "bucketCapabilities": [ "touch", "sync", "couchapi" ], "bucketCapabilitiesVer": "sync-1.0", "bucketType": ”couchbase", "name": "default", "nodeLocator": "vbucket", "nodes": [ ….  

Cluster  ConfiguraQon  over  REST  

Couchbase  Server  Node  

Couchbase  Server  Node  

Couchbase  Server  Node  

8  

Client  at  RunQme:  Adding  a  node  

8  

Couchbase  Client  

Couchbase  Server  Node  

Couchbase  Server  Node  

Couchbase  Server  Node  

Couchbase  Server  Node  

Couchbase  Server  Node  

Cluster ���Topology���Update

New  node    coming  online  

9  

Client  Set  up  at  a  Code  Level  

// Set up at least two URIs in case one server fails!URI server = new URI("http://10.1.6.171:8091/pools/");!List<URI> servers = new ArrayList<URI>(); servers.add(server);!!!// Now create a client talking to the default bucket!CouchbaseClient cbc = new CouchbaseClient(servers, "default", "");!!System.err.println(cbc.get(“Rags") + ! " is off Minding the Gap!!");!

Use  the  Java  client  with  your  favorite  JSON  library:  Je@son,  Google  GSON,  etc.  Use  other  client  libraries  such  as  yajl  for  Ruby  

10  

OperaQons  Available  to  a  Client  of  Couchbase  Server  

•  Store  OperaQons  –  Add:  Store  the  document  if  it  does  not  yet  exist  –  Set:  Store  the  document,  overwriQng  exisQng  if  necessary  

•  Retrieve  OperaQons  –  Get:  Fetch  the  document  –  Get  and  touch:  Fetch  the  document  and  update  the  TTL  –  MulQget:  Fetch  mulQple  documents  at  the  same  Qme  

•  Update  operaQons  –  Append/prepend:  Add  data  in  front  of  or  on  the  end  of  a  document  –  Delete:  Remove  the  document  from  the  store  –  Compare  and  Swap  (CAS):  Replace  the  current  document,  if  CAS  matches  –  Replace:  Replace  the  document  if  it  exists,  otherwise  do  not  –  Touch:  Update  the  TTL  for  a  document  

Note:  Not  all  Client  Libraries  are  created  equal  

11  

A  Distributed  Hash  Table  

ApplicaQon  

set(key,  value)   get(key)  returns  value  

DData  

Cluster  

Database  

12  

A  Distributed  Hash  Table  Document  Store  

ApplicaQon  

set(key,  json)   get(key)  returns  json  

DData  

Cluster  

Database  

{ !“id": ”brewery_Legacy”,!“type” : “brewery”,!"name" : "Legacy Brewing”,!“address": "525 Canal Street Reading", "updated": "2010-07-22 20:00:20", }!

13  

Distributed  System  Design:  Concurrency  Controls  

•  Compare  and  Swap  OperaQons  –  Olen  referred  to  as  “CAS”  –  OpQmisQc  concurrency  control  –  Available  with  many  mutaQon  

operaQons,  depending  on  client.  

•  Get  with  Lock  –  Olen  referred  to  as  “GETL”  –  PessimisQc  concurrency  control  –  Locks  have  a  short  TTL  –  Locks  released  with  CAS  

operaQons  –  Useful  when  working  with  object  

graphs  

13  

Actor  1   Actor  2  

Couchbase  Server  

CAS  mismatch  Success  

A  

B  

F  

C   D  

E  

14  

Common  QuesQons  with  Metadata  

Document  Metadata:  •  TTL  •  CAS  value  •  Flags  

Q:  What  happens  to  a  document  persisted  aKer  it’s  TTL?  

A:  A  regular  background  job  will  remove  expired  documents.    

Q:  How  do  I  use  flags?  

A:  Frequently  used  to  idenQfy  data  type  or  other  anributes,  such  as  compression.    Behavior  varies  from  client  to  client.  

15  

INTRODUCING  DOCUMENTS  

16  

A  JSON  Document  

16  

{! “id": "beer_Hoptimus_Prime",! “type”: “beer”,! "abv": 10.0,! "brewery": "Legacy Brewing Co.",! "category": "North American Ale",! "name": "Hoptimus Prime",! "style": "Imperial or Double India Pale Ale",!}!

The primary key

A float

The type information

17  

Other  Documents  and  Document  RelaQonships  

{ ! “id": "beer_Hoptimus_Prime",! “type” : “beer”, ! "abv": 10.0,! "brewery": ”brewery_Legacy_Brewing_Co",! "category": "North American Ale",! "name": "Hoptimus Prime",! "style": “Double India Pale Ale”!}!

{ ! “id": ”brewery_Legacy_Brewing_Co”,! “type” : “brewery”,! "name" : "Legacy Brewing Co.", "address": "525 Canal Street Reading, Pennsylvania, 19601 United States", "updated": "2010-07-22 20:00:20", "latitude": -75.928469, "longitude": 40.325725!}!

Afterthought

18  

Simplicity  of  Document  Oriented  Datastore  

•  Schema  is  opQonal  –  Technically,  each  document  has  an  implicit  schema  –  Extend  the  schema  at  any  Qme!  

•  Need  a  new  field?    Add  it.    Define  a  default  for  similar  objects  which  may  not  have  this  field  yet.  

•  Data  is  self-­‐contained  –  Documents  more  naturally  support  the  world  around  you,  the  data  structures  around  you  

•  Model  data  for  your  App/Code  instead  for  the  Database  

19  

Adding  a  Document:  ObservaQons  and  ConsideraQons  

•  ObservaQons  –  Conversion  to  document  was  very  simple,  many  JSON  opQons  –  Flexible  schema:  Did  not  need  to  add  the  laQtude  and  longitude    to  

every  record  –  Flexible  schema:  Can  add  the  brewery  detail  later    

•  ConsideraQons  –  Use  a  “type”  field  for  high  level  filtering  on  object  types  

–  Why  use  a  parQcular  key/_id  :  ”beer_My_Brew”!–  Should  I  have  a  TTL?  

20  

Common  QuesQons  when  AdopQng  Couchbase  

Q:  What  if  I  need  to  fetch  referenced  documents?  A:  Simply  get  them  one  aler  another  or  use  another  View.        

Q:  How  can  I  update  just  a  small  por1on  of  a  document?  A:  The  best  approach  is  to  keep  the  document  model  live  in  your  applicaQon,  then  use  CAS  operaQons  to  store  modified  documents.    The  Ruby  sample  applicaQon  has  a  good  example.    

Q:  I  currently  use  serialized  objects  with  memcached  or  Membase,  can  I  do  this  s1ll  with  Couchbase  Server?  A:  Absolutely!    Everything  previously  supported  and  used  is  sQll  there.    JSON  offers  advantages  with  heterogenenous  plasorm  support  and  preparing  for  Couchbase  2.0  views.  

21  

COUCHBASE  SERVER  2.0    COMPATIBLE  LIBRARY  FEATURES  

22  

Couchbase  Server  2.0:  Views  

•  Views  can  cover  a  few  different  use  cases  –  Simple  secondary  indexes  (the  most  common)  –  AggregaQon  funcQons  

•  Example:  count  the  number  of  North  American  Ales  

–  Organizing  related  data  

23  

Indexing  and  Querying  

APP  SERVER  1            

COUCHBASE  CLIENT  LIBRARY      

§ Indexing  work  is  distributed  amongst  nodes  §  Large  data  set  possible  §  Parallelize  the  effort  

§ Each  node  has  index  for  data  stored  on  it  

§ Queries  combine  the  results  from  required  nodes  

CLUSTER  MAP  

Doc  4  

Doc  2  

Doc  5  

SERVER  1  

Doc  6  

Doc  4  

SERVER  2  

Doc  7  

Doc  1  

SERVER  3  

Doc  3  

APP  SERVER  2            

COUCHBASE  CLIENT  LIBRARY      CLUSTER  MAP  

Doc  9  

Doc  7  

Doc  8   Doc  6  

Doc  3  

DOC  

DOC  

DOC  

DOC  

DOC  

DOC  

DOC  

DOC  

DOC  

DOC  

DOC  

DOC  

DOC  

DOC  

DOC  

Doc  9  

Doc  5  

DOC  

DOC  

DOC  

Doc  1  

Doc  8   Doc  2  

Replica  Docs   Replica  Docs   Replica  Docs  

AcQve  Docs   AcQve  Docs   AcQve  Docs  

COUCHBASE  CLIENT  LIBRARY      

COUCHBASE  CLIENT  LIBRARY      CLUSTER  MAP  

CLUSTER  MAP      

APP  SERVER  1   APP  SERVER  2  

Query  Response  

24  

RESOURCES  AND  SUMMARY  

25  

•  Couchbase  Server  Downloads  –  hnp://www.couchbase.com/downloads-­‐all  –  hnp://www.couchbase.com/couchbase-­‐server/overview  

•  Developing  with  Client  libraries  –  hnp://www.couchbase.com/develop/java/current  (example)  

•  Couchbase  forums  –  hnp://www.couchbase.com/forums/    

Resources,  Summary  and  Call  For  AcQon  

26  

THANKS  -­‐  Q&A  

26  

Raghavan “Rags” Srinivas, Couchbase Inc.  (rags@couchbase.com, @ragss)