GeekCamp SG 2009 - CouchApps with CouchDB

#geekcampsg 2009CouchApps - Applications built on CouchDB

Arun Thampi@iamclovin

http://mclov.in

Saturday, August 22, 2009

http://mclov.in

http://mclov.in

About myself

• Ruby Developer + Dabbler at Wego.com

• The kick-ass travel meta-search engine

• Work with great set of people

@chuyeow@winstonyw @reds @mongeslani

@jefflimar

@phungleson @garytheis@ijuzwannatwit


CouchDB FTW• Schema-less Key-Value based

document store

• Erlang

• Speaks HTTP / REST

• Map Reduce

• JSON

• Social Media

• MVCC (No Locks)

• Replication

• Scaling out of the box (HAProxy/Nginx)

• Full Text Search (Lucene/Sphinx)

• Attachments (binary files)

• Web Interface (Futon)


Profit


CouchDB - Built for the Web

{ “_id” : “abc”, “_rev” : “abc123def”, “text” : “Hello World”, “screen_name” : “mclovin”, “timestamp” : 123232, “type” : “thread”, “replies”: [ “reply 1”, “reply 2”, “reply 3”]}


CouchDB - Built for the webInserting a document HTTP PUT

Retrieving a document HTTP GET

Deleting a document HTTP DELETE

Updating a document HTTP POST

Scaling Replication + load balancing

Querying ?



Map Reduce with Javascript Views(No I wasn’t making that up)

Document 1

Document 2

Document n

Map Function

[Key 1, Value 1]

Reduce Function[Key 2, Value 2]

[Key k, Value k]

Value X


Example Map Reduce

function(doc) { if(doc.type == “customer”) { emit(doc.first_name, 1); }}

Map Function


Example Map Reduce (contd)Reduce Function

function(keys, values, re) { return sum(values);}


Example Map Reduce (contd){ .. { “key”: “Seth”, “value”: 1 }, { “key”: “Fogell”, “value”: 1 } ..}

reduce([“Seth”, “Fogell”], [1, 1])


CouchApps

• HTTP / HTML / Javascript is the language of the web

• Coincidentally, that’s what CouchDB speaks too

• Why not have CouchDB apps serve apps?


CouchApps - Why?

• Barriers to entry to develop powerful data-backed applications is lowered

• Replication enables load-balancing + scaling

• No Impedance Mismatch - Data is Javascript, your views are in Javascript, your HTML pages are rendered in Javascript


CouchApps

• Available at http://github.com/couchapp/couchapp

• couchapp generate icanhazthread

• couchapp push icanhazthread http://127.0.0.1:5984/icanhazthread


http://github.com/couchapp/couchapp




http://127.0.0.1:5984/icanhazthread




CouchApps - List & Show Functions

• A view is queried and the results are passed to a list function

• List Function then renders the results of your view in whatever format you please: HTML, JSON, XML, RSS, etc

• In a twisted way, its like a controller + view concept in an MVC framework

Model Schema-less JSON

Controller Map/Reduce View

View List/Show


Introducing i.canhazthread.com

• Inspired (in fact, cloned) from a.tinythread.com by Joshua Schacter

• Backend powered by a CouchApp (http://github.com/arunthampi/icanhazthread)

• Uses Rails to do Twitter authentication + HTML rendering


http://github.com/arunthampi/icanhazthread




icanhazthread.com

nginx

Rails Mongrels CouchDB

CouchDB API [RSS, Realtime]


Document Structure

{ “_id” : “abc”, “_rev” : “abc123def”, “text” : “Hello World”, “screen_name” : “mclovin”, “timestamp” : 123232, “type” : “thread”}

{ “_id” : “def”, “_rev” : “234abcdef”, “text” : “oh hai”, “screen_name” : “snehamenon”, “timestamp” : 1232310, “type” : “reply”, “thread_id” : “abc”}

Thread Reply


Interesting Map/Reduce problem(at least for me)

• Interesting because replies are not contained within a thread

• Also, because threads on the homepage need to be sorted according to timestamp (newest ones first)

• Views need to be commutative & associative


Show me the code

function(doc) { if(doc.type == ‘thread’) { emit([doc.id, doc.time_created], doc); } else if(doc.type == ‘reply’) { emit([doc.thread_id, doc.time_created], doc); }}

Map Function (yawn)


Show me the code

[K1, V1] [K2, V2]

[K1, RV1]

[K1, RV2]

group_level =1

[K1, Summary for Thread 1]

[K2, Summary for Thread 2]

num_replies: 2

num_replies: 0

Sorting?

Thread Reply Result

Reduce Function


Future Directions

• Make icanhazthread real-time (Utilize the _changes API in CouchDB)

• RSS Feeds (Dead simple with the CouchApp list)

• etc etc


Resources

• CouchDB Wiki: http://wiki.couchdb.org

• CouchDB Mailing lists (user + dev)

• Blogs: Chris Anderson, Harish Mallipeddi, Jan Lenhardt

• CouchDB Futon Test Suite (http://127.0.0.1:5984/_utils/couch_tests.html)

• Other CouchApps: Toast, Chris Anderson’s blog


http://127.0.0.1:5984/_utils/couch_tests.html




Thank you

Q & A


Technology

GeekCamp SG 2009 - CouchApps with CouchDB