Dropping ACID with MongoDB

Preview:

DESCRIPTION

http://yapc2010.com/yn2010/talk/2578

Citation preview

Who am I?

what is ?

Application

Perl

Linux

Apache

Application

Perl

Linux

DatabaseApache

Application

Perl

Windows

Apache

Application

Perl

Linux

Apache

Application

Perl

Linux

Apache

Application

Perl

Linux

Apache

Application

Perl

Linux

Apache

The world's mostpopular open source

database

Application

Perl

Linux

Apache

The world's mostadvanced open source

database

databases

contain tables

contain rows

databases

contain tables

contain rows

schema

databases

contain tables

contain rows

joins

schema

databases

contain tables

contain rows

joins

schema

Application

Perl

Linux

Apache

getting started

www.mongodb.org

cpan -i MongoDB

$conn = MongoDB::Connection->new;

$conn = MongoDB::Connection->new;$db = $conn->foo;

$conn = MongoDB::Connection->new;$db = $conn->foo;

NO ADMINISTRATION

$conn = MongoDB::Connection->new;$db = $conn->foo;$table = $db->bar;

$conn = MongoDB::Connection->new;$db = $conn->foo;$table = $db->bar;

server

databases

server

databases

contain tables

server

databases

contain tables

server

databases

contain tables

server

collections

databases

contain rows

server

contain tables collections

databases

contain rows

server

contain tables collections

databases

contain rows

server

contain tables collections

documents

databases

contain rows

server

contain tables collections

documents

schema

databases

contain rows

server

contain tables collections

documents

schema

joins

databases

contain collections

contain documents

server

$conn = MongoDB::Connection->new;$db = $conn->foo;$collection = $db->bar;

$conn = MongoDB::Connection->new;$db = $conn->foo;$collection = $db->bar;

NO ADMINISTRATION

$conn = MongoDB::Connection->new;$db = $conn->foo$collection = $db->bar;

$doc = {"name" => "kristina", "contact info" => { "twitter" => "@kchodorow", "email" => "kristina@10gen.com" }, "friends" => 400232, "pic" => \$file_str, "member since" => DateTime->now};

$conn = MongoDB::Connection->new;$db = $conn->foo$collection = $db->bar;

$doc = {"name" => "kristina", "contact info" => { "twitter" => "@kchodorow", "email" => "kristina@10gen.com" }, "friends" => 400232, "pic" => \$file_str, "member since" => DateTime->now};

$collection->insert($doc);

mongo is type-sensitive and type-rich

•null•boolean•integer•double•string•array•hash

•qr/fobar/i•\$bindata•DateTime•MongoDB::OID•MongoDB::Code•MongoDB::MinKey•MongoDB::MaxKey

$collection->find_one({ "name" => "kristina"});

$collection->find_one({ "name" => "kristina"});

$collection->find_one({ "contact.twitter" => "@kchodorow"});

$collection->find_one({ "name" => "kristina"});

$collection->find_one({ "contact.twitter" => "@kchodorow"});

$collection->find({ "member since" => { '$gt' => $yesterday, '$lt' => $now}});

$collection->find_one({ "name" => "kristina"});

$collection->find_one({ "contact.twitter" => "@kchodorow"});

$collection->find({ "member since" => { '$gt' => $yesterday, '$lt' => $now}})->sort({"friends" => 1});

$collection->find_one({ "name" => "kristina"});

$collection->find_one({ "contact.twitter" => "@kchodorow"});

$collection->find({ "member since" => { '$gt' => $yesterday, '$lt' => $now}})->sort({"friends" => 1})->limit(10);

$collection->find_one({ "name" => "kristina"});

$collection->find_one({ "contact.twitter" => "@kchodorow"});

$collection->find({ "member since" => { '$gt' => $yesterday, '$lt' => $now}})->sort({"friends" => 1})->limit(10) ->skip(100);

$collection->find_one({ "name" => "kristina"});

$collection->find_one({ "contact.twitter" => "@kchodorow"});

$cursor = $collection->find({ "member since" => { '$gt' => $yesterday, '$lt' => $now}})->sort({"friends" => 1})->limit(10) ->skip(100);

while (my $doc = $cursor->next) { # ... }

my @results = $cursor->all;

x < 3

{"x" => {'$lt' => 3}}

$gt$lte$ne$mod

$exists$type...

$collection->update( {"name" => "kristina"},

$collection->update( {"name" => "kristina"}, {'$set' =>

$collection->update( {"name" => "kristina"}, {'$set' => {"contact.website" => "snailinaturtleneck.com"}});

updates are...•atomic•in-place (when possible)•able to create and delete fields•able to create documents

$collection->update( {"url" => "example.com"}, {'$inc' => {"pageviews" => 1}}, {"upsert" => 1});

$set$inc$push$pop$pull...

$set$inc$push$pop$pull...

$set$inc$push$pop$pull...

$set$inc$push$pop$pull...

"$gte"

"<null>"

""

'$gte' or "\$gte"

Or define your own!

":gte""=gte""?gte"

Or define your own!

" gte"

Or define your own!

" gte"" ♕ gte"

Or define your own!

" gte"" ♕ gte""xgte"

features

I suddenly have a deep love for stored procedures

I suddenly have a deep love for stored procedures

now with stored

procedures!

$db->system->js->insert({ "_id" => "x", "value" => 3});

$db->system->js->insert({ "_id" => "y", "value" => 4});

$db->eval("return x+y");

my $log = <<<LOGfunction(msg, level) { var date = "[" + new Date() + "] "; var lvl = level ? level+" " : ""; print(date + lvl + msg);}LOG;

$db->system->js->insert({ "_id" => "log", "value" => $log});

$db->eval("log('all your base are belong to us', 'FATAL');");

$db->eval("log('all your base are belong to us', 'FATAL');");

[Fri May 19 18:34:57] FATAL all your base are belong to us

J J J

J J J

J J J

$metadata = { "_id" => " " "date" => DateTime->now "downloads" => 0, "user" => $userId, "filename" => "rickroll.flv"}

J

$grid = $db->get_gridfs;

$grid->put($fh, $metadata);

$grid = $db->get_gridfs;

$grid->put($fh, $metadata);

$file = $grid->find_one({ "filename" => qr/rickroll/i});

$grid = $db->get_gridfs;

$grid->put($fh, $metadata);

$file = $grid->find_one({ "filename" => qr/rickroll/i});

$file->print($fh2);

Benefits:• Automatic backups• Unlimited metadata• Millions of files per directory• Random access• Better disk locality

Geospatial Indexes

$coll->insert({location => (-40, 78)});

$coll->insert({location => (-40, 78)});

$coll->find({location => { '$near' => (-40, 77)}})

$coll->insert({location => (-40, 78)});

$coll->find({location => { '$near' => (-40, 77)}})->limit(10)

what doesn't it have?

transactions

are you practicing safedata storage?

Insert this.

Okay, got it.

Phew, my data's safe.

Write this to disk

All over it!

I know better than he does, I'll just let this sit in a buffer for a while.

? I have no idea what you're talking about.

trust no one!

...trust a bunch of ones. Mostly.

prod1.example.com

prod2.example.com

prod1.example.com

prod2.example.com

prod1.example.com

prod2.example.com

prod1.example.com

prod2.example.com

prod1.example.com

prod2.example.com

prod1.example.com

prod2.example.com

prod1.example.com ?

prod2.example.com

prod2.example.com

prod1.example.com

prod2.example.com

prod1.example.com

?

prod2.example.com

prod1.example.com

prod2.example.com

prod1.example.com

primary

secondary

passive

secondary

okay

?

make sure two slaves have this

all set

EXCEPTION

EXCEPTION

EXCE

PTIO

N

$conn->w(3);

$collection->insert({"x" => 1}, {"safe" => 1});

scaling

sometimes read-only

sometimes read-onlysometimes inconsistent

U - ZK - OA - E F - J P - T

U - ZK - OA - E F - J P - T

U - ZK - OA - E F - J P - T

config

mongod mongod mongod

mongos

mongos

mongod mongod mongod

I want posts from April 2009 - June 2009

mongos

mongod mongod mongod

April and May is on #2. June is on #3.

mongos

mongod mongod mongod

mongos

mongod mongod mongod

mongos

mongod mongod mongod

Add this post for March 26, 2010.

mongos

mongod mongod mongod

#3 hold June 2009-present

config

mongod mongod mongod

mongos

So, what's going on?

config

mongod mongod mongod

mongos

$conn = MongoDB::Connection->new;# ...

try.mongodb.org

www.mongodb.org

thank you!

@kchodorow

http://www.snailinaturtleneck.com

Recommended