DrupalCon Chicago 2011 talk by douggreen and chx on MongoDB and Drupal, see also http://chicago2011.drupal.org/sessions/practical-mongodb-and-drupal
- 1.
- Practical MongoDB and Drupal
- Doug Green (douggreen)
- Karoly Negyesi (chx)
- 2. #drupalchi, #drupalchimongo for questions
3.
http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal
4.
- Design for Speed and Scalability
5.
I work at Examiner.com, aQuantified top 100 North American
website, probably the highest traffic Drupal 7 website in the
world.Most minutes on most days we delivercustomized non-cached
content to over a thousand people, every page built to the
individual based on location or preference.In the time it takes you
to count the number of people near you, we'll probably touch more
people. well maybe not in this room today,but still you'll have to
count fast! 6.
... our daily traffic has increased and our monthly availability
has gone from 99% uptime to 99.97% uptime hours of downtime are now
measured in minutes.Just today we hit another record day, breaking
the record from just yesterday, breaking the record from last
week.That's all significant! - Doug Green, Drupal Watchdog, March
2011 7. Design for Speed and Scalability
- Understand databases, queries, and indexes, and know the pros
and cons and tradeoffs
- Cache, memcache entities, Drupal render cache, mongodb block
cache
8. Never call a web page from your web page 9. CSS and JS
aggregation 10. CDN's 11.
RDBMS traditional SQL solutions NoSQL solutions but let's change
the name ... Document Object DB NO sql 12.
Document-oriented Database ? DODB ? Distributed Key-Value Pair
Graph Database ? We really can't agree, I think NoSQL is the wrong
Frame, let's just call this a Document-oriented Database Management
System or DODBMS 13.
14.
15.
- A query is just a query in a different language
16.
- Open Source?Community? Support?
17.
- DODBMS built for 2010 not 1980
18.
-
- Fast, document structure is perfect for Drupal entities
- 19. Takes best advantage of 2010 Operating Systems that manage
memory well
20. Takes best advantage of 2010 Hardware the recognizes disk
and memory is cheap 21. Open source code can be read and modified
(40-64 index example)
-
- Bleeding edge, software is new and not as well battle
tested
- 22. Lack of GUI based tools
23. Small user community and new language that must be learned
24. 10gen support is excellent, free and paid (but as with all
companies they are seeking ways of funding their efforts, so this
is changing) 25.
- SELECT * FROM foo WHERE bar = 1;
- This is the session on Practical MongoDB so let's do a few
more...
26.
- SELECT some FROM foo WHERE bar = 1;
- db.foo.find({bar:1}, {some:1});
27.
- INSERT INTO foo (bar) VALUES (1);
28.
- CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES
(1);
- Inserts are simple, but to be fair...
29.
- CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES
(1);
- db.foo.insert({bar:1}); db.foo.insert({bar:1});
- Inserts are simple, but be more fair...
30.
- db.foo.insert({bar:1}); db.foo.find(); { _id: ObjectId(4d...),
bar : 1 }
- Inserts are simple, but let's be accurate, mongo auto-creates
unique keys
31.
- db.foo.insert({bar:1, _id:123456}); db.foo.find(); { id: 123456
bar : 1 }
- Or you can add your own unique key
32.
- INSERT foo set some= 1 WHERE bar = 2;
- db.foo.update({bar:2}, {some:2}); db.foo.update({bar:2},
{$set:{some:2}});
- Updates aren't that hard either
33.
- db.foo.update({_id:2}, {$set:{bar:1}},true);
- But then there's upserts!
34.
- But, we use Drupal, not the database command prompt, so...
chx 35.
- FieldAPI is perfect for MongoDB!
36.
-
- 37. Blocks conditions in DB instead of code
38. Queues mostly to avoid another daemon and it's fast enough
39. Watchdog capped collections, never run out of space, counts and
groups by id 40. Cache not used on Examiner.com 41. Session perfect
fit for D7
- http://drupal.org/project/mongodb
42.
- t() replaceable arguments i.e., page not found: %q
43.
- Arguments filled in on details
44.
- $query = new EntityFieldQuery; $results
->entityCondition('entity_type', 'node')
->propertyCondition('uid', (int) $account->uid)
->fieldCondition('bar', 'value', 1)
->propertyCondition('status', 1) ->execute(); $nodes =
empty($results['node'])? array():
node_load_multiple(array_keys($result['node']));
45.
- $query = array( 'uid' => (int) $account->uid, 'bar.value'
=> 1, 'status' => 1 ); $fields = array( '_id' => TRUE, );
$results = mongodb_collection('fields_current', 'node')
->find($query, $fields); foreach ($results as $result) { }
46.
- You can only traverseone branch at a time
47.
- 1. The sort column must be the last column used in the
index.
- find({a:1}).sort({a:1}); find({a:1}).sort({b:1})
find({a:1,b:2}).sort({c:1});
- Assume an index on a, b, c on all examples to follow
48.
- 2. The range query must also be the last column in an index,
this is an corollary of 1 above.
- find({a:1, b:{$gt:2}); find({a:{$gt:1, $lt:10}})
find({a:{$gt:1}).sort({a:1});
49.
- 3. Only use a range query or a sort on one column.
- find(a=1,b=2).sort(c) find(a=1,b>2)
find(a=1,b>2,b2).sort(b)
- find(a>1,b>2) find(a=1,b>2).sort(c) find(a=1,b in [1,
2]).sort(c)
- Note, I'm switching to psuedo code shorthand, not BSON
syntax
50.
- 4. Conserve indexes by re-ordering columns used in straight =
queries If you have the following two queries, then you only need a
single index a, b, d, c find(a=1, b=1, d=1) find(a=1, b=1, c=1,
d=1)
51.
-
- Examiner.com is mostly happy with MongoDB solution.
- 52. We miss relational nature of MySQL with a GUI query
reporting tool, all reports have to be programmed.
53. Site uptime is much higher than before, but still have some
instability related to an immature product 54. Drupal 7, Field API,
DBTNG, MongoDB, and Community are awesome 55.
-
- DBTNG, will require hooks for modules to provide translations
of relational SQL to Document object store, we could provide this
for core, but each contrib module would then also need to provide
this not ideal, but it is a solution
- 56. I think MongoDB is a perfect fit for search, and I'm the
right guy to do it, but every time I mention this, people say, but
we have SOLR, why would you do this?
57.
- Locate this session on the DCC website:
- http://chicago2011.drupal.org/sessions
-
http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal
- Click the Take the Survey link.