Persistence Smoothie: Blending SQL and NoSQL (RubyNation Edition)

Persistence SmoothieBlending SQL and NoSQL

Michael BleighIntridea, Inc.

photo by Nikki L. via Flickr

Saturday, April 10, 2010


present.ly


The Buzz


You’ve (probably)heard a lot about

NoSQL


NoSQL is a new way to think about

persistence


AtomicityConsistency

IsolationDurability


DenormalizationEventual Consistency

Schema-FreeHorizontal Scale

Map Reduce


Map Reduce

•Massively parallel way to process large datasets

• First you scour data and “map” a new set of data

• Then you “reduce” the data down to a salient result


map = function() { this.tags.forEach(function(tag) { emit(tag, {count: 1}); });}

reduce = function(key, values) { var total = 0; for (var i = 0; i < values.length; i++) { total += values[i].count; return {count: total};}


NoSQL tries to scale (more) simply


NoSQL is going mainstream


New York TimesBusiness InsiderBBC ShopWikiGitHub Meebo

Disqus SourceForgeSony Digg


...but not THAT mainstream.


A word of caution...



NoSQL can divide by zero

NoSQL doesn’t

sleep, it waits

NoSQL counted to infinity, twiceSaturday, April 10, 2010

NoSQL is a (growing) collection of tools, not

a new way of life


The Ecosystem


Key-Value Stores

•Redis

•Riak

• Voldemort

• Tokyo Cabinet

•MemcachedDB


Document Stores

•MongoDB

•CouchDB• Riak

• FleetDB


Column(ish) Stores

•Cassandra

•HBase


Graph Databases

•Neo4j

•HypergraphDB

•InfoGrid


When should I use this stuff?


Complex, slow joins for “activity stream”


Complex, slow joins for “activity stream”

Denormalize, use Key-Value Store


Variable schema,vertical interaction


Variable schema,vertical interaction

Document Databaseor Column Store


Modeling deep relationships


Modeling deep relationships

Graph Database


NoSQL solves real scalability and data

design issues


Ben Scofieldbit.ly/state-of-nosql


Ready to go?


Just one problem...


Your data is already in a SQL database


So now we need to ask the question...



Yeah, it blends.


The “Hard” Way:Do it by hand.


class Post include MongoMapper::Document key :title, String key :body, String key :tags, Array key :user_id, Integer def user User.find_by_id(self.user_id) end def user=(some_user) self.user_id = some_user.id endend

class User < ActiveRecord::Base def posts(options = {}) Post.all({:conditions => {:user_id => self.id}}.merge(options)) endend


Pros & Cons

• Simple, maps to your domain

• Works for small, simple ORM intersections

• MUCH simpler in Rails 3

• Complex relationships are a mess

• Makes your models fat

• As DRY as the ocean


The “Easy” Way:DataMapper


DataMapper

•Generic, relational ORM

• Speaks pretty much everything you’ve ever heard of

• Implements Identity Map

•Module-based inclusion


DataMapper.setup(:default, "mysql://localhost")DataMapper.setup(:mongodb, "mongo://localhost/posts")

class Post include DataMapper::Resource def self.default_repository_name; :mongodb; end property :title, String property :body, String property :tags, Array belongs_to :userend

class User include DataMapper::Resource property :email, String property :name, String has n, :postsend


Pros & Cons

• The ultimate Polyglot ORM

• Simple relationships between persistence engines are easy

• Jack of all trades, master of none

• Perpetuates (sometimes) false assumptions

• Legacy stuff is in ActiveRecord anyway


Show and Tell: Social Storefront



The Application

• Dummy version of a store that lets others “follow” your purchases (like a less creepy version of Blippy)

• Four requirements:

• users

• purchasing

• listings

• social graph


Users

• I already have an authentication system

• I’m happy with it

• It’s Devise and ActiveRecord

• Stick with SQL


Purchasing

• Users need to be able to purchase items from my storefront

• I can’t lose their transactions

• I need full ACID

• SQL Again


Social Graph

• I want activity streams and one and two way relationships

• I need speed

• I don’t need consistency

• I’ll use Redis


Product Listings

• I am selling both books about Ruby and movies about zombies

• They have very different properties

• Products are relatively non-relational

• I’ll use MongoDB


Demo and Walkthrough


Wrapping Up


These systems can (and should) live and

work together


Most important step is to actually think about data design


When you have a whole bag of tools, things stop looking

like nails


@mbleigh


Questions?


Technology

Persistence Smoothie: Blending SQL and NoSQL (RubyNation Edition)