Let the Tiger Roar! MongoDB 3.0

Alvin Richards Senior Director, Engineering alvin@mongodb.com @jonnyeight

Agenda

MongoDB 3.0 Pluggable Storage Engine API Storage Engines

MMAPv1 WiredTiger WT vs MMAPv1

Improvements

MongoDB 3.0

h"p://www.pixelresort.com/wp-content/uploads/2013/06/3point0.jpg

3.0 is like your "classic " vinyl

MongoDB 3.0

Pluggable Storage Engine API Storage Engines Large Replica Sets Big Polygon Security Enhancements SCRAM-SHA-1 Query Introspection Audit & Logging Simplified Operations Ops Manager Tools Rewrite

MongoDB 3.0 is a bag full of goodies!

Storage Engine

h"p://files.ecomagina@on.com/wp-content/uploads/2012/08/PowerHaul-Engine-Green_844x680.jpg

How does MongoDB persist data?

Pluggable Storage Engine API

h"p://www.livingincebuforums.com/ipb/uploads/monthly_10_2011/post-198-0-67871200-1318223706.jpg

Storage Engine API

Allows to "plug-in" different storage engines Different workloads require different performance

characteristics mmapv1 is not ideal for all workloads More flexibility

Can mix storage engines on same replica set/sharded cluster

And more in the future J

MMAPv1

h"ps://angrytechnician.files.wordpress.com/2009/05/memory.jpg

MMAPv1

MMAPv1

Improved concurrency control Database Locking -> Collection Level Work now parallelized between collections

Data & Indexes memory mapped into virtual address space

Data access is paged into RAM

OS evicts using LRU

More frequently used pages stay in RAM

WiredTiger

What is WiredTiger?

Storage engine company founded by BerkeleyDB alums Recently acquired by MongoDB Available as a storage engine option in MongoDB 3.0

Why is WiredTiger Awesome

Document-level concurrency Disk Compression

Index Prefix compression Consistency without journaling Better performance on certain workloads

write heavy

Improving Concurrency

2.2 Global Lock 2.4 Database-level Locking 3.0 MMAPv1 Collection-level Locking 3.0 WT Document-level

Writes no longer block all other writes Higher level of concurrency leads to more

throughput & lower latency

Compression

WT uses snappy compression by default Data is compressed on disk 2 supported compression algorithms

snappy: default. Good compression, relatively low overhead

zlib: Better Indexes are compressed using prefix

compression Allows compression in memory

Consistency without Journaling

MMAPv1 uses write-ahead log (journal) to guarantee consistency

WT doesn't have this need: no in-place updates Write-ahead log committed at checkpoints

2GB or 60sec by default configurable! No journal commit interval: writes are written to

journal as they come in Better for insert-heavy workloads

Replication guarantees the durability

MMAPv1 vs. WT

How to run MMAPv1

MMAPv1 is the default

How to run WT

mongod now has --storageEngine option

MMAPv1 Database Files

MMAPv1 persists data to files per databases Indexes Data

WT Database Files

Each collection & indexes stored in own file

Playing nice together

Can not Can't copy database files Can't just restart w/ same dbpath

Yes we can! Initial sync from replica set works perfectly! mongodump/restore

Rolling upgrade of replica set to WT: Shutdown secondary Delete dbpath Relaunch w/ --storageEngine=wiredTiger Rollover

Other WT configuration options

Compression: --wiredTigerCollectionBlockCompressor

YAML format for configuration

Gotcha's!!!

No 32-bit Support WT is 64bit only

system.indexes & system.namespaces deprecated Explicit commands: db.getIndexes() db.getCollectionNames()

h"ps://@ngbudongchine.files.wordpress.com/2012/08/lemonde1.jpeg

Small Demo

Improvements

Benefits

Wider Range of Use Cases

How: Flexible Storage Architecture Fundamental rearchitecture, with new pluggable storage engine API Same data model, same query language, same ops But under the hood, many storage engines optimized for many use

cases

Single View Content Management

Real-Time Analytics Catalog

Internet of Things (IoT) Messaging

Log Data Tick Data

Up to 95% Lower Operational Overhead

How: MongoDB Ops Manager The best way to run MongoDB Automates core management

tasks Single-click provisioning, scaling,

upgrades, administration Monitoring, with charts,

dashboards & alerts on 100+ metrics

Backup and restore, with point-in-time recovery

7x-10x Performance, 50%-80% Less Storage

How: WiredTiger Storage Engine Same data model, same query

language, same ops Write performance gains driven

by document-level concurrency control

Storage savings driven by native compression

100% backwards compatible Non-disruptive upgrade

MongoDB 3.0 MongoDB 2.6

Performance

h"p://www.humanandnatural.com/data/media/178/badan_jaran_desert_oasis_china.jpg

Please go and use it!

3.0.0 https://www.mongodb.org/downloads https://jira.mongodb.org/secure/Dashboard.jspa

h"p://www.@nypm.com/blog/wp-content/uploads/2015/01/hammer.jpg

h"p://www.humanandnatural.com/data/media/178/badan_jaran_desert_oasis_china.jpg

Questions?

@jonnyeight alvin@mongodb.com

h"p://www.mandywalker.com.au/wp-content/uploads/2013/07/Wall-with-Tools.jpg

We are hiring!

http://www.mongodb.com/careers