MATH is Hard :TTL Index Configuration and

Considerations

Kimberly Wilkins

Senior DBA, ObjectRocketkimberly@objectrocket.com

Background

• 16 yrs in databases, operations, replication &

infrastructure

• MongoDB

• Oracle (RAC, Exadata, Golden Gate, Enterprise Mgr)

• Vertica

• MySQL

• Supporting databases of all sizes and types

• Senior DBA @ ObjectRocket

• What and Why

• Design and Creation

• Considerations and Gotchas

• MATH and Queries

• Enabling / Disabling

Overview

What is a TTL Collection/Index - “Time To Live”

• Starting in 2.2, normal collection - special TTL index type

• TTL index defines how many seconds a document exists

• TTLMonitor - mongod background thread

• Why/When to Use TTL Indexes

• Control collection/database size

• Automatically remove data based on BSON date type

• No MANUAL mass or batch deletes or cron jobs

• Any time data needs to persist only for a specific period of time

• Useful with short term event data, logs, cache, and session info

TTL Collections and Indexes

Goal?

• Save space? Improve Performance?

• Data retention policies?

• Business logic or requirements?

• System load? More Reads vs. Writes?

Use Expire AFTER vs. AT ?

Global type expiration for all documents ‘expireAfterSeconds’ (after 3 days)

• { "createdAt": 1 } , { expireAfterSeconds: 259200 } -global policy determines

Per document type expiration for each document at a Certain Clock Time

• {"expiresAt": 1 } , { expireAfterSeconds: 0 } -application sets FUTURE time

Combo - {"expiresAt": 1 } , { expireAfterSeconds: 86400 } –application and global

Things to Think About Before Creating a TTL Index

Create a new index with ensureIndex

Specify expireAfterSeconds option

use "createdAt or "expiresAt” leader

2 real Types : global & application set, then combination

‘expireAfterSeconds’ -global type – all docs expire X seconds after CREATED

How to Create TTL’s

Expire at Certain Clock Time -application set

- a date set in future by the application code

- expireAfterSeconds set to ZERO

combo type

- set an “expires_at” date in application code

- but apply a global type expiration policy with "expireAfterSeconds”

Messages documents set to expire 24 hours after creation

How to Create TTL’s –cont’d

TTL Mods

Sample seconds minutes hours days math

Total seconds / 60 seconds

TTLSeconds / (60 *1minute)

3600 seconds/60 seconds = 60 minutes/60 minutes = 1 hour

1200 seconds/60 seconds = 20 minutes

How to Modify your TTL Time

- Index must be created as a TTL index from the beginning

{"expiresAt" : 1 } alone != a TTL index

- Must use the collMod command to change seconds

Example – increase to a 3 day TTL

db.runCommand({"collMod" : "messages" , "index" : { "keyPattern" :

{"expiresAt" : 1 } , "expireAfterSeconds" : 259200 } })

Considerations and Gotchas

Aggressive

deletes

Space Usage

usePowerOf2Sizes

fragmentationTTL Isolation

drivers

Stale timestamps – downstream clients – supposed missing docs

No nullsNo _Id field

timezone

languagesNo compounds

-Very heavy writes/inserts and turn off TTL…@#$!

-Hard to catch up if app is very busy (7-15 Million inserts per day)

-Version Differences

-TTLMonitor thread runs every 60 seconds

-Background or Foreground Build implications

Look at # and size of documents

Check oplog size – ttl expire time and oplog time at least =

Warning: “The TTL index does not guarantee that expired data

will be deleted immediately.”

The duration of the removal operation depends on the workload of your

mongod instance. Expired data may exist beyond the 60 second period

between runs of the TTL monitor.

- http://docs.mongodb.org/manual/tutorial/expire-data/

More Considerations and Gotchas

How to Tell if Your TTL is Working

Do a little pre-checking – gather your information, get prepared,

know what to look for/at

How to Tell if Your TTL is Working –cont’d

-Confirm ttl index exists AND is correctly defined

-Define a variable to check with -ttlTime

-Set new Date for checking

How to Tell if Your TTL is Working –cont’d

db.messages.find({expires_at: {$lt:ISODate("2014-11-

23T03:11:00.013Z") }}).count()

MATH and Associated Queries :

*ISODate – is in MICROSECONDS; TTL is in SECONDS*

Other Associated Queries

If older than exact ttl exists, wait & re-query

get oldest documents in a collection using the _id as the determiner

Good Rule – +20% buffer in TTL seconds = > Math.round(1.2 * …)

THE MATH- Determine Starting TTL seconds

TTL Reducer snippet javascript example

- Step - # of seconds to shave off at a time, a time interval,

number of seconds to delete for

- Hop – number of iterations to take of your desired step

This is a Process. . . .

- Run in a screen!!!

- Create new ttl index with 20% padded seconds

- Start w/ beginning version of script - high padded ttl time

until deletes start – steps and a low interval

- Monitor impact with mongostat

- Minimize db lock % on Primary to less than 10 seconds

http://goo.gl/W7HAZw

OK, have Beginning and Ending Seconds– now what?

-Start with a larger step (interval 2000 secs) and fewer hops/iterations (30)

-High initial threshold (+20%) -might take time to 1st start hitting the deletes

ttl_reducer_begin.js – beginning

- Midway - do fewer steps(500 secs) and more hops (2930)

- Caution: deletes may negatively impact performance

ttl_reducer_mid.js – mid-way

- At end, optimize step (seconds to be deleted) and # hops

to reach desired permanent TTL ‘expireAfterSeconds’

ttl_reducer_end.js – end

-Single global setting –so no need to drop all TTL indexes

-Version 2.4.x and above: TTLMonitor thread is enabled by default

-Via a command-line parameter and/or server command.

To disable at startup:

setParameter ttlMonitorEnabled=false

#config file of the MONGOD's only

setParameter=ttlMonitorEnabled=false

## To turn it back on (just remove that line)

To disable at runtime:

on ALL replica set members - primary and secondary

on secondaries - -- rs.slaveOk()

db.adminCommand({setParameter:1, ttlMonitorEnabled:false})

To re-enable when ready:

db.adminCommand({setParameter:1, ttlMonitorEnabled:true})

Enabling / Disabling TTL Instance Wide

A Mass Pre-TTL Delete Loop

Useful MATH, Queries, and Loop for non-TTL based deletes

Run the below query and delete loop for a 30 minute period defined by x below

* Questions for me?

Blogs will be posted in ~2 weeks on

http://engineering.objectrocket.com/

Gist of the ttl_reducer.js scriptlets :

http://goo.gl/W7HAZw

MongoDB Inc. would like feedback if you are happy with the session. You can state your opinion at the kiosk on the way out the door. Please take a second to give us yours. Here's a sample of what the kiosks look like: http://www.happy-or-not.com/en_us/.

More questions? Ask-the-Experts area in the Atrium on first floor

Gist of additional queries and examples :

http://goo.gl/C1uYhb