Upload
kenny-gorman
View
245
Download
1
Embed Size (px)
Citation preview
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
1/21
MongoDB profiler
deep diveMongoAustin 2013
Kenny GormanFounder, ObjectRocket
@objectrocket @kennygorman
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
2/21
What is the profiler?
Captures metadata about what operations ran on thesystem
Saves data into capped collection
Designed for basic performance analysis
In the spirit of < instrument everything >
Very interesting advanced analysis possible
Aggregation Historical/Time-series analysis
Operational monitoring
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
3/21
Using the profiler
Turn it on, leave it.
Development cycle
Production debugging
Overall performance management
Find candidates, pull out query, use explain()
Rinse and Repeat
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
4/21
Simple example
example: https://gist.github.com/kgorman/4756589
https://gist.github.com/kgorman/4756589https://gist.github.com/kgorman/47565897/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
5/21
Annotated
example: https://gist.github.com/kgorman/4957922
https://gist.github.com/kgorman/4957922https://gist.github.com/kgorman/49579227/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
6/21
ts: timestamp of the operation
ns: namespace of the db and collection accessedop: the operation type
nreturned: the number of documents being returned
nscanned: the number of document scanned to return the result
moved: if the operation caused a move
millis: the total time in milliseconds that the operation tookkeyupdates:the number of indexes that required update
http://docs.mongodb.org/manual/reference/database-profiler/
Important Profiler Attributes
http://docs.mongodb.org/manual/reference/database-profiler/http://docs.mongodb.org/manual/reference/database-profiler/7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
7/21
fastMod
Good! Fastest possible update. In-place atomic operator ($inc,$set)
nretunred vs nscanned
If nscanned != nscanned, you may have opportunity to tune. Indexing.
key updates
Secondary indexes. Minimize them
~10% reduction in performance for each secondary index
moved Documents grow > padding factor
You can't fix it other than to pad yourself manually
db.collection.stats() shows padding
2.3.1+ usePowerOf2Sizes
https://jira.mongodb.org/browse/SERVER-1810 nreturned; high number of them
cardinality
Just pure I/O
What to look for
https://jira.mongodb.org/browse/SERVER-1810https://jira.mongodb.org/browse/SERVER-18107/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
8/21
IndexOnly
Very fast, use explain() instead of profiler
I/O
Page accesses
Page calls
Random I/O Data density/locality
https://jira.mongodb.org/browse/SERVER-3546 (track I/O)
Plans
Only explain() shows the full plan(s)
What doesn't it show?
https://jira.mongodb.org/browse/SERVER-3546https://jira.mongodb.org/browse/SERVER-35467/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
9/21
Profiler Analysis - FCS
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
10/21
Profiler Analysis - FCS
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
11/21
Profiler Analysis - Moved
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
12/21
Profiler Analysis - helpful queries
example: https://gist.github.com/kgorman/c5774670feb7436f4d69
https://gist.github.com/kgorman/c5774670feb7436f4d69https://gist.github.com/kgorman/c5774670feb7436f4d697/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
13/21
Going Deeper with Profiler Analytics
In prod environment profiler has lots of data
Prioritize tuning opportunities
Prioritize performance issues
Aggregation, summarization required
Enter Aggregation Framework
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
14/21
Aggregation Framework - Example
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
15/21
Aggregation Framework - Example
example: https://gist.github.com/kgorman/995a3aa5b35e92e5ab57
https://gist.github.com/kgorman/995a3aa5b35e92e5ab57https://gist.github.com/kgorman/995a3aa5b35e92e5ab577/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
16/21
Response time = service time + queue time
Each document in profile collection has a 'millis' attribute.
The only true measure of response time in MongoDB
Response time analysis
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
17/21
Response time analysis
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
18/21
When rolling new code
Customer activity patterns
Any time based patterns
Capacity planning
Why is this useful?
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
19/21
Going nuts with profiler and time-series
Turn on profiling
Pull delta response time data from db.system.profile in aggregate ina loop
Push to your favorite graphing/time-series program
https://github.com/kgorman/slum
https://github.com/kgorman/slum7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
20/21
Going nuts with profiler and time-series
7/29/2019 MongoDB Profiler Deep Dive; MongoDB Austin 2013
21/21
Contact
@kennygorman
@objectrocket
https://www.objectrocket.com