5 Ways To Reduce Database Infrastructure Costs

Baron Schwartz, VividCortexJuly 9, 2015 Webinar

Intro & Logistics

• Contact me at baron@vividcortex.com or @xaprb

• Tweet questions to #vividcortex at any time

• Slides and video will be posted

Agenda

• Are your database costs too high? How would you know?

• What drives high costs

• Key opportunities to save money and improve efficiency

• Five steps to maximize database ROI

Are Your Databases Too Expensive?

• You may be spending too much on databases if…

• You’re not sure if you’re over-provisioned

• You’re brute-forcing for performance/availability

• You’re hammering the databases with useless workload

• You’re buying fixed resources for elastic workloads

How To Know

• You already know

• You measure and find out(what should you measure?)

What Drives Costs?

1. Servers2. Power, space, and cooling3. DBAs and sysadmins4. Developer productivity5. IT’s ability to deliver

Five Opportunities

• Query efficiency

• Reducing diversity

• Server consolidation

• Right-sizing

• Cloud computing

Query Efficiency Inefficient queries waste resources and cause slow user experiences.

Key Causes of Query Inefficiency

• Using no indexes or poor indexing.

• N+1 query pattern instead of a join or bulk query.

• Wasteful use of prepared statements.

• Long-running transactions.

How To Measure It

• Measure index usage for queries

• Examine per-index statistics if available

• In MySQL, use protocol bits NO_GOOD_INDEX etc

• Examine EXPLAIN plans, beginning with the most time-consuming queries

• What about the so-called “index ratio” of rows read versus rows returned? It doesn’t work — it falsely highlights GROUP BY and other queries

Queries Missing Indexes

Wasteful Prepared Statements

Further Reading

• https://vividcortex.com/docs/top-queries/

• https://vividcortex.com/blog/2014/11/19/analyzing-prepared-statement-performance-with-vividcortex/

Database Diversity

The Cost Explosion of Diversity

• “Choose the right tool for the job” can have exploding cost consequences

• There’s a tradeoff between efficiency for particular use cases, and the cost of complexity

• Training, knowledge sharing, bottlenecks around specialized people, ability to troubleshoot in an outage, interconnections between moving parts…

• More on complexity cost: http://firstround.com/review/The-one-cost-engineers-and-product-managers-dont-consider/

Consolidation and Right-Sizing

Key Indicators You Can Consolidate

• Imbalanced workload across many similar servers

• Under-utilized server instances (CPU, memory, etc)

Imbalanced Workload

Capacity Analysis

• Check capacity of key resources: CPU, memory, disk IO, disk storage space, network IO

• It may turn out that servers are under-utilized in one dimension but not another (balancing different types of tenants in multitenant architecture for example)

• Don’t forget to check utilization of fixed-size resource pools such as InnoDB’s buffer pool (in MySQL)

A different picture!

How Much Can You Consolidate?

• This is a tricky question that depends on the technology

• Ideally you can utilize CPU, memory, IO and network in a balanced way

• But what if the database can’t or doesn’t drive utilization of one resource? e.g. older MySQL couldn’t drive much IO on fast storage devices

• Systems almost always scale nonlinearly, and bin-packing smaller systems on larger hardware can drive cost efficiency

Right-Sizing

• There are ranges of price-to-performance for hardware purchases

• Example: which size DIMMs are most cost efficient right now? How many can fit in a chassis? What’s the cost of that form factor?

• These tradeoffs undergo continual change, and you need to stay current to get the most purchasing power

Cloud Computing a.k.a. “make it someone else’s problem” ?

When is Cloud Computing a Solution?

• When load is highly dynamic

• When you want to make short-term bets

• When capacity needs grow quickly

Key Ways to Waste Money In The Cloud

• Buying on-demand instead of reserved instances

• Resource imbalances (e.g. buying lots of CPU for an IO-bound database)

• Buying more/bigger instances instead of driving existing ones harder

How To Drive Servers Harder

• Characterize workload

• Examine top resource consumers

• Optimize top hogs

1. Eliminate queries

2. Reduce queries

3. Improve queries

• Repeat until done

Top Queries IsYour Go-To Solution

All of these processes in a single visual tool

How Much Can You Improve?

• You can’t improve what you can’t measure

• Fine-grained, comprehensive measurement of queries (not global status counters), with drilldown, is the starting point

• Some of VividCortex’s customers have been able to drive their servers 40-50% harder with confidence

Final Note

It’s really not a server optimization problem. The true problem (and opportunity) is optimizing IT teams, processes, and communications.

Questions

• Slides will be posted• Recording will be on YouTube• August’s webinar awaits you• I’m baron@vividcortex.com• Thankyouverymuch

Helpful Resources

Upcoming Webinar

Breaking Databases - Keeping your Ruby on Rails ORM under control

August 4, 2015 - Owen Zanzal

Credits• Bridge https://www.flickr.com/photos/photommo/19060494190/

• Broken computer https://www.flickr.com/photos/mandyxclear/3461234232/

• Goals https://www.flickr.com/photos/angietorres/4564135455/

• Sports car https://www.flickr.com/photos/bongosphotographie/7038281073/

• Money sprouting https://www.flickr.com/photos/pictures-of-money/17123254699/

• Turbofan https://www.flickr.com/photos/jimmy241st/6954770596/

• Water drops https://www.flickr.com/photos/yogendra174/5061486537/

• Dachshunds https://www.flickr.com/photos/78428166@N00/4415190041/

• Storm Trooper https://www.flickr.com/photos/kalexanderson/6266452817/

• Clouds https://www.flickr.com/photos/damienpollet/5817618029/

• Race car https://www.flickr.com/photos/i-am-mclovin/9239960203/