View
64
Download
0
Category
Preview:
Citation preview
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
CEO Fortified Data
Working with SQL Server for 20yrs
Author, speaker, and consultant focusing
on Performance, Scalability, and Availability
E-mail: bdebow@fortifieddb.com
Twitter: @BBQSQL
Web/Blog: http://www. fortifieddb.com
ABOUT ME
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
DEFINING THE PEAK LOAD
We need to scale, but what does that mean?
Do you know the applications and servers involved?
What are the business transactions?
# of sales ordered in 1hour
1 sales order = 50 or 10,000 database calls?
What are the service level agreements (SLA)?
When is the peak load on the system?
Daily, Monthly, Seasonal
Before any tuning takes place, answer these
questions.
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
APPLICATION ASSESSMENT Configuration
Operation System
SQL Server
What is installed and being used?
OLTP versus OLAP
Supporting Infrastructure
Disk Subsystem
Server Hardware
Network Topology
Need to know current state!!
Leverage existing CMDB’s
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
SLAS AND TRANSACTIONS
Number of transactions at the peak hour?
What % of transactions exceed the SLA?
What are the core transactions?
Most important to the business!
What is the busiest day of the week? Hour?
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
“
”
I THINK THE SYSTEM CAN HANDLE THE
LOAD, WE PROCESS 8 MILLION
RETURNS?
6
Yes, BUT the avg. response
time is 5 seconds and not < 3.
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
BASELINE THE SYSTEM
Create a Performance Profile
Collect Performance Data
Leverage DMVs
Check SQL uptime before collecting
Know what the configuration is
DBs size and properties
Always look at current system activity!!
Get to know the typical activity
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
PERFORMANCE DATA
Disk
Reads/sec
0
20
40
60
80
100
120
140
160
Apr
22
(17)
Apr
29
(18)
May
06
(19)
May
13
(20)
May
20
(21)
May
27
(22)
Jun
03
(23)
Jun
10
(24)
Jun
17
(25)
Jun
24
(26)
Jul 01
(27)
Jul 08
(28)
Jul 15
(29)
Host Latency - Node A
Average (ms)
Batch
Request/sec
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
APPLICATION LATENCY August
The average request took 414 ms
5% of requests took over 1.4 seconds
1% of requests took over 4.1 seconds
Weekly request peak of 7,900,000
September
The average request took 512.7 ms
5% of requests took over 1.9 seconds
1% of requests took over 6.2 seconds
Weekly request peak of 9,400,000
October
The average request took 499 ms
5% of requests took over 1.8 seconds
1% of requests took over 5.4 seconds
Weekly request peak of 9,500,000
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
THE ANALYSIS
Start at the OS and end at the code
Changing code increases the risk and timeline
Look for misconfigurations
OS, SQL, NICs, SAN, DBs, drivers and firmware
Clean up the room first
Align configurations to “Best Practices”
Changes system behavior
Identify the real issues
Collect more data
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
THE ANALYSIS - PERFMON Time Slices
Sample intervals
Seconds versus minutes – GOAL?
**Correlation of object counters
Thresholds
Every system is different
Baseline to define your ranges
Start high and drill down
Chart versus report
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
ARCHITECTURES Scale up
Scale out
Reporting Tier with Availability Groups
Re-direct read-only requests with Availability Groups
Sharding
The CLOUD
Caching Servers
Active/Active
Replication – Bi-directional Transactional
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
ACTION PLAN 2 Scenarios
More juice out of the lemon
Have money, will spend
Affects recommendations
Let’s squeeze more juice out of the lemon
Identify system constraints
What is limiting factor(s) for scaling?
How do the findings play into the peak goals?
How do you measure the system?
Stress test if possible
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
PROVEN APPROACH
1. Infrastructure and OS configuration
2. SQL Server configuration
3. Database configuration
4. Scheduling and non-essential processes
5. Reduce contention
6. Performance tuning
1. Indexes
2. Statements
3. Process re-design
7. Re-assess performance
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
SUPPORTING THE SYSTEM Monitoring
Proactive and reactive
Fast identification and resolution
Ex. Blocking chains will impact the SLA’s
Communication
Contact numbers
On-call schedules
Perfmon Maintenance before peak load
Turn off during peak window
www.devconnections.com
SURVIVING YOUR PEAK DATABASE LOAD
SUMMARY Understand system architecture
Capture configuration and performance
data to determine the limiting factors
Iterative approach helps reduce risk and
gauge impact
Plan for 20% above peak
Monitoring is key
Fast identification and resolution
Recommended