Upload
richard-douglas
View
162
Download
0
Embed Size (px)
Citation preview
1 SQLBits XIV - The Big Backup Theory
I know, I know, who schedules a 75 minute session on
Backups!?!
In your shoes my reaction would be…
2 SQLBits XIV - The Big Backup Theory
“Oh what fresh hell is this?”
The Dumpling Paradox S1 E7
The Big Backup Theory
Richard Douglas
6 SQLBits XIV - The Big Backup Theory
The goal of this
presentation
The Cooper-Hofstadter Polarization - S1 E9
7 SQLBits XIV - The Big Backup Theory
I do work for a vendor, there may be some
subliminal messages in this presentation
8 SQLBits XIV - The Big Backup Theory
Don’t have a “Backup Strategy”
9 SQLBits XIV - The Big Backup Theory
Agenda
• Why do we need backups?
• Backups available in SQL Server
• Understanding the Transaction Log
• Backup Strategy vs. Restore Strategy
• How to make your backups faster
• How to make your restores faster
10 SQLBits XIV - The Big Backup Theory
Have a “Restore Strategy”
11 SQLBits XIV - The Big Backup Theory
If we haven’t met yet…
• Richard Douglas
• Sales Engineer / SQL Evangelist
• SQL Server MCITPro
• SQL User Group Leader
• Blog: http://SQL.RichardDouglas.co.uk
• Twitter: @SQLRich
• Email: [email protected]
• Slides: http://www.Slideshare.net/SQLRich
12 SQLBits XIV - The Big Backup Theory
Seriously, think about it.
A “Restore Strategy” makes so much sense!
13 SQLBits XIV - The Big Backup Theory
This time Sheldon is wrong…Backups are Essential!
The Cooper-Hofstadter Polarization S1 E9
14 SQLBits XIV - The Big Backup Theory
Why do we need backups?
• Protect valuable data
• Provides the potential to return to a “point in time”
• Confusion between Disaster Recovery and High Availability
• Backup misconceptionsImage backups are sufficient
Snapshots are sufficient
Differential backups are incremental
System databases only need to be backed up once a week
Only need to keep my last backup
Buffer pool usage
15 SQLBits XIV - The Big Backup Theory
Have a “Restore Strategy”
16 SQLBits XIV - The Big Backup Theory
Recovery Models• Simple
• Full
• Bulk Logged
Data
FileLog File
SQL
Database
Automated administration Manual Administration
Note: Recovery model will be defaulted to the one in the Model database
17 SQLBits XIV - The Big Backup Theory
Backup Types• Full
• Differential
• Log
• File / Filegroup
• Tail of the log
• Database Snapshot
18 SQLBits XIV - The Big Backup Theory
Have a “Restore Strategy”
20 SQLBits XIV - The Big Backup Theory
Transaction Log Internals
• The Transaction Log is segmented into pieces calledVirtual Log Files (VLF’s)
• There will always be at least one “active” VLF
• The number of VLF’s is determined by the size at creation / growth time.– Under 64MB 4VLF’s
– >=64 and > 1GB 8 VLF’s
– >= 1GB 16 VLF’s
• But wait, there’s more. VLF’s aren’t the smallest segment of the Transaction Log …
Active VLF
Inactive VLF
Inactive VLF
Inactive VLF
Inactive VLF
21 SQLBits XIV - The Big Backup Theory
Log Blocks
• Each VLF will have a number of “Log Blocks” inside it
• These blocks vary in size:– Anywhere between 512bytes and 60KB
• A new block is created when:– The data inside the block hits the 60KB limit
– A commit or abort occurs within a transaction
• Each log block contains a parity bit of 64 or 128 which is flipped every time it is used.
• Inside each log block we have…
22 SQLBits XIV - The Big Backup Theory
Explaining the transaction log is a bit like nesting dolls
23 SQLBits XIV - The Big Backup Theory
Log Records
• Log records are identified by Log Sequence Numbers or LSN’s– VLF Sequence nbr : Log block nbr : Log record nbr
• Each data page contains the LSN of the last record that made changes to that page.
• Log records are written in the order they occur.i.e. SQL Server will not wait for a transaction to finish before writing away another concurrent transaction.
• Log records often store before and after images of changes.
• They contain a list of pages that this change has affected.
• Log records are stored in the buffer pool (log cache) until flushed to diskLook for WAITLOG wait type.
This will become clear when we
cover the restore process
24 SQLBits XIV - The Big Backup Theory
Checkpoint
• Data pages are marked as “dirty” when they are changed.
• The checkpoint process writes all dirty pages to disk and marks them as cleared.
• Checkpoint is an optimisation process, it reduces the amount of writes to disk to keep data up to date.
26 SQLBits XIV - The Big Backup Theory
Have a “Restore Strategy”
27 SQLBits XIV - The Big Backup Theory
Restore Strategy main considerations
• Time taken to restore dataaka Recovery Time Objective (RTO)
• Data loss acceptabilityaka Recovery Point Objective (RPO)
• Cost associated with system unavailable
Backup Strategy main considerations
• Time taken to perform a backup
• Cost associated with longer maintenance window
• Cost associated with storing backup files
Backup Strategy vs Restore Strategy
28 SQLBits XIV - The Big Backup Theory
Showing you why you need to have a “Restore
Strategy”
29 SQLBits XIV - The Big Backup Theory
Full backup00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
M
T
W
T
F
S
S
30 SQLBits XIV - The Big Backup Theory
Full backup00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
M
T
W
T
F
S
S
31 SQLBits XIV - The Big Backup Theory
Full backup00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
M
T
W
T
F
S
S
32 SQLBits XIV - The Big Backup Theory
Differential00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
M
T
W
T
F
S
S
33 SQLBits XIV - The Big Backup Theory
Differential00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
M
T
W
T
F
S
S
34 SQLBits XIV - The Big Backup Theory
Differential00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
M
T
W
T
F
S
S
35 SQLBits XIV - The Big Backup Theory
Transaction Log Backup00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
M
T
W
T
F
S
S
36 SQLBits XIV - The Big Backup Theory
Transaction Log Backup00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
M
T
W
T
F
S
S
37 SQLBits XIV - The Big Backup Theory
Figured out it’s safer to have a
“Restore Strategy” yet?
38 SQLBits XIV - The Big Backup Theory
Optimizing backups
Server Architecture
Database Architecture
Backup Options
39 SQLBits XIV - The Big Backup Theory
Have a “Restore Strategy”
40 SQLBits XIV - The Big Backup Theory
Server Architecture
41 SQLBits XIV - The Big Backup Theory
What computer do you have? And please don’t say “a white one”.
The Peanut Reaction – S1 E16
42 SQLBits XIV - The Big Backup Theory
• IO Subsystem– RAID Levels
– Disk Partition Alignment
– Dedicated drives
• Network cards– Add more NICs
– Use different subnets
– Use different switches
Optimizing backups – Server Architecture
The DBA team start negotiations with the Infrastructure team
http://www.flickr.com/photos/jakesutton/859193891/in/photostream
43 SQLBits XIV - The Big Backup Theory
Have a “Restore Strategy”
44 SQLBits XIV - The Big Backup Theory
Optimizing backups – Database Architecture• Table design
– See Natural Born Killers webcast November 2012 (http://dell.to/15RwEtz)
– Blogged at http://bit.ly/GHuUrH
– Space that a table uses is not completely released after you use a DELETE statement to delete data from the table in SQL Server: http://support.microsoft.com/kb/913399
• Index Design – See indexing webcasts October 2012 (http://dell.to/19urm8s)
• File placement
• Compression
45 SQLBits XIV - The Big Backup Theory
End of subliminal messages
46 SQLBits XIV - The Big Backup Theory
Actually, I wouldn’t mind
a caramel macchiato
47 SQLBits XIV - The Big Backup Theory
Optimizing Backups
• Compression
• BufferCount
• MaxTransferSize
• BlockSize
• Multiple backup devices
Other Backup Options of note
• Init / NoInit
• Format / NoFormat
• Copy Only
• Checksum
• Continue After Error
Backup Options
For a full list of all the backup options refer to Books Online (BOL)http://msdn.microsoft.com/en-us/library/ms186865.aspx
48 SQLBits XIV - The Big Backup Theory
Backup Options Continued
• Memory used is accessed from outside of the bufferpool
• You can calculate memory usage:Buffercount * MaxTransferSizeSetting these values too high may cause out of memory conditions
Buffercount calculations for Full / Diff backups:
• 2000– (NumberofBackupDevices * GetSuggestedIoDepth)
+ NumberofBackupDevices + (DatabaseDeviceCount)
• 2005 and above– (NumberofBackupDevices * GetSuggestedIoDepth)
+ NumberofBackupDevices + (2*DatabaseDeviceCount)
49 SQLBits XIV - The Big Backup Theory
Calculating BufferCount
• GetSuggestedIODepth– Disk = 3
– Tape = 3
– VDI = 1 default and 4 max based on device spanning needs http://bit.ly/1i69GUG
• DatabaseDeviceCount = The number distinct drives the database files reside on.
• I think the formula for 2008+ is wrong. Waiting for CSS to return my email.The following seems to work:
(NumberofBackupDevices * GetSuggestedIoDepth [4]) + NumberofBackupDevices + (2*DatabaseDeviceCount)
50 SQLBits XIV - The Big Backup Theory
Demo
51 The Big Backup Theory
Things that break the log chain
52 SQLBits XIV - The Big Backup Theory
Restore Problems• Broken Transaction Log Chain
– Clearing the transaction log (using WITH TRUNCATE ONLY or WITH NO LOG)
– Changing the database to SIMPLE recovery model (checkpoint will clear log))
– Reverting from a database snapshot
– Deleting a log file in the chain
– Backups to NUL devices
53 SQLBits XIV - The Big Backup Theory
Backups are like
Schrodinger’s cat
54 SQLBits XIV - The Big Backup Theory
Restore Myths• Downgrading a backup
• No need to test backups
• Restore time will be the same as the backup time
• Table restores
55 SQLBits XIV - The Big Backup Theory
The Restore ProcessA restore operation can have up to four phases:
1. File creation and initialization
2. Data and/or log copy
3. REDO phase of recovery
4. UNDO phase of recovery
56 SQLBits XIV - The Big Backup Theory
The Restore Process File creation and initialization
• In the beginning there was file creation.
• Then there was the zeroing of the files.
• If the database already exists overwrite using the “WITH REPLACE” clause,do not drop and restore.
• Use “Instant File Initialization” (IFI) if possible.Remember the log file still needs to be zeroed out.
• There is a security risk.
• TF 3004 will provide more information about IFI
• TF 1806 will disable IFI
57 SQLBits XIV - The Big Backup Theory
The Restore ProcessData and/or log copy
• In this stage SQL Server:
– Reads the data from the backup file(s)
– Writes the data to the database files
• Having multiple data files and backup files can improve I/O throughput.
• Using native or third party compression reduces the amount of data to be read (and potentially copied across the network).
Compression vs DeDuplication
• Whilst deduplication does reduce storage costs it will be rehydrated before copying back across the network and so will not aid the speed of a restore.
58 SQLBits XIV - The Big Backup Theory
The Restore ProcessREDO phase of recovery (1)
• The database being recovered has it’s boot page (page 9) checked.
• This page will provide the details of the most recent checkpoint.
• Checkpoint log records are examined to see if there is a list of uncommitted transactions
• Recovery will start at the oldest uncommitted transaction for the most recent checkpoint
59 SQLBits XIV - The Big Backup Theory
The Restore ProcessREDO phase of recovery (2)
• Each log record is tied to an LSN and contains page numbers of records it has altered
• Each data page’s page header contains the last LSN which made a change to it
• The redo phase compares the two LSN’s
• The redo phase uses something akin to the following formula:
IF PageHeaderLSN < LogRecordLSN
BEGIN
Replay Log Record –Record has not been updated
END
ELSE
DoNothing –Record has this change and possibly some others
60 SQLBits XIV - The Big Backup Theory
The Restore ProcessREDO phase of recovery (3)
Finding the end of the log
There are two ways:
1. If the log has not been wrapped then it checks for the occurrence of 512 byte block full of zeros.
2. If it has been wrapped then it reads through all the active VLF’s in the order of their sequence number and reads through each log block until it finds one with the wrong flipped parity bit
61 SQLBits XIV - The Big Backup Theory
The Restore ProcessUNDO phase of recovery
• The redo phase keeps track of uncommitted transactions. If there are uncommitted transactions during recovery then the undo phase kicks in
• As mentioned in the redo section the data page page header contains an LSN. This is compared to the Log Record LSN
IF PageHeaderLSN >= LogRecordLSN
BEGIN
Undo Log Record –Record requires rollback
Create a “compensation” log record
Update PageHeader LSN with compensation log record LSN
END
63 SQLBits XIV - The Big Backup Theory
Further ways to optimize restores• Using “Instant File Initialization”
• Using the correct restore strategy
• Server Architecture
• Database Architecture
• Backup compression
• Restore options– Buffercount
– MaxTransferSize
65 Global MarketingThe Big Backup Theory
Any questions?
Thank you for attendingRichard [email protected]@SQLRich