SQLBits XIV - The Big Backup Theory

1 SQLBits XIV - The Big Backup Theory

I know, I know, who schedules a 75 minute session on

Backups!?!

In your shoes my reaction would be…

“Oh what fresh hell is this?”

The Dumpling Paradox S1 E7

The Big Backup Theory

Richard Douglas

The goal of this

presentation

The Cooper-Hofstadter Polarization - S1 E9

I do work for a vendor, there may be some

subliminal messages in this presentation

Don’t have a “Backup Strategy”

Agenda

• Why do we need backups?

• Backups available in SQL Server

• Understanding the Transaction Log

• Backup Strategy vs. Restore Strategy

• How to make your backups faster

• How to make your restores faster

Have a “Restore Strategy”

If we haven’t met yet…

• Richard Douglas

• Sales Engineer / SQL Evangelist

• SQL Server MCITPro

• SQL User Group Leader

• Blog: http://SQL.RichardDouglas.co.uk

• Twitter: @SQLRich

• Email: Richard.Douglas@Software.Dell.com

• Slides: http://www.Slideshare.net/SQLRich

Seriously, think about it.

A “Restore Strategy” makes so much sense!

This time Sheldon is wrong…Backups are Essential!

The Cooper-Hofstadter Polarization S1 E9

Why do we need backups?

• Protect valuable data

• Provides the potential to return to a “point in time”

• Confusion between Disaster Recovery and High Availability

• Backup misconceptionsImage backups are sufficient

Snapshots are sufficient

Differential backups are incremental

System databases only need to be backed up once a week

Only need to keep my last backup

Buffer pool usage

Recovery Models• Simple

• Full

• Bulk Logged

FileLog File

Database

Automated administration Manual Administration

Note: Recovery model will be defaulted to the one in the Model database

Backup Types• Full

• Differential

• Log

• File / Filegroup

• Tail of the log

• Database Snapshot

Transaction Log Internals

• The Transaction Log is segmented into pieces calledVirtual Log Files (VLF’s)

• There will always be at least one “active” VLF

• The number of VLF’s is determined by the size at creation / growth time.– Under 64MB 4VLF’s

– >=64 and > 1GB 8 VLF’s

– >= 1GB 16 VLF’s

• But wait, there’s more. VLF’s aren’t the smallest segment of the Transaction Log …

Active VLF

Inactive VLF

Log Blocks

• Each VLF will have a number of “Log Blocks” inside it

• These blocks vary in size:– Anywhere between 512bytes and 60KB

• A new block is created when:– The data inside the block hits the 60KB limit

– A commit or abort occurs within a transaction

• Each log block contains a parity bit of 64 or 128 which is flipped every time it is used.

• Inside each log block we have…

Explaining the transaction log is a bit like nesting dolls

Log Records

• Log records are identified by Log Sequence Numbers or LSN’s– VLF Sequence nbr : Log block nbr : Log record nbr

• Each data page contains the LSN of the last record that made changes to that page.

• Log records are written in the order they occur.i.e. SQL Server will not wait for a transaction to finish before writing away another concurrent transaction.

• Log records often store before and after images of changes.

• They contain a list of pages that this change has affected.

• Log records are stored in the buffer pool (log cache) until flushed to diskLook for WAITLOG wait type.

This will become clear when we

cover the restore process

Checkpoint

• Data pages are marked as “dirty” when they are changed.

• The checkpoint process writes all dirty pages to disk and marks them as cleared.

• Checkpoint is an optimisation process, it reduces the amount of writes to disk to keep data up to date.

Restore Strategy main considerations

• Time taken to restore dataaka Recovery Time Objective (RTO)

• Data loss acceptabilityaka Recovery Point Objective (RPO)

• Cost associated with system unavailable

Backup Strategy main considerations

• Time taken to perform a backup

• Cost associated with longer maintenance window

• Cost associated with storing backup files

Backup Strategy vs Restore Strategy

Showing you why you need to have a “Restore

Strategy”

Full backup00

Differential00

Transaction Log Backup00

Figured out it’s safer to have a

“Restore Strategy” yet?

Optimizing backups

Server Architecture

Database Architecture

Backup Options

Server Architecture

What computer do you have? And please don’t say “a white one”.

The Peanut Reaction – S1 E16

• IO Subsystem– RAID Levels

– Disk Partition Alignment

– Dedicated drives

• Network cards– Add more NICs

– Use different subnets

– Use different switches

Optimizing backups – Server Architecture

The DBA team start negotiations with the Infrastructure team

http://www.flickr.com/photos/jakesutton/859193891/in/photostream

Optimizing backups – Database Architecture• Table design

– See Natural Born Killers webcast November 2012 (http://dell.to/15RwEtz)

– Blogged at http://bit.ly/GHuUrH

– Space that a table uses is not completely released after you use a DELETE statement to delete data from the table in SQL Server: http://support.microsoft.com/kb/913399

• Index Design – See indexing webcasts October 2012 (http://dell.to/19urm8s)

• File placement

• Compression

End of subliminal messages

Actually, I wouldn’t mind

a caramel macchiato

Optimizing Backups

• Compression

• BufferCount

• MaxTransferSize

• BlockSize

• Multiple backup devices

Other Backup Options of note

• Init / NoInit

• Format / NoFormat

• Copy Only

• Checksum

• Continue After Error

Backup Options

For a full list of all the backup options refer to Books Online (BOL)http://msdn.microsoft.com/en-us/library/ms186865.aspx

Backup Options Continued

• Memory used is accessed from outside of the bufferpool

• You can calculate memory usage:Buffercount * MaxTransferSizeSetting these values too high may cause out of memory conditions

Buffercount calculations for Full / Diff backups:

• 2000– (NumberofBackupDevices * GetSuggestedIoDepth)

+ NumberofBackupDevices + (DatabaseDeviceCount)

• 2005 and above– (NumberofBackupDevices * GetSuggestedIoDepth)

+ NumberofBackupDevices + (2*DatabaseDeviceCount)

Calculating BufferCount

• GetSuggestedIODepth– Disk = 3

– Tape = 3

– VDI = 1 default and 4 max based on device spanning needs http://bit.ly/1i69GUG

• DatabaseDeviceCount = The number distinct drives the database files reside on.

• I think the formula for 2008+ is wrong. Waiting for CSS to return my email.The following seems to work:

(NumberofBackupDevices * GetSuggestedIoDepth [4]) + NumberofBackupDevices + (2*DatabaseDeviceCount)

51 The Big Backup Theory

Things that break the log chain

Restore Problems• Broken Transaction Log Chain

– Clearing the transaction log (using WITH TRUNCATE ONLY or WITH NO LOG)

– Changing the database to SIMPLE recovery model (checkpoint will clear log))

– Reverting from a database snapshot

– Deleting a log file in the chain

– Backups to NUL devices

Backups are like

Schrodinger’s cat

Restore Myths• Downgrading a backup

• No need to test backups

• Restore time will be the same as the backup time

• Table restores

The Restore ProcessA restore operation can have up to four phases:

1. File creation and initialization

2. Data and/or log copy

3. REDO phase of recovery

4. UNDO phase of recovery

The Restore Process File creation and initialization

• In the beginning there was file creation.

• Then there was the zeroing of the files.

• If the database already exists overwrite using the “WITH REPLACE” clause,do not drop and restore.

• Use “Instant File Initialization” (IFI) if possible.Remember the log file still needs to be zeroed out.

• There is a security risk.

• TF 3004 will provide more information about IFI

• TF 1806 will disable IFI

The Restore ProcessData and/or log copy

• In this stage SQL Server:

– Reads the data from the backup file(s)

– Writes the data to the database files

• Having multiple data files and backup files can improve I/O throughput.

• Using native or third party compression reduces the amount of data to be read (and potentially copied across the network).

Compression vs DeDuplication

• Whilst deduplication does reduce storage costs it will be rehydrated before copying back across the network and so will not aid the speed of a restore.

The Restore ProcessREDO phase of recovery (1)

• The database being recovered has it’s boot page (page 9) checked.

• This page will provide the details of the most recent checkpoint.

• Checkpoint log records are examined to see if there is a list of uncommitted transactions

• Recovery will start at the oldest uncommitted transaction for the most recent checkpoint

• Each log record is tied to an LSN and contains page numbers of records it has altered

• Each data page’s page header contains the last LSN which made a change to it

• The redo phase compares the two LSN’s

• The redo phase uses something akin to the following formula:

IF PageHeaderLSN < LogRecordLSN

Replay Log Record –Record has not been updated

DoNothing –Record has this change and possibly some others

Finding the end of the log

There are two ways:

1. If the log has not been wrapped then it checks for the occurrence of 512 byte block full of zeros.

2. If it has been wrapped then it reads through all the active VLF’s in the order of their sequence number and reads through each log block until it finds one with the wrong flipped parity bit

The Restore ProcessUNDO phase of recovery

• The redo phase keeps track of uncommitted transactions. If there are uncommitted transactions during recovery then the undo phase kicks in

• As mentioned in the redo section the data page page header contains an LSN. This is compared to the Log Record LSN

IF PageHeaderLSN >= LogRecordLSN

Undo Log Record –Record requires rollback

Create a “compensation” log record

Update PageHeader LSN with compensation log record LSN

Further ways to optimize restores• Using “Instant File Initialization”

• Using the correct restore strategy

• Server Architecture

• Database Architecture

• Backup compression

• Restore options– Buffercount

– MaxTransferSize

65 Global MarketingThe Big Backup Theory

Any questions?

Thank you for attendingRichard DouglasRichard.Douglas@Software.Dell.com@SQLRich

SQLBits XIV - The Big Backup Theory

Technology

EVENT DRIVEN CUBE MONITORING. David Prime & David Elliott SQLBits 6

ADL/U-SQL Introduction (SQLBits 2016)

Backup is not backup. Restore is Backup

Cartas Celestes XIV Celestial Maps XIV by Almeida Prado

SQLBits X SQL Server 2012 Spatial

Europe’s Premier Community SQL Server Conference # SQLBITS

Symantec Backup Exec 2014 Icon List - Renjith Menon Backup Exec... · Dependent Backup Set Dependent Backup Set Backup Set Expiration Backup Set Expiration Backup Sets Synthetic backup

XIV - Microsoft

ChineseGrammar XIV

U-SQL Does SQL (SQLBits 2016)

U-SQL Partitioned Data and Tables (SQLBits 2016)

NHibernate - SQLBits IV

SQLBits X Scaling out with SQL Azure Federations

SQLBits X SQL Server 2012 Spatial Indexing

Coneia Xiv

Designing an SSIS Framework Andy Leonard SQLBits 9 1 Oct 2011

10 Things Not To Do With SQL SQLBits 7. Some things you shouldn’t do

KoprowskiT - SQLBITS X - 2am a disaster just began

XIV REUNION CONSULTATIVE XIV … · rio dejaneiro 1987 5 - 16 october antarctic treaty xiv consultative meeting ~orobop06ahtapkt~ke xiv kohcvnbtar",shoecobeiuah",e ant/xiv/inf 18

SQLBits 8, 9 th April 2011, Brighton Vincent Rainardi vrainardi@gmail.com Blog: dwbi1.wordpress.com