RockSolidSQL Server Presentation on Compression

DisclaimerWe recommend that you seek further professional advice before deciding on the suitability of any recommendations in this presentation for you. While all care is taken to ensure information is accurate, we do not make any guarantees about the suitability of advice and advice given may contain technical inaccuracies or topographical errors. The answers provided are our own opinions and may differ from advice provided by Microsoft and/or other SQL Server professionals. In no event shall we be liable for any special, incidental, indirect, economic or consequential damages or for loss of profit, revenue or data howsoever caused, regardless of whether we could foresee or was advised of the possibility or likelihood of such loss or damage.

Compression

What is it -– Objects that can be configured – Flexibility compression offers a DBA– How data compression works– Different types of compression

How do you determine -– If it’s worth using compression– Can I use compression– When to compress– How to use compression

Why would I use it -– Case studies – Advantages & disadvantages– Best practices

This presentation is for DBA’s considering using compression

What is compression?

Compression is a tool that the DBA can use to –• Make efficient use of disk space which can translate into a cost

saving.• Potentially, increase I/O performance as more data is stored in

memory.• Data compression can be configured for a number of specific

database objects.– A table stored as a heap or clustered index, A non-clustered

index, An indexed view or Partitioned tables and indexes

• You don’t need to compress all objects in a database. You can choose what objects and what type of compression to use.

• It’s flexible as you don’t need to compress all objects in a database– For example:

• Row compress some tables, page compress others and leave others uncompressed

• Row compress some table PARTITIONS, page compress others and leave others uncompressed

What is compression?

How does compression work?• Compression is handled entirely by the storage engine & most other

components within SQL Server are not aware compression exists• Data compressed on disk & in memory meaning more data on fewer

pages

Storage Engine

Relational Engine

Buffer cache

Files on disk

Data is compressed

Data is compressed

Storage engine compresses and decompresses data

Data is compressed

What is compression?

Different types of compression?• Row compression: Saves space by changing fixed data types

into a variable length format

• Page compression: Uses row compression then optimizes storage by removing repeating patterns of data & replacing it with abbreviated ref. Stores data once in CI structure.– Prefix– Dictionary– Page compression progresses in the following order:

DictionaryRow Prefix

Free SpaceSean

Char(20) Before

Sean

Char(20) After

Free space

What is compression?

What is Prefix & Dictionary?– Stores repeating data once

• Prefix:– Prefix: looks for repeating PREFIXED byte patterns on a

given column across all rows on each data page • Dictionary:

– Searches for repeating byte patterns ANYWHERE across all columns and all rows on each data page

– Stores Compression Information (CI) on the data same page immediately following page header.

Why have Row & Page compression options?– On average rebuilding an index using –

• Row compression takes approx 1.5x normal CPU usage• Page compression takes approx 2x - 5x normal CPU usage

What is compression?

How does the page store Prefix & Dictionary compression?

Uncompressed Page (Prefix) compression

Page Header

aaabb aaaab abcd

aaabcc bbbb abcd

Page Header

4b 4b

0bbbb

aaabcc

abcd

abcd

aaaacc abcd

What is compression?

Backup compression– Offers no options other than to turn it on or off

– If you are using data compression or encryption you may not notice any benefit

– Using backup compression has the following advantages:

• A typical database using data compression will compress some 60%-80% smaller

• We’ve found that using backup compression reduces disk space issues during maintenance windows as the backup is smaller.

• Quicker to back up a database using compression.• Quicker to restore a compressed backup resulting in a faster

recovery. • It can save you money when investing in DR solutions.

What is compression - Summary?

• We’ve talked about - – What compression is and how SQL Server manages it– The various objects that can be compressed & the flexibility this

offers– The various compression options

• Data – Row– Page

• Backup

How do I use compression?

Is it worth using compression in your environment?

• Estimate disk space savings– Use the sp_estimate_data_compression_savings. This stored proc

takes a sample of the data and compresses it in tempdb.

• Understand the workload & determine % of updates & scans for different objects 

• Understand the structure - Data types that compress are mainly numeric or fixed length character types.– Smallint, Int, Bigint, Decimal, Numeric, Bit, Smallmoney, Money,

Float, Real, Datetime, Char, nchar– NULL data values or repeating data will compress well – The type of database operation – datawarehouse

How do I use compression?

Is it worth using compression in my env continued ..?• Data types that don’t compress

– Smalldatetime, date, time, varchar(max), nvarchar(max), text, ntext, image, cursor, xml, user-defined data types (varbinary)

– Small number of repeating data, encrypted data– FILESTREAM unstructured data (video, images, documents etc.)

Can I compress in my environment?

How do I use compression?

When to use compression?• If you decide to compress a table or index make sure it is done in a

low activity or maintenance window• Online –v- Offline• Compression is transparent. It’s not obvious what's compressed so

make sure you maintain compression as the data/workload changes.

You can compress a table or index when it is initially created or after it has already been created using the –

– CREATE INDEX or CREATE TABLE – ALTER INDEX or ALTER TABLE – BACKUP DATABASE .. COMPRESSION

How do I use compression?

How do I reclaim space?After you’ve compressed an object you won’t see any reclaimed disk space until you -

• Option 1 – You shrink the file and reclaim the space. However, this can cause fragmentation.

• Option 2 – Create a new, compressed, empty table and copy the data to the new table

• Option 3 – You don’t reclaim the disk space and decide to keep the released free space in the file group for future data growth.

Demo

How do I use compression - Summary

We’ve talked about –– How to estimate compression savings– Why you should understand the workload and type of data your

database is storing– Which data types compress well– Which data type don’t compress very well– How to implement compression for new and existing objects– Which versions of SQL Server have compression options– Online-v-offline– Reclaiming space post compression– A couple of practical examples of implementing SQL Server

compression

Why use compression?

Case study implementing compression for JDE

Disk space issues resulting in SR’s - Want to save on

disk space & perhaps increase performance

ENABLE BACKUP COMPRESSION by default and compare before and

after stats

DATA COMPRESSION - Understand the workload!

Take a performance baseline without

compression.

ESTIMATE compression savings (row, page)Compress in test env first.

Testing successful compress smallest objects. Re-iterate

process. Take performance stats &

compare

If significant CPU pressure being experienced consider

using Row compression only

End result 570GB db down to 160GB - I/O response

times improved

Why use compression?

• Advantages– Can significantly reduce disk space – Can significantly reduce I/O– Can significantly improve performance as more data is stored in

memory– Works extremely well for databases where frequent updates of

data are not experienced. • Disadvantages

– Only certain data types will compress– If you have CPU issues compressing database objects may

intensify those issues

Why use compression?

Best Practices – Determine if you’ll benefit from compression

• Type of data (image, character etc.)• What is the database used for (Datawarehouse, OLTP)?• Consider the database workload. What tables are used for

scans and updates?• Use sp_estimate_data_compression_savings to estimate

compression savings

– Determine if you are suffering from high CPU usage before implementing compression.

– Compress during maintenance windows as indexes are rebuilt when initially compressed

– Test compression in test environment first to see what benefit you achieve

– Regularly check your environment for uncompressed objects

Why use compression?

Best Practices continued .. 

– Plan your compression • Compress one table, index or partition at a time rather than

concurrently• If you’ve decided to compress a number of objects start with

the smallest in the list. • Decide if you want to compress while Online or Offline. Offline

is faster and compresses quicker but the table is locked for the duration of the compression operation.

Summary

• Implement backup compression unless you have a good reason not to do so as the impact is usually low and space savings high.

• JDE example when compression was a success for us

• Compression can offer you substantial cost saving in terms of disk space usage and implementing DR solutions.

• Compression can increase database performance resulting in fewer user issues and fewer issues for the DBA

• Beyond the initial analysis using data compression does not take a great deal of time to maintain

References

• MS Books on-line• https://www.dbaouttask.com/blog• http://www.sqlservercentral.com• http

://blogs.msdn.com/b/sqlserverstorageengine/archive/tags/data+compression/

• http://bradmcgehee.com/

Contact

• Sean Young• youngs@redrock.net.au

www.rocksolidsql.com