CHANGE DATA CAPTURE& CHANGE TRACKING

DEEP DIVE

W. Kevin HazzardLinchpinPeople.com

Group Principal

CHANGE DATA CAPTURE

• Turn it onEXECUTE sys.sp_cdc_enable_db;

• Which generates for the whole database• A capture job• A cleanup job

• Then include at least one table withEXECUTE sys.sp_cdc_enable_table …;

• Which creates a capture instance containing• A change table• An all-changes function• A net-changes function (optional)

transactions

warehouse

sources

changes

capture job

archivesectorETL

analysis

datamart

ETL

CAPTURE INSTANCE

• Contains• The capture table• cdc.fn_cdc_get_all_changes_<instance name> function• cdc.fn_cdc_get_net_changes_<instance name> function

(optional)• Maximum two per source table

SYS.SP_CDC_ENABLE_TABLE

• @source_schema• @source_name• @supports_net_changes• @role_name• @filegroup_name

• @index_name• @captured_column_list• @allow_partition_switch• @capture_instance

DEMONSTRATION

Enabling Change Data Capture (CDC)Creating a CDC Capture Instance

Querying CDC Tables as Things Change

SQL SERVER AGENT JOBS

• cdc.Capture_capture• Starts with SQL Agent• RAISERROR(22801, 10, -1);• EXEC sys.sp_MScdc_capture_job;

• Just a wrapper for sys.sp_cdc_scan• cdc.Capture_cleanup

• Runs at 02:00 daily• EXEC sys.sp_MScdc_cleanup_job;

CDC DATA RETENTION

• Based on LSN Validity Intervals• Database• Capture Instance

• The cleanup job deletes CDC data acording to retention policySELECT * FROM msdb.dbo.cdc_jobs WHERE job_type = N'cleanup';

• You can start the cleanup job manuallyEXEC sys.sp_cdc_start_job @job_type = N'cleanup';

HANDLING SCHEMA CHANGES

• Deleted and new columns are handled well• Modified columns require specific steps:

• Stop the capture and cleanup jobs• Change the schema as necessary• Generate new capture instances for all modified tables• Process all data in the old and new capture instances• Manually run the cleanup job• Delete the old capture instance• Turn the capture and cleanup jobs back on

CHANGE TRACKING

• Turn it on for one databaseALTER DATABASE <DBNAME> SET CHANGE_TRACKING = ON;

• Then enable a tableALTER TABLE <TBLNAME> ENABLE CHANGE_TRACKING;

transactions warehouse

sources archivesectorETL

sync

CHANGE TRACKING TABLES & FUNCTIONS

• sys.change_tracking_databases• sys.change_tracking_tables• CHANGETABLE(CHANGES)• CHANGETABLE(VERSION)• CHANGE_TRACKING_CURRENT_VERSION()• CHANGE_TRACKING_MIN_VALID_VERSION()• CHANGE_TRACKING_IS_COLUMN_IN_MASK()• WITH CHANGE_TRACKING_CONTEXT

DEMONSTRATION

Enabling Change Tracking (CT) for a DatabaseEnabling CT on a Table

Querying CT Tables and Functions as Things Change

HANDLING SCHEMA CHANGES

• No modifications to the primary key are allowed including related indexes• Dropping columns is OK but they may still appear in the change data• When adding columns, changes are tracked but the metadata change is not

reported• Switching partitions will fail on change tracked changes• Data type changes are not tracked

WHICH ONE IS RIGHT FOR ME?

Change Data Capture• Works in Enterprise Edition only• Stores every discrete change• Storage intensive• Good for auditing• Requires SQL Server agent• No special serialization required

Change Tracking• Works in all versions of SQL Server• Returns differences from current• Storage light• Good for device synchronization• No job agent required• Operates best with snapshot isolation