Change Data Capture & Change Tracking Deep Dive W. Kevin Hazzard LinchpinPeople.com Group Principal

Change Data Capture analysis transactions warehouse sources data mart Turn it on EXECUTE sys.sp_cdc_enable_db; Which generates for the whole database A capture job A cleanup job Then include at least one table with EXECUTE sys.sp_cdc_enable_table …; Which creates a capture instance containing A change table An all-changes function A net-changes function (optional) transactions sources data mart ETL capture job warehouse ETL archive sector changes

Capture Instance Contains Maximum two per source table The capture table cdc.fn_cdc_get_all_changes_<instance name> function cdc.fn_cdc_get_net_changes_<instance name> function (optional) Maximum two per source table

Sys.sp_CDC_ENABLE_TABLE @source_schema @source_name @supports_net_changes @role_name @filegroup_name @index_name @captured_column_list @allow_partition_switch @capture_instance

DEMONSTRATION Enabling Change Data Capture (CDC) Creating a CDC Capture Instance Querying CDC Tables as Things Change

SQL Server Agent JOBS cdc.Capture_capture Starts with SQL Agent RAISERROR(22801, 10, -1); EXEC sys.sp_MScdc_capture_job; Just a wrapper for sys.sp_cdc_scan cdc.Capture_cleanup Runs at 02:00 daily EXEC sys.sp_MScdc_cleanup_job;

CDC Data Retention Based on LSN Validity Intervals Database Capture Instance The cleanup job deletes CDC data acording to retention policy SELECT * FROM msdb.dbo.cdc_jobs WHERE job_type = N'cleanup'; You can start the cleanup job manually EXEC sys.sp_cdc_start_job @job_type = N'cleanup';

Handling Schema Changes Deleted and new columns are handled well Modified columns require specific steps: Stop the capture and cleanup jobs Change the schema as necessary Generate new capture instances for all modified tables Process all data in the old and new capture instances Manually run the cleanup job Delete the old capture instance Turn the capture and cleanup jobs back on

Change Tracking transactions warehouse Turn it on for one database ALTER DATABASE <DBNAME> SET CHANGE_TRACKING = ON; Then enable a table ALTER TABLE <TBLNAME> ENABLE CHANGE_TRACKING; sync transactions warehouse ETL sources archive sector

Change Tracking TABLES & FUNCTIONS sys.change_tracking_databases sys.change_tracking_tables CHANGETABLE(CHANGES) CHANGETABLE(VERSION) CHANGE_TRACKING_CURRENT_VERSION() CHANGE_TRACKING_MIN_VALID_VERSION() CHANGE_TRACKING_IS_COLUMN_IN_MASK() WITH CHANGE_TRACKING_CONTEXT

DEMONSTRATION Enabling Change Tracking (CT) for a Database Enabling CT on a Table Querying CT Tables and Functions as Things Change

Handling Schema Changes No modifications to the primary key are allowed including related indexes Dropping columns is OK but they may still appear in the change data When adding columns, changes are tracked but the metadata change is not reported Switching partitions will fail on change tracked changes Data type changes are not tracked

Which one is RIGHT FOR ME? Change Data Capture Change Tracking Works in Enterprise Edition only Stores every discrete change Storage intensive Good for auditing Requires SQL Server agent No special serialization required Works in all versions of SQL Server Returns differences from current Storage light Good for device synchronization No job agent required Operates best with snapshot isolation