Monitoring Data Changes with Change Data Capture Arie D. Jones (AJ) Principal Technology Manager Perpetual Technologies, Inc Blog : http://www.programmersedge.com Company Site: http://www.pti.net Email: arie.jones@pti.net Twitter: programmersedge #45 Louisville

Arie Jones (AJ) 20 + years application development experience 10 + years database experience Bunch of Alphabet Soup behind name The blog, the webcasts, the events….. Prolific writer SQL Functions Learn SQL in 24 Hours Learn SQL in 1Hour a Day Live Lessons: SQL

Agenda How does Change Data Capture work How to implement How to get to the data Sneaky Ninja DBA Tips

? How it Works So how does this mysterious Change Data Capture work? Let‘s take a look at how the magic happens….. That big question mark is editable – use whatever character you want!

Why use this? Gives you consistency Reduces programmatic overhead Reduces computational overhead

How it all works! Database Source Tables Log File Change Tables Capture Process User Queries

? How to Implement That’s awesome AJ but how to you get the thing working?? Let’s find out! That big question mark is editable – use whatever character you want!

Your Pre-Flight Checklist SQL Agent is on and set to Auto-Start You or someone you know has SysAdmin privileges CDC user does not exist in database A CDC schema does not exist in database You have a good cover story to explain how difficult the following process is to your manager/boss so as to strengthen your argument for a raise

Step #1 Enable on the instance Use sp_cdc_enable_db_change_data_capture Use sp_cdc_enable_db You may double check in sys.databases This you must have SysAdmin privileges for

Step #2 Wipe sweaty brow Enable CDC per table Use sp_cdc_enable_table_change_data_capture Use sp_cdc_enable_table You can double check in sys.tables You will notice 2 jobs having been created 1 = Get Stuff 2 = Cleanup Stuff

Sp_cdc_enable_table_change_data_capture @source_schema @source_name @role_name @capture_instance @supports_net_changes @index_name @captured_column_list @filegroup_name

Demo Let’s look at this in action!

? How to Query the Data Now, how do we get to the actual data that we are saving up? That big question mark is editable – use whatever character you want!

Querying Data Super easy syntax TableName_CT Will normally need system functions to get LSN Sys.fn_cdc_map_time_to_lsn Then will use one of 2 CDC query TVF Cdc.fn_cdc_get_all_changes_<schema>_<table> Cdc.fn_cdc_get_net_changes_<schema>_<table>

Sneaky Ninja DBA TIP #1! ETL Processes!

Demo Straight to a demo!

Sneaky Ninja DBA TIP #2! data forensics

Who’s making changes? You’ll notice in the CDC tables that there is no username… 2 Things you need to think about How are the users accessing the system? How much flexibility do you have to change things?

Easy way! ID Customer User Updated 1 Jane Good DBA 2 Steve Evil Developer 3 Bob ID Customer 1 Jane 2 Steve 3 Bob Change Data Capture takes care of the rest!!

Sneaky Ninja DBA TIP #3! Performance tuning!

Let me explain the process Job Sys.sp_MScdc_capture_job No Parameters! That’s CRAZY! Sys.sp_cdc_scan Hey wait! Isn’t that used in transactional replication? Sys.sp_replcmds

Demo Let’s jump into some code!

So What’s a DBA to Do? @captured_column_list @supports_net_changes Limit # of columns to only those you truly need @supports_net_changes Only use if you need net changes Puts an additional nonclustered index on change table @filegroup_name If not specified then change table is placed in default filegroup

Performance Tips Storage Workload Watch your log space Watch you log I/O Make sure you know which file group you have things stored in Workload Reduce the number of columns to the minimum Modify default scan parameters on BOTH the capture and cleanup jobs!

Thank You ! Questions and Answers Evaluation Sheets! Be sure to check out my upcoming events! PTI Monthly Webcasts: http://events.pti.net

SQL Saturday #45 Louisville @ SQL Saturday #45 Louisville Monitoring Data Changes with Change Data Capture Arie Jones(AJ) http://www.programmersedge.com http://www.sqlsherpa.com Email: arie.jones@pti.net Twitter: programmersedge Company: http://www.pti.net