Module 4: Implementing Transactional Replication

Overview Understanding Transactional Replication Architecture Replicating with Transactional Replication Implementing Updatable Transactional Replication Facilitating Transactional Replication by Using Stored Procedures Troubleshooting Transactional Replication Tuning Transactional Replication

Lesson: Understanding Transactional Replication Architecture How Transactional Replication Moves Data The Role of the Log Reader Agent How the Log Reader Agent Reads the Transaction Log How the Log Reader Agent Changes Transaction Syntax The Role of the Distribution Agent Independent and Shared Agents Concurrent Snapshot Generation

How Transactional Replication Moves Data Snapshot ensures synchronized table schema and data Log Reader Agent scans transaction log Log Reader Agent copies transactions marked for replication to Distributor Distribution Agent moves transactions from Distributor to Subscribers

The Role of the Log Reader Agent 11 The Log Reader Agent is created when the first transactional publication is created on the Publisher Log Reader Agent CreatedCreated 22 The Log Reader Agent looks in the distribution database for the last LSN read Each time the Log Reader Agent runs: 33 The Log Reader Agent reads the transaction log of the publishing database starting with the next LSN distribution LSN: 18 Transaction log 11 The Log Reader Agent is created when the first transactional publication is created on the Publisher Log Reader Agent 22 The Log Reader Agent looks in the distribution database for the last LSN read Each time the Log Reader Agent runs: 33 The Log Reader Agent reads the transaction log of the publication database starting with the next LSN distribution LSN: 18 Transaction log

How the Log Reader Agent Reads the Transaction Log 11 Finds the next LSN 22 Starts reading transactions 33 Ignores transactions related to nonpublished objects 44 Copies the published transactions to the distribution database 55 Marks the log as transactions read to here LSN:************** 29: XXXXX LSN:************** 17: UPDATE LSN:************** 18: XXXXX LSN:************** 19: UPDATE LSN:************** 20: UPDATE LSN:************** 21: INSERT LSN:************** 22: DELETE LSN:************** 23: UPDATE LSN:************** 24: INSERT LSN:************** 25: INSERT LSN:************** 26: UPDATE LSN:************** 27: INSERT LSN:************** 28: INSERT UPDATE UPDATE INSERT UPDATE …66 Anything prior to the mark can be truncated

How the Log Reader Agent Changes Transaction Syntax INSERT INTO Emp VALUES (57, 'Bremer', 'Ted') CALL sp_MSins_Emp (57, 'Bremer', 'Ted') May convert to a stored procedure May convert to a stored procedure UPDATE XYZ SET … DELETE XYZ … INSERT XYZ … May convert to an DELETE/INSERT May convert to an DELETE/INSERT UPDATE Emp SET Phone = ' ' WHERE LastName = 'Bremer' AND FirstName = 'Ted' CALL sp_MSupd_Emp (57, ' ') Uses primary key

The Role of the Distribution Agent distribution database subscribing database Distribution Agent DTS Package DTS Data Pump DTS Transformable subscriptions only

subscribing database Independent and Shared Agents Shared Agent Independent Agent

Concurrent Snapshot Generation Marks the log Copies the tables Marks the log Checks the log between the marks for transactions marked for replication Applies the initial snapshot Applies the transactions copied from the marked portion of the log Publishing Database Transaction log INSERT… X Subscribing Database Tables procs views Distributor SQL X INSERT…

Lesson: Replicating with Transactional Replication Designing Tables for Transactional Replication Replicating text, ntext, and image Data Replicating Horizontally Filtered Data Overhead of Filtering for Transactional Replication

Designing Tables for Transactional Replication Y Related rowINSERT X, Y, Z Insert on Publisher checks the relationship It is not necessary to check again on the Subscriber ReplicationReplication ReplicationReplication Use NOT FOR REPLICATION for relationships Y Related row INSERT X, Y, Z

Practice: Designing Tables for Transactional Replication Adding syntax ALTER TABLE and CLUSTERED INDEX ALTER TABLE and FOREIGN KEY CREATE TRIGGER

Replicating text, ntext, and image Data Reinitialize the publication if nonlogged operations have occurred Use the WITH LOG option with text operations Use max text repl size with ODBC subscribers

Replicating Horizontally Filtered Data CREATE PROCEDURE CustFilter FOR REPLICATION AS IF EXISTS ( SELECT Country FROM Customers WHERE Country IN (SELECT Country FROM Promotion (NOLOCK)) RETURN 1 ELSE RETURN 0 CREATE PROCEDURE CustFilter FOR REPLICATION AS IF EXISTS ( SELECT Country FROM Customers WHERE Country IN (SELECT Country FROM Promotion (NOLOCK)) RETURN 1 ELSE RETURN 0

Overhead of Filtering in Transactional Replication Number of filters Every row tested against every filter Each filter is evaluated independently Complexity Complexity determines overhead Joins, functions, comparisons, and indexes contribute to overhead

Lesson: Implementing Updatable Transactional Replication What Are Updating Subscriptions? Limitations of Updating Subscriptions How Subscribers Using Immediate Updating Subscriptions Update Tables Schema Alterations for Subscribers Using Immediate Updating Subscriptions Choosing a Queue Type How Queued Updating Subscriptions Update Tables (SQL Server 2000 queue) How Queued Updating Subscriptions Update Tables (MSMQ) Managing Conflicts with Subscribers Using Queued Updating Subscriptions Implementing Immediate Updating with Failover to Queued Updating Managing Identity Values for Updating Subscriptions

Subscriber using immediate updating subscription Subscriber using queued updating subscription What Are Updating Subscriptions? Publisher Distributor Queue Reader Agent Reliable Connection Queue Intermittent Connection Intermittent Connection

Limitations of Updating Subscriptions No merge replication No republishing replicated data at the Subscriber All text and image data should be treated as read-only Cannot disable the updating subscription Cannot use INSERT statements without a column list Cannot combine with transformable subscriptions Cannot use pre-existing data at Subscribers

How Subscribers Using Immediate Updating Subscriptions Update Tables Publisher updates table and forwards transactions Subscriber starts transaction and connects with MS DTC Subscriber starts transaction and connects with MS DTC MS DTC completes data modification with two-phase commit (2PC) Publisher distributes update to other Subscribers according to schedule

Schema Alterations for Subscribers Using Immediate Updating Subscriptions MSrepl_tran_version Added by wizard if needed Include vertical filter Add to column list System-defined triggers Added automatically when subscribers enabled Make calls to MS DTC Avoiding a trigger-firing loop Fire user-defined triggers after immediate updating triggers

Choosing a Queue Type SQL Server 2000 queue Works with all SQL Server platforms Easy to set up No additional components to install Faster Microsoft Message Queue (MSMQ) Centralized queue administration and monitoring Propagate changes when Subscriber is not running SQL Server Does not require available Publisher when Subscriber reconnects

Publisher updates table and forwards transactions using typical transactional replication Subscriber trigger places copy of new transactions in MSreplication_queue SQL Server Queue Reader Agent checks for new rows to be moved to Distributor queue SQL Server Queue Reader Agent uses MS DTC to move rows to Publisher Publisher distributes update to other Subscribers SQL Server 2000 Queue: How Queued Updating Subscriptions Update Tables (SQL Server 2000 Queue)

Publisher updates table and forwards transactions using typical transactional replication MSMQ places transaction in MSMQ-managed queue MSMQ service on Subscriber moves row to MSMQ service on Distributor by using MS DTC MSMQ service on Distributor connects to Publisher and updates Subscriber table by using MS DTC Publisher distributes update to other Subscribers MSMQ:MSMQ: How Queued Updating Subscriptions Update Tables (MSMQ)

Managing Conflicts with Subscribers Using Queued Updating Subscriptions How SQL Server detects conflicts Choosing a conflict resolver Keep change made at Publisher Keep change made at Subscriber Reinitialize the subscription

Implementing Immediate Updating with Failover to Queued Updating Immediate updating 11 Using immediate updating 22 Connection is unavailable 33 Switch to queuing 44 When connection is available – empty the queue 55 Resume immediate updating Queue

Managing Identity Values for Updating Subscriptions Methods for managing identity values Use SQL Server 2000 automatic management Use the NOT FOR REPLICATION option Use a primary key other than the identity column Identity values and immediate updating Publisher controls identity values You cannot assign identity ranges Identity values and queued updating Assigned by the Subscriber Identity values and queued updating failover Assign identity ranges to Subscribers Inserted rows generate identity values from the range

Lesson: Facilitating Transactional Replication by Using Stored Procedures Understanding Custom Stored Procedures for Replication Creating Custom Stored Procedures Implementing Custom Stored Procedures

Understanding Custom Stored Procedures for Replication Benefits of custom stored procedures Improved performance Less data passed over the network Command types CALL MCALL XCALL SQL NONE

Creating Custom Stored Procedures sp_scriptinsproc sp_scriptdelproc sp_scriptupdproc sp_scriptmappedupdproc sp_scriptdynamicupdproc sp_scriptpublicationcustomprocs

Implementing Custom Stored Procedures

Lesson: Troubleshooting Transactional Replication Troubleshooting by Using the Distribution and Log Reader Agents Troubleshooting by Using Stored Procedures Skipping Errors to Prevent Replication Halts Reinitializing Subscriptions Troubleshooting Steps Troubleshooting Guidelines

Troubleshooting by Using the Distribution and Log Reader Agents "C:\Program Files\Microsoft SQL Server\80\COM\logread" -Publisher [VANCOUVER] -PublisherDB [Northwind] -DistributorSecurityMode 1 "C:\Program Files\Microsoft SQL Server\80\COM\distrib" -Publisher [VANCOUVER] -PublisherDB [Northwind] -Publication [CustomersTransactional] -DistributorSecurityMode 1 "C:\Program Files\Microsoft SQL Server\80\COM\distrib" -Publisher [VANCOUVER] -PublisherDB [Northwind] -Publication [CustomersTransactional] -DistributorSecurityMode 1 Log Reader Agent Distribution Agent

Troubleshooting by Using Stored Procedures sp_browsereplcmds – Commands waiting at Distributor sp_replshowcmds – Commands waiting at Publisher sp_repltrans – LSNs in transaction log sp_replflush – Stop acting as the Log Reader Agent sp_repldone – System use only sp_replqueuemonitor – Lists queue messages DBCC OPENTRAN – Returns oldest LSNs sp_replcmds – System use only

Practice: Troubleshooting by Using Stored Procedures Create a push subscription and subscribe Identify commands at Distributor Identify commands at Publisher

Skipping Errors to Prevent Replication Halts

Reinitializing Subscriptions

Troubleshooting Steps Check Replication Monitor Change -BatchCommitSize Disable Cleanup Rerun distribution Check MSreplication_subscriptions Identify the row Narrow the query Map the sequence number Display commands

Troubleshooting Guidelines Use SQL error logs Run DBCC checks Check for blocking issues Analyze index and query performance on subscribing tables View application and System Event Viewer logs Check triggers

Lesson: Tuning Transactional Replication Identifying Subscriber Factors That Degrade Performance Improving Performance Through Design and Configuration Improving Performance by Using Appropriate Subscription Types and Scheduling Improving Performance by Using Agent Properties

Identifying Subscriber Factors That Degrade Performance Subscriber computer factors Subscription database or SQL Server setup factors Network speed or connection factors Additional factors

Improving Performance Through Design and Configuration Design factors Use custom stored procedures Avoid individual row evaluation Publish as indexed views instead of filtered tables Avoid the update of unique constrained columns Configuration factors Set the distribution database to a fixed size Configure the Distributor on a dedicated server

Improving Performance by Using Appropriate Subscription Types and Scheduling Subscription factors Use push and pull subscriptions appropriately Use anonymous subscriptions appropriately Scheduling factors Run agents continuously Reduce distribution frequency Schedule snapshots during periods of low activity

Improving Performance by Using Agent Properties Property of both the Log Reader Agent and Distribution Agent Use -PollingInterval parameter to adjust latency Log Reader Agent Properties Increase -ReadBatchSize parameter Retain -ReadBatchThreshold default Adjust the -MaxCmdsInTran setting Distribution Agent Properties Increase -CommitBatchSize and -CommitBatchThreshold defaults

Review Understanding Transactional Replication Architecture Replicating with Transactional Replication Implementing Updatable Transactional Replication Facilitating Transactional Replication by Using Stored Procedures Troubleshooting Transactional Replication Tuning Transactional Replication

Lab 4: Implementing Transactional Replication Exercise 1: Designing a Transactional Replication Solution

Course Evaluation