Microsoft Replication Technologies Hilary Cotter Microsoft Replication Technologies

Clients

Agenda Why Replicate? Alternatives Snapshot Transactional Merge Peer to Peer Sync Services Monitoring Troubleshooting Performance Replication and Mirroring Replication and AlwaysOn

Why Replicate? Making a copy of the data/Massaging Data in Flight Offload reporting Moving data closer to the consumer Scale out/Load Balancing Application requirements Data Aggregation POS Poor man’s DR (missing dependencies, failback difficult) Poor man’s HA (no client redirection, no predictable latency and unpredictable exposure to data loss)

Making a copy of the data Different database Different server Same database

Same database – different table declare @counter int=1 declare @rowcounter int while @counter<=1000 begin insert into testTable(Charcol) values(REPLICATE('X',20)) select @counter=@counter+1 end GO exec sp_replicationdboption traces,publish,true exec sp_addpublication test, @status=active, @alt_snapshot_folder='c:\temp', @snapshot_in_defaultfolder=false exec sp_addpublication_snapshot test exec sp_addarticle test, testtable, @source_object=testtable, @destination_table=testtaableNew exec sp_addsubscription test, 'ALL', @Subscriber=@@Servername exec sp_startpublication_snapshot test waitfor delay '00:01' select * from testtaableNew

Same database - different schema SameDatabaseDifferentSchema.sql

Same database - 2 tables to 1 table IndexedView3.sql

Indexed View to a table if exists(select * from sys.databases where name='IndexedView') begin exec sp_replicationdboption IndexedView, publish, false alter database IndexedView set single_user with rollback immediate drop database IndexedView end create database IndexedView GO exec sp_replicationdboption IndexedView, publish, true use IndexedView create table Table1(PK int identity primary key, charcol varchar(20)) create table Table2(PK int identity primary key, charcol varchar(20), FK int constraint FK_Table1 foreign key (FK) references Table1(PK)) declare @counter int=1 while @counter<1000 insert into table1(charcol) values(replicate('X',20)) insert into table2(charcol,FK) values(replicate('Y',20), @counter) select @counter=@counter+1 create view IndexedView with schemabinding as select table1.PK , table1.charcol, table2.charcol [table2.charcol] from dbo.Table1 join dbo.Table2 on Table1.PK=Table2.FK create unique clustered index IndexViewCL on IndexedView(PK) sp_Addpublication IndexedView, @status=active, @snapshot_in_defaultfolder=false, @Alt_snapshot_folder='c:\temp' sp_addpublication_snapshot IndexedView sp_addarticle indexedView, IndexedView, @source_object=IndexedView, @destination_table=Destination,@type ='Indexed view LogBased' sp_addsubscription IndexedView, indexedView, @subscriber=@@servername sp_startpublication_snapshot IndexedView

BiDi BiDiv2.sql

Offload Reporting Quote from David Baxter Browne (Microsoft) Support concurrent OLTP and Reporting with Snapshot-based Isolation. Limit Reporting users access to resources with Resource Governor. Reduce the cost of reporting queries with non-clustered Columnstore indexes. Reduce the cost of high-volume OLTP with Memory-Optimized tables. Monitor the cost of workloads with Query Store and Session-level wait stats. And (not a SQL Sever feature, but a computing mega-trend) Supply a single SQL Server with a large pool of Memory, IOPS and CPU resources.

Offload Reporting Microsoft preferred method of offloading reporting is AlwaysOn, followed by Peer to Peer Problems Latency – not so problematic with SQL 2016 Reporting Indexes must be on the Primary, punishes writes there. Enterprise Edition feature only Alternatives Bi-directional transactional replication – only 2 nodes Peer to Peer replication Enterprise Edition only feature, limited feature set – no identity property support Need to quiesce when making schema changes Difficult to maintain

Alternatives Golden Gate Log Shipping Mirroring AlwaysON Service Broker Does not scale well to multiple subscribers How to do it? http://rusanu.com/2011/07/20/how-to-multicast-messages-with-sql-server-service-broker/ Roll your own Need to implement store and forward and change tracking tracking deletes is hard, CT/CDC options

Performance Snapshot Initialize from a backup Database Snapshot Log Reader Agent

Replication Types Snapshot Transactional Peer to Peer Merge

Snapshot Replication Point in time image of data sent to subscriber Best fit when the majority of the data changes at one time Catalogs Price lists Poor use cases Tables don’t have PKs No real time synchronization requirements

Transactional Replication Replicates transactions from publisher to subscriber in a transactional context Store and Forward mechanism Tlog read asynchronously Commands constructed and written to distribution database Markers placed in log saying Transaction X read LSN – log sequence numbers correlates to XactSequence in profiler xact_seqno in msrepl_transactions Transactional boundaries are respected when applying commands on the subscribe 20 rows updated on publisher 20 rows read by log reader and written to distribution database 20 rows applied on subscriber in a transaction. Upon failure the 20 row application is rolled back

Transaction Replication Can Replicate execution of a stored procedure Serializable -- SET TRANSACTION ISOLATION LEVEL Serializable; Begin tran tran1 Exec Proc1 Begin tran tran2 Exec proc2 -- 2 is committed before 1 Commit tran tran2 Commit tran tran1

Updateable Subscribers Designed for the case where the majority of the DML originates at the Publisher Contrast with merge where it is anywhere Immediate - MS DTC – linked server committed at publisher before subscriber Queued Uses queue reader Long standing bug in Replication Scripting adds a queue reader for all transactional replication types

Updateable Subscribers Immediate with queued failover If Publisher goes down Published tables go readonly Deprecated in SQL 2012 Can still use it through stored procedures Don’t use it Small user base

Bi-Directional Transactional Two way replication No guid key required Publisher is also a subscriber sp_addsubscription @loopback_detection=true Faster than merge and p2p Only scalable to 2 nodes Can do three Great for scaling read outs

Bi-Directional Transactional demo

Bi-Directional Transactional Caveats Swapping data Identity seeding Downtime The longer the downtime the greater the probability of duplicate data

Peer to Peer (P2P) EE Feature only Bi-directional Uses a mesh technology Each node can replicate to any other nodes in the topology. A node can drop off and come back on the topology and automatically synchronize SQL 2008 has conflict detection No conflict resolution Smaller feature set the Transactional Replication No custom stored procedures Network saturated at 10 nodes

P2P demo

Merge Replication Designed for frequently disconnected clients which need to bi-directionally replicate Ideal for POS applications Uses a tracking column (rowguid) to uniquely identify rows across a replication topology Uses tracking triggers to do change detection Merge agent enumerates changes occurring between the publisher and subscriber between synchronizations and downloads changes to either side. Should a change to a row occur on both sides of a merge replication topology the lineage column on msmerge_contents – sync cookie

Merge Replication Conflicts Conflict Priority Error Changes to same row (row level tracking) or column (column level tracking) Conflict Priority Server/Global Client/Local Determines who conflicts are persisted. Server – first to publisher wins and persists Client – determined by assigned priority Publisher changes always persist

Merge Replication Conflict resolution Microsoft SQL Server Additive Conflict Resolver Microsoft SQL Server Averaging Conflict Resolver Microsoft SQL Server DATETIME (Earlier Wins) Conflict Resolver Microsoft SQL Server DATETIME (Later Wins) Conflict Resolver Microsoft SQL Server Download Only Conflict Resolver Microsoft SQL Server Maximum Conflict Resolver Microsoft SQL Server Merge Text Columns Conflict Resolver Microsoft SQL Server Minimum Conflict Resolver Microsoft SQL Server Priority Column Resolver Microsoft SQL Server Subscriber Always Wins Conflict Resolver Microsoft SQL Server Upload Only Conflict Resolver Microsoft SQLServer Stored Procedure Resolver

Merge Replication Rich Filtering Web Synchronization Secure transport Rich Filtering Can filter vertically and horizontally Changes to parent rows will cause motion in child rows. For example if you are filtering on region, and a sales person gets a new region all the child rows belonging to that region will be replicated down to the sales person’s subscription. Termed a partition Precomputed Partitions Partitions are evaluated at run time (when changes happen) as opposed to filter/sync time Can result in very short syncs, but sometimes depending on the hierarchies in the filters can result in longer DML time

Business Logic Resolver .Net class that allows you to Handle Merge Errors Conflicts Update Insert Delete Do things on each synchronization Overhead Stored Procedure Resolver

Performance Tuning - transactional

Performance Tuning - transactional

Performance Tuning

Merge Replication Maximize Generations per batch Minimize Conflicts UploadGenerationsPerBatch DownloadGenerationsPerBatch UploadReadChangesPerBatch DownloadReadChangesPerBatch UploadWriteChangesPerBatch DownloadWriteChangesPerBatch Minimize Conflicts Minimize Concurrent merge Agents

Troubleshooting Enable agents for logging Run from the command line Break up publication For Transactional Replication sp_replcounters sp_browsereplcmds sp_setsubscriptionxactseqno For Merge Replication – tracking changes sp_showlineage sp_showcolv sp_mergedummyupdate sp_showpendingchanges

Troubleshooting Limit number of concurrent merge syncs sp_changemergepublication 'MyPublication','max_concurrent_merge','10‘ StartQueueTimeout Reindex merge system tables nightly

Questions

Content Only

Section Title