1. Microsoft Replication Technologies
Hilary Cotter
Microsoft Replication Technologies

2. Clients

3. Agenda
Why Replicate? Alternatives Snapshot Transactional Merge Peer to Peer Sync Services Monitoring Troubleshooting Performance Replication and Mirroring Replication and AlwaysOn

4. Why Replicate? Making a copy of the data/Massaging Data in Flight
Offload reporting
Moving data closer to the consumer
Scale out/Load Balancing
Application requirements
Data Aggregation
POS
Poor man’s DR (missing dependencies, failback difficult)
Poor man’s HA (no client redirection, no predictable latency and unpredictable exposure to data loss)

5. Making a copy of the data
Different database Different server Same database

6. Same database – different table
int=1 int begin insert into testTable(Charcol) values(REPLICATE('X',20)) select end GO exec sp_replicationdboption traces,publish,true exec @snapshot_in_defaultfolder=false exec sp_addpublication_snapshot test exec sp_addarticle exec sp_addsubscription test, 'ALL', exec sp_startpublication_snapshot test waitfor delay '00:01' select * from testtaableNew

7. Same database - different schema
SameDatabaseDifferentSchema.sql

8. Same database - 2 tables to 1 table
IndexedView3.sql

9. Indexed View to a table
if exists(select * from sys.databases where name='IndexedView') begin exec sp_replicationdboption IndexedView, publish, false alter database IndexedView set single_user with rollback immediate drop database IndexedView end create database IndexedView GO exec sp_replicationdboption IndexedView, publish, true use IndexedView create table Table1(PK int identity primary key, charcol varchar(20)) create table Table2(PK int identity primary key, charcol varchar(20), FK int constraint FK_Table1 foreign key (FK) references Table1(PK)) int=1 insert into table1(charcol) values(replicate('X',20)) insert into table2(charcol,FK) select create view IndexedView with schemabinding as select table1.PK , table1.charcol, table2.charcol [table2.charcol] from dbo.Table1 join dbo.Table2 on Table1.PK=Table2.FK create unique clustered index IndexViewCL on IndexedView(PK) @Alt_snapshot_folder='c:\temp' sp_addpublication_snapshot IndexedView sp_addarticle indexedView, ='Indexed view LogBased' sp_addsubscription IndexedView, indexedView, sp_startpublication_snapshot IndexedView

10. BiDi
BiDiv2.sql

11. Offload Reporting Quote from David Baxter Browne (Microsoft)
Support concurrent OLTP and Reporting with Snapshot-based Isolation.
Limit Reporting users access to resources with Resource Governor.
Reduce the cost of reporting queries with non-clustered Columnstore indexes.
Reduce the cost of high-volume OLTP with Memory-Optimized tables.
Monitor the cost of workloads with Query Store and Session-level wait stats. And (not a SQL Sever feature, but a computing mega-trend)
Supply a single SQL Server with a large pool of Memory, IOPS and CPU resources.

12. Offload Reporting
Microsoft preferred method of offloading reporting is AlwaysOn, followed by Peer to Peer
Problems
Latency – not so problematic with SQL 2016
Reporting Indexes must be on the Primary, punishes writes there.
Enterprise Edition feature only
Alternatives
Bi-directional transactional replication – only 2 nodes
Peer to Peer replication
Enterprise Edition only feature,
limited feature set – no identity property support
Need to quiesce when making schema changes
Difficult to maintain

13. Alternatives Golden Gate Log Shipping Mirroring AlwaysON
Service Broker
Does not scale well to multiple subscribers
How to do it?
Roll your own
Need to implement store and forward and change tracking
tracking deletes is hard, CT/CDC options

14. Performance Snapshot Initialize from a backup Database Snapshot
Log Reader Agent

15. Replication Types
Snapshot
Transactional
Peer to Peer
Merge

16. Snapshot Replication Point in time image of data sent to subscriber
Best fit when the majority of the data changes at one time
Catalogs
Price lists
Poor use cases
Tables don’t have PKs
No real time synchronization requirements

17. Transactional Replication
Replicates transactions from publisher to subscriber in a transactional context
Store and Forward mechanism
Tlog read asynchronously
Commands constructed and written to distribution database
Markers placed in log saying Transaction X read
LSN – log sequence numbers correlates to
XactSequence in profiler
xact_seqno in msrepl_transactions
Transactional boundaries are respected when applying commands on the subscribe
20 rows updated on publisher
20 rows read by log reader and written to distribution database
20 rows applied on subscriber in a transaction.
Upon failure the 20 row application is rolled back

18. Transaction Replication
Can Replicate execution of a stored procedure
Serializable -- SET TRANSACTION ISOLATION LEVEL Serializable;
Begin tran tran1
Exec Proc1
Begin tran tran2
Exec proc is committed before 1
Commit tran tran2
Commit tran tran1

19. Updateable Subscribers
Designed for the case where the majority of the DML originates at the Publisher
Contrast with merge where it is anywhere
Immediate - MS DTC – linked server
committed at publisher before subscriber
Queued
Uses queue reader
Long standing bug in Replication Scripting adds a queue reader for all transactional replication types

20. Updateable Subscribers
Immediate with queued failover
If Publisher goes down
Published tables go readonly
Deprecated in SQL 2012
Can still use it through stored procedures
Don’t use it
Small user base

21. Bi-Directional Transactional
Two way replication
No guid key required
Publisher is also a subscriber
sp_addsubscription
@loopback_detection=true
Faster than merge and p2p
Only scalable to 2 nodes
Can do three
Great for scaling read outs

22. Bi-Directional Transactional
demo

23. Bi-Directional Transactional
Caveats
Swapping data
Identity seeding
Downtime
The longer the downtime the greater the probability of duplicate data

24. Peer to Peer (P2P) EE Feature only Bi-directional
Uses a mesh technology
Each node can replicate to any other nodes in the topology.
A node can drop off and come back on the topology and automatically synchronize
SQL 2008 has conflict detection
No conflict resolution
Smaller feature set the Transactional Replication
No custom stored procedures
Network saturated at 10 nodes

25. P2P demo

26. Merge Replication
Designed for frequently disconnected clients which need to bi-directionally replicate Ideal for POS applications Uses a tracking column (rowguid) to uniquely identify rows across a replication topology Uses tracking triggers to do change detection Merge agent enumerates changes occurring between the publisher and subscriber between synchronizations and downloads changes to either side. Should a change to a row occur on both sides of a merge replication topology the lineage column on msmerge_contents – sync cookie

27. Merge Replication Conflicts Conflict Priority Error
Changes to same row (row level tracking) or column (column level tracking)
Conflict Priority
Server/Global
Client/Local
Determines who conflicts are persisted.
Server – first to publisher wins and persists
Client – determined by assigned priority
Publisher changes always persist

28. Merge Replication Conflict resolution
Microsoft SQL Server Additive Conflict Resolver
Microsoft SQL Server Averaging Conflict Resolver
Microsoft SQL Server DATETIME (Earlier Wins) Conflict Resolver
Microsoft SQL Server DATETIME (Later Wins) Conflict Resolver
Microsoft SQL Server Download Only Conflict Resolver
Microsoft SQL Server Maximum Conflict Resolver
Microsoft SQL Server Merge Text Columns Conflict Resolver
Microsoft SQL Server Minimum Conflict Resolver
Microsoft SQL Server Priority Column Resolver
Microsoft SQL Server Subscriber Always Wins Conflict Resolver
Microsoft SQL Server Upload Only Conflict Resolver
Microsoft SQLServer Stored Procedure Resolver

29. Merge Replication Rich Filtering Web Synchronization
Secure transport
Rich Filtering
Can filter vertically and horizontally
Changes to parent rows will cause motion in child rows.
For example if you are filtering on region, and a sales person gets a new region all the child rows belonging to that region will be replicated down to the sales person’s subscription.
Termed a partition
Precomputed Partitions
Partitions are evaluated at run time (when changes happen) as opposed to filter/sync time
Can result in very short syncs, but sometimes depending on the hierarchies in the filters can result in longer DML time

30. Business Logic Resolver
.Net class that allows you to
Handle Merge Errors
Conflicts
Update
Insert
Delete
Do things on each synchronization
Overhead
Stored Procedure Resolver

31. Performance Tuning - transactional

32. Performance Tuning - transactional

33. Performance Tuning

34. Merge Replication Maximize Generations per batch Minimize Conflicts
UploadGenerationsPerBatch
DownloadGenerationsPerBatch
UploadReadChangesPerBatch
DownloadReadChangesPerBatch
UploadWriteChangesPerBatch
DownloadWriteChangesPerBatch
Minimize Conflicts
Minimize Concurrent merge Agents

35. Troubleshooting Enable agents for logging Run from the command line
Break up publication
For Transactional Replication
sp_replcounters
sp_browsereplcmds
sp_setsubscriptionxactseqno
For Merge Replication – tracking changes
sp_showlineage
sp_showcolv
sp_mergedummyupdate
sp_showpendingchanges

36. Troubleshooting Limit number of concurrent merge syncs
sp_changemergepublication 'MyPublication','max_concurrent_merge','10‘
StartQueueTimeout
Reindex merge system tables nightly

37. Questions

38. Content Only

40. Section Title