1. Migrate multi-terabyte DB from On-Prem to Azure SQL with minimal downtime

2. WHY MOVE TO ASD? Dynamic Scaling Pay-per-usage
Low maintenance overhead
HA / DR
Native Scale-out capability (sharding)
Elastic Queries/Jobs
WHY MOVE TO ASD?
S / Mention transitioning pains at the end – 5
Image credit:

3. Azure SQL DB. A different kind of animal
Compatibility is only guaranteed for DML
No native backup restore
No support for operations that read from the transaction log
No support for log-replay operations which eliminates conventional on-prem style migration options
Mirroring
Always-on
Log shipping
Azure SQL DB. A different kind of animal
5 min

4. Pre-Migration checklist
Verify that database schema is compatible with ASD
Certain DDL options are not supported (For example NON-Primary filegroups)
Some DML is optimized differently (NON-ANSI style joins would not perform well)
No matter which migration strategy is chosen, DMA assessment tool can be used.
For further details refer to DMA documentation ( assesssqlonprem)
Pre-Migration checklist
I 5 min

5. On-PREM to ASD migration METHODS
Using Database Migration Assistant.
Designed with simplicity in mind
Ideal for smaller databases
Migration is offline
No resume on runtime error
Time it takes to create/upload/import is prohibitive for large database migrations (2+ hours for 10GB database)
Bacpac backup restore
Not transactionally consistent
200GB max if using blob storage
If export takes >20 hrs (6-12 if no CI) it may be canceled
Transactional replication
Online
Added flexibility (schema vs data)
Added complexity for migration project (negative)
On-PREM to ASD migration METHODS
DMA
S 5 min

6. MIGRATION TIMELINE Overview2

7. Transactional replication DATABASE CONFIGURATION MANAGEMENT Challenges
Database schema must adhere to certain standards:
PK on every table
Certain DDL operations are not allowed on published tables (Truncate/Drop) which may complicate database CM for the replicated database
Subscriber database schema gets mutated while applying the snapshot using default options
Transactional replication DATABASE CONFIGURATION MANAGEMENT Challenges
2 min

8. DEMO #1 Deploy Database Schema to subscriber Enable distribution
Create publication template
Add and initialize the subscription
Watch the data flow from IaaS to PaaS
Initialization using the default article snapshot options
10 min

9. TRANSACTIONAL REPLICATION COMPONENT INTERACTION DIAGRAM
2 min

10. CM migration Process summary
Use wizard for template creation only. After that, generate scripts and operate with them to achieve the following benefits:
Ability to control advanced replication behavior options
Repeatable SDLC environment deployments
Override default snapshot article creation command
= ‘delete’
Do not rely on replication to deliver subscriber database schema. Use your established database CM process instead.
Modify DML Triggers to use NFR option
If encryption is in use, encryption keys will need to by synchronized
CM migration Process summary
2 min

11. Migration Timeline Position:
Monitoring Replication
1 min

12. Demo#2 Replication MONITOR iS your best friend
5 min
Adding publisher
Checking Agents’ status
Using tracer tokens

13. Monitoring the initialized publications - Summary
Replication Monitor is your best friend
Latency estimation is not accurate
Use tokens to measure latency
Do not rely on Replication Monitor GUI for replication health validation
Set alerts for suspect agent conditions (latency, errors, etc..)
Monitoring the initialized publications - Summary
2 min

14. Migration Timeline Position:
Making the switch
1 min

15. DEMO #3
Application Tier
S 5 min

16. Migration Timeline Position:
Adding consistency checks
S 1 min

17. In-flight subscriber consistency
There is no built in mechanism on the subscriber to prevent direct modifications to the destination database outside of the replication agent.
Unlike Mirroring or Always On, the destination database is not in read-only mode
Accidental modification statements will not be prevented and therefore will invalidate the integrity of the migration effort
Use the script#3 to automatically add rollback triggers to every table on the subscriber (Demo on how NFR option works)
In-flight subscriber consistency
I 2min

18. questions

19. Transactional replication overview
First introduced in SQL 6.0 (log-based)
Low IO impact on the source database
Asynchronous in nature (will not increase OLTP transactional latency)
Guaranteed delivery mechanism
Flexible synchronization solution. Can synchronize across different database versions or even platforms
Scalable (with some limitations)
Designated distributor option
Push vs. Pull model determines location of the distribution agent. Important for two reasons:
Can further relief pressure from the distributor
Firewall rules may impact choice of topology
*Pull is not relevant to our use migration use case
Still, only one distributor per published instance
Included with SQL Standard edition
Transactional replication overview I

20. Solving performance Issues WITH Replication
Potential bottleneck areas:
Log Reader agent.
Distribution Database
Distribution Agent
Solving performance Issues WITH Replication

21. Publisher Performance Optimizations
IO overhead is only added to the Log file
Slow read IO on the publisher log file – doubtful (log reads are a freebie)
Unnecessarily large active log file portion. (verify the regular log release suspects)
Publisher Performance Optimizations

22. Distribution Database Performance optimizations
Normal troubleshooting technique for busy OLTP
Distribution clean-up job schedule
Distribution Database Performance optimizations

23. Subscriber Performance Optimizations
Subscriber latency with high-transactional publishers
In most cases, command replay is a scalar operation. Set-based statements get broken down into individual proc calls during distribution in order to deliver data to subscriber.
Use SNAPHOT isolation level on the subscriber
Consider replicating procs that affect large number of rows (where possible)
Break up publisher database into multiple publications
Multithread distribution agents using the -SubscriptionStreams parameter
Subscriber Performance Optimizations
* Beware of multithreading trade-off: performance vs. consistency. Use NFR option for FKs on subscriber in case of large number of distribution agent errors.

24. Igor Chmil
Twitter
LinkedIn:
Stan Makarsky
LinkedIn:
Contact INFO