Distributing Data for Availability and Scalability Don Vilen Program Manager SQL Server Microsoft Corporation

Agenda  Replication Technology Positioning  SQL Server Replication Customer Scenarios  Upgrade Strategy  Key Improvements in SQL Server 2005 Replication

SQL Server Replication A Spectrum of Solutions MergeReplicationQueuedUpdatingSubscribers Automatic conflict detection & resolution Linked Servers and Microsoft Distributed Transaction Coordinator Transactional Replication Peer-to-Peer ImmediateUpdatingSubscribers Conflicts are avoided through partitioning or two- phase commit Increasing autonomy Increasing consistency

Replication Technologies Server Scenarios  Transactional Replication  Best choice continuously synchronizing databases to support scaling out query workloads  SQL Server Integration Services (was DTS)  Best choice for performing transformations on data during transfer between databases  Database Mirroring  Best choice for providing database high availability  Disk / Device Mirroring  Third-party solutions  Ensure it meets SQL Server I/O Basics requirements

Mobile Synchronization Comparison Merge Replication Offline Datasets Web Services Incremental Changes Conflict detection and resolution Automatically initialize schema and data Supports large datasets Query processor locally available Unique IDs across remote sites Database constraints locally maintained Supports heterogeneous databases Does not affect database schema Deploy application and db together

Agenda  Replication Technology Positioning  SQL Server Replication Customer Scenarios  Upgrade Strategy  Key Improvements in SQL Server 2005 Replication

Customer Scenarios  Offloading reporting and Data Warehousing  Database scale out for websites  Empowering mobile users  Enabling Point of Sale applications  Consolidating and distributing regional data  Integrating heterogeneous data

Reporting and Data Warehousing  Reporting databases are continuously synchronized in near real-time with one or more source OLTP databases  Reporting and Staging user queries are offloaded from the OLTP databases  Reporting data can be a subset of source tables and columns and also indexed differently  Read-only replicas

Database Scale Out  Queries scaled out (often geographically) similar to reporting cases  But databases replicate reciprocally and are writable  Redundancy provides fault tolerance and lowers maintenance downtime  Online upgrades possible London Chicago Tokyo

Mobile CRM / SFA Customer Relationship Management Sales Force Automation  Key Traits  Subset of data from server  Conflicts  Data realignment  Business Logic

Point of Sale  Key Traits  High latency low bandwidth network  Need low cost software and administration  Sync at scheduled times  Upload orders then remove

Regional Replication  Central headquarters data replicated to 1200 stores  Each store also replicates between two databases to provide a backup in case one fails  Database administrators are centralized

Regional Replication  Each ship contains a SQL Server database  When the ship docks, it replicates to the headquarters’ database  Replication was customized to perform additional processing when synchronizing SQL Server CustomizedReplicationLogic Transactional Replication Headquarters

Oracle Replication  SQL Server 2005 Transactional Replication  SQL Server synchronized with Oracle production line system in near real- time  Used to monitor manufacturing  Reporting: Quality control, status, and “what if” queries spread across multiple databases Manufacturing System Oracle10G SQL Server 2005 Transactional Replication

Top SQL Server Customers Transactional Replication  Runs many core trading applications on SQL Server  Replicates data to integrated multiple trading systems  15+ TB data in multiple systems, multiple hospitals  Replicates to meet HA requirements  Leading discount retailer, SQL Server in 1200 stores  Replicates data up and down between stores & HQ [Click logo for case-study]

Agenda  Replication Technology Positioning  SQL Server Replication Customer Scenarios  Upgrade Strategy  Key Improvements in SQL Server 2005 Replication

Upgrade: Replication  SQL Server 2005 Express Edition does not support Merge publishing  SQL Server 2000 MSDE did support it  SQL Server Express requires custom code or Windows Sync Manager to synchronize  If you run agents as DBO, now requires a job_login and job_password to be added to replication scripts

Upgrade: Merge Replication  Snapshot must be executed after setup  Upgrade order  Distributor, publisher, subscriber  Applications must be built on Visual Studio 2005 to use programming APIs  SQL Server 2005 subscribers outperform previous versions

Agenda  Replication Technology Positioning  SQL Server Replication Customer Scenarios  Upgrade Strategy  Key Improvements in SQL Server 2005 Replication

Database Schema Changes  One of the key problems when replicating data is accommodating schema changes without disrupting replication  SQL Server 2005 supports adding, dropping, and altering columns without stopping replication  Alter table alter column ( foo int )  Tables and other supported objects can also be added or dropped

Demo Schema Change (DDL) support

Database Scale-Out Requirements  Maximize website uptime  Database upgrades or failures shouldn’t bring down the application system  Database upgrades should be done without outages  Scale query workloads beyond what’s possible with a single database  Must be possible to add or remove databases without an outage while also maintaining consistency  SQL Server 2005 Peer-to-Peer Replication provides this capability  Workbench wizard allows easy replication setup

Distribution Agent Dist DB Logreader Agent Distribution Agent Dist DB Logreader Agent Distribution Agent Dist DB Logreader Agent LondonChicago Tokyo Peer-to-Peer Topology Transactional Replication

Demo Peer-to-Peer Replication

Replicating Large Databases  The key problem with replicating large databases is initializing the subscriber  This can be time-consuming and requires downtime to ensure consistency of the replicated database  SQL Server 2005 has support for using an existing database backup to perform the initial synchronization  Initialization with a backup  Changes that have occurred on the source system after the backup are tracked and synchronized automatically

Synchronizing Data Over the Web  Key customer drivers  Wireless increasingly pervasive  Virtual Private Network (VPN) too expensive or complicated for organizations -InternetUrlhttps://myserver.mydomain.com Publisher/Distributor Firewall Internet Information Services (IIS) Subscriber Replisapi.dll

Logical Records  Problem  Merge replication tracks changes row by row  An actual order is represented in the Customers, Orders, OrderDetails tables  Disconnection during sync results in incomplete order on server  Solution  Logical records create a logical consistency unit between tables  All three changes are sent or nothing at all

Business Logic  APIs to allow custom programming during the synchronization of changes  Allows developer to take custom actions on different states of data  Publisher insert, update, delete  Subscriber insert, update delete  Business Logic is registered on a per article basis

Merge Performance Comparison

Scalability Comparison

Monitoring Goals  Provide answers to common questions:  Why is the system slow?  How long until it catches up or finishes?  Where are the potential problems?

Monitoring

Oracle Publishing  Designed specifically for Oracle Publishers  v8+ on any operating system  Administered like SQL Server, from SQL Server  No Oracle-side software install necessary  Requires minimal knowledge of Oracle  Leverages existing SQL Server skills  Standard Transactional and Snapshot Publications SQL Server 2005Distributor Subscribers TransactionalReplication ORACLE

Many Other New Features  Transactional tracer token  Merge Article types  Tables, views, stored procedures, user- defined functions, and stored procedure execution  Resumable snapshot  Parallelism and contention improvements  Wizards simplified  Replication Management Objects (RMO) managed replication APIs  Security

Summary  Replication Technology Positioning  SQL Server Replication Customer Scenarios  Upgrade Strategy  Key Improvements in SQL Server 2005 Replication

For More Information  Books Online Replication Section  Merge Replication  White Paper  Oracle  Oracle Publishing Quick Start Whitepaper  SQL Server 2005 Oracle Replication Overview  Peer-to-Peer  Books Online

© 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.