Overview – Chapter 11 SQL 710 Overview of Replication SQL Server Replication Types SQL Server Replication Agents SQL 710

Need for Distributed Data Reasons for distributing data: Redundancy (eg if power out at one site data still available at other sites) Site independence Reduced networking costs Improved response time Optimize applications by separating update-intensive online transaction processing (OLTP) from read-intensive online analytical processing (OLAP) applications such as data warehouses SQL 710

Replication Data can be distributed can be distributed by replicating (copying) data onto different servers Replication supports a distributed data environment that: Improves response time Allows sites to operate independently Separates online transaction processing (OLTP) from online analytical processing (OLAP) applications SQL 710

Replication Terminology PUBLISHER: server containing data to be replicated ARTICLE: database object such as a table (or filtered partitions of table), view, user-defined function or stored procedure, to be replicated PUBLICATION: group of articles DISTRIBUTOR: transfers a publication SUBSCRIBER: server that receives replicated data SUBSCRIPTION: information required to request a publication from publisher AGENT: oversees replication process SQL 710

Replication Model SQL 710 PUBLISHER SUBSCRIBER Maintains source databases Makes data available for replication Detects and sends changes of all published data to Distributor Holds copy of data Receives copy of changes to data May even update data and replicate it back to publisher (if an updating subscriber) SQL 710 DISTRIBUTOR Has distribution database which stores metadata, history data and transactions May forward changes to subscribers

Push Subscription initiated by publisher (publisher “pushes” data to subscribers) many subscribers can be set up for each publication normally used in applications that must send changes to subscribers whenever and as soon as they occur best for publications that require higher security and where higher processor overhead at distributor does not affect performance. SQL 710

Pull Subscription SQL 710 initiated by subscriber publication must be enabled to allow pull subscriptions subscriber must be registered to subscribe or publication must permit anonymous subscriptions best for publications that may have low security and support a high number of subscribers, such as subscribers that use the internet SQL 710

Filtering Data Subsets of a table can also be an article. This is known as filtering data. Two methods: Vertical filtering: contains a subset of the columns in a table (eg. non-sensitive data like name and extension for all rows of employee table) Horizontal Filtering : contains a subset of rows in a table (eg. all rows from employee table where location = ‘Toronto’) SQL 710

Benefits of Filtering Data Filtering data helps to avoid conflicts when multiple sites are allowed to update data Minimizes amount of data sent over network Reduces amount of storage required by subscriber Provides better security; subscriber sees only data that they need to see Customizes publications based on individual subscriber requirements SQL 710

SQL Server Types of Replication SNAPSHOT: replicates entire set of data TRANSACTIONAL: starts from a snapshot replication and as time passes replicates only the changed data MERGE: starts from a snapshot replication and then each database is updated separately until databases are compared and differences between them are transferred SQL 710

Replication Method SQL 710 Type of Replication method is determined by requirements of business application and site autonomy SQL 710

Transactions A transaction is a sequence of operations performed as a single logical unit of work Programmers are responsible for starting and ending transactions at points that enforce the logical consistency of the data as determined by the business application Programmer must define sequence of data modifications within a transaction that leave the data in a consistent state relative to the organization's business rules Transaction is made permanent by COMMIT statement or rolled back by ROLLBACK SQL 710

ACID properties for Transaction Atomic: all of a transaction’s data modifications are performed or none of them are performed Consistent: a completed transaction must leave all data in a consistent, logically correct state Isolation: a transaction reads data in the state it was in before another concurrent transaction modified it (without yet committing the transaction). Concurrent modifications that are in progress do not affect the transaction Durable: changes made by a transaction will persist after a commit is acknowledged SQL 710

Transaction Integrity Integrity of a transaction is enforced by: Locking that preserves transaction isolation Logging ensures transaction durability - even if HW, OS or SQL Server fail, transaction logs are used to automatically roll back any uncompleted transactions Transaction management features enforce transaction atomicity and consistency - after a transaction starts, it must be successfully completed, or SQL Server undoes all of data modifications made since the transaction started SQL 710

Transaction Processing BEGIN TRANSACTION: marks start of a transaction COMMIT TRANSACTION: saves all changes made since the start of a transaction, guarantees that all of the transaction's modifications are made permanent and frees resources, such as locks ROLLBACK: erases all changes made since start of transaction (Statements not bound in a transaction are automatically committed when statement executes without an error) SQL 710

Snapshot Replication SQL 710 A copy of the entire current data (changed and unchanged) from a source server replaces data at a destination server on a scheduled basis or on demand. SQL 710

Snapshot Replication Criteria data changes are infrequent but may be substantial small amounts of data involved, since entire set of data must be copied have read-only data for subscribers, i.e. data is only being changed at publisher high latency (lag time between change at publisher and subscriber) is acceptable, because data is typically refreshed only periodically subscribers require site autonomy SQL 710

Snapshot Replication Scenario Sales employees in sales offices throughout country require product price list. Prices are only updated after business hours. Servers in sales offices can obtain snapshot of product price list at beginning of day and employees use data in a static mode throughout the day. SQL 710

Transactional Replication Starts from a snapshot replication Subsequently only changed data is distributed Sequence of transactions is maintained Because there is only one location where data is maintained, conflicts between data cannot arise SQL 710

Transactional Replication Criteria Useful in an environment where subscribers must receive data changes as they occur with minimal latency Provides option for subscribers to update data at the publisher immediately, by using a two-phase commit protocol on a deferred basis or by using queued updates, or a combination of both SQL 710

Transactional Replication Scenario Sales are recorded centrally at headquarters as they occur during the day and inventory is reduced. Field offices need to know an accurate inventory count so that they won’t sell what has already been sold! To do this inventory recorded at all sites must be current – maintained by distributing and applying transactions at each site SQL 710

Merge Replication SQL 710 Starts with snapshot replication initially Allows multiple sites to make changes to data independently of one another and then update the publisher periodically Conflicts can happen but are resolved as specified Transaction ordering is not maintained; merge replication only synchronizes copies Note: vertical filtering not supported by merge replication SQL 710

Merge Replication Criteria multiple subscribers need to update data at various times and propagate those changes to publisher and other subscribers subscribers need to receive data, make changes offline, and synchronize changes later with publisher and other subscribers latency requirement is either high or low site autonomy is critical SQL 710

Merge Replication Scenario Sales employees report although they enjoy having a copy of the inventory, what they’d really like is to be able to record sales information on the spot. To do this, they need to reduce inventory by the sales amount and bill the customer SQL 710

Merge Conflict Resolution Merge Agent tracks every update to a row Merge Agent evaluates both the arriving and current data values, and any conflicts between new and old values are automatically resolved based on assigned priorities Data values are replicated to other sites only when synchronization occurs; synchronizations can occur minutes, days, or even weeks apart SQL 710

Merge Replication Criteria Merge Replication good when: Multiple subscribers need to update data at various times and propagate changes to the publisher and to other subscribers Subscribers need to receive data, make changes offline, and later synchronize changes with the publisher and other subscribers Conflicts between data at multiple sites is not common. If conflicts do occur there must be a method to resolve them. SQL 710

Replication Planning SQL 710 Maximize data consistency and minimize demands on network resources by considering: If, when and by whom replicated data needs to be updated Needs regarding consistency, autonomy, and latency of data Environment including: data characteristics, users, technical infrastructure, network and security SQL 710

SQL SERVER REPLICATION AGENTS Snapshot Agent: prepares initial snapshots and stores then on the distributor Distribution Agent: moves snapshot and transaction data to subscribers Log Reader Agent: copies data from transaction log to the distribution database Merge Agent: merges changes from multiple sites Queue Reader Agent: applies changes from a queue and applies them to multiple subscribers SQL 710