Jerry Post Copyright © 2013 DATABASE Database Management Systems Chapter 11 Distributed Databases 1

Objectives  Why do you need a distributed database?  What are distributed databases?  How is data distributed with client/server systems?  Can a Web approach solve the data distribution issues?  How much data can you send to a client form?  How do you transfer data across diverse systems?  How will Sally’s employees access the database? 2

Distributed Databases  Definition  Advantages / Uses  Problems / Complications  Client-Server / SQL Server  Microsoft Access 3 Britain Germany France Italy SELECT Sales FROM Britain.Sales UNION SELECT Sales FROM France.Sales UNION SELECT Sales FROM Italy.Sales

Distributed Database Definition  Multiple independent databases  Each DBMS is a complete DBMS (engine, queries, locking, transactions, etc.)  Usually on different machines.  Usually in different locations.  Connected by a network.  Might be different environments  Hardware  Operating System  DBMS Software 4 Database Zeus Database Apollo Database Athena United States England France

Distributed Database Rules  C.J. Date  Rule 0: Transparency: the user should not know or care that the database is distributed.  Local autonomy.  No reliance on a central site.  Continuous operation.  Location independence.  Fragmentation independence (physical storage).  Replication independence.  Distributed query processing.  Distributed transaction management.  Hardware independence.  Operating system independence.  Network independence.  DBMS independence. 5

Distributed Features  Each database can continue to run even if portion fails.  Data and hardware can be moved without affecting operations or users.  Expanding operations.  Performance issues.  System expansion and upgrades.  Add new section without affecting others.  Upgrade hardware, network and DBMS. 6

Advantages and Applications  Business operations are often distributed  Work and data are segmented by department.  Work and data are segmented by geographical location.  Improved performance  Most updates and queries are performed locally.  Maintain local control and responsibility over data.  Can still combine data across the system.  Scalability and expansion  Add on, not replacement. 7 local transactions future expansion

Creating a Distributed Database  Design administration plan.  Choose hardware and DBMS vendor, and network.  Set up network and DBMS connections.  Choose locations for data.  Choose replication strategy.  Create backup plan and strategy.  Create local views and synonyms.  Perform stress test: loads and failures. 8

Network Transfer Rates  Drives: MB per sec. (with SSD or RAID)  LANs: MB per sec ( mbps).  WANs: MB per sec.  Faster is possible but expensive!  Goal is to minimize transmissions.  Each system must be capable of evaluating queries--preferably SQL.  Results depend heavily on how the system joins tables – 400 MB MB MB Disk drive LAN WAN

Distributed Query Processing  Example  NY: Customers: 1 M rows  LA: Production: 10 M rows  Chicago: Sales: 20 M rows  Query: List customers who bought blue products on March 1  Bad idea #1 Transfer all rows to Chicago Then JOIN and select.  Better idea #2 (probably) Transfer blue products from LA to Chicago  Better idea #3 Get sale items on March 1 Get blue products from LA Send C# to NY 10 Customers(C#, …) 1,000,000 NY Products(P#, Color…) 10,000,000 Sales(S#, C#, Sdate) 20,000,000 SaleItem(S#, P#,…) 50,000,000 Chicago LA P# sold on March 1 Blue P# sold on March 1 C# list from desired P# Matching Customer data

11 Data Replication: Publish/Subscribe Main Replica subscribe Changed data Published changes

Data Replication  Goals  Minimize transmissions  Improve performance  Support heavy multiuser access.  Problems  Updating copies Bulk transmissions Site unavailable  Concurrency Easier for two people to change the same data at the same time.  Decision support systems.  Data warehouse. 12 Britain: Customers & Sales France: Customers & Sales Spain: Customers & Sales Britain Britain: Customers & Sales France: Customers & Sales Spain: Customers & Sales Spain Update data. Market research & data corrections. Periodic updates

Concurrency and Locks  Each DBMS must maintain lock facility.  To update, each DBMS must utilize and recognize other lock mechanisms and return codes.  Each DBMS must have a deadlock resolution protocol that recognizes the distributed databases.  Random wait.  Optimistic updates.  Two-phase commit. 13 DBMS #1 Accounts Jones8898 DBMS #2 Accounts Jones3561 Transaction A Locked Waiting Transaction B Waiting Locked

Transactions & Two-Phase Commit  Two (or more) separate lock managers.  DBMS initiating update serves as the coordinator.  Two phases  Coordinator sends message and data to all machines to “get ready.”  Local machines save data in logs, verify update status and return message.  If all locals report OK, then coordinator writes log and instructs others to proceed. If any fail, it sends Rollback message. 14 Database 1 Initiate Transaction Database 2 Database 3 1. Prepare to commit. All agree? 2. Commit Lock tables. Save log. Update all tables.

Distributed Transaction Managers 15 Transaction Processing Monitor Transaction Manager Resource Manager Transaction Manager Resource Manager Transaction Manager Resource Manager DBMS The distributed transaction coordinator/transaction processing monitor handles the transaction decisions and coordinates across the participating systems.

Distributed Design Questions 16

Distributed Databases In Oracle  Database Links  Full database names.  CONNECT command.  Linking through synonyms.  CREATE SYNONYM …  Central control over permissions.  Linking through Views/queries.  CREATE VIEW AS …  Can assign local permissions.  Linking through stored procedures.  DELETE …  Strong control over actions. 17 Server database View Synonym: Employee Procedure: DELETE FROM Employee WHERE... user permissions User can only run procedure. No other access.

Client-Server 18 Server Clients Shared Database Front-end User Interface Older, but common for retail stores/checkout computers.

LAN File Server  Not a distributed database.  Data file stored on server.  Server is passive, appears as giant disk drive to PC.  PC processes all data.  Retrieves all needed data across the network.  Performance improvements.  Indexes are crucial.  Store some data on each PC (replication).  Store applications on PC (graphics & forms).  Convert to SQL-Server 19 File Server DBMS data file Application Shared Data SELECT Name, SaleDate FROM Customer INNER JOIN Sales ON Customer.C# = Sales.C# WHERE SaleDate BETWEEN #1-Mar-97# AND #9-Mar-97#; All data from all tables are read by PC, which performs JOIN and WHERE test. If available, reads index first.

LAN File Server: Slow 20 File Server CustIDName… 115Jenkins… 125Juarez... Order... MyFile.mdb Forms SELECT * FROM Customer WHERE City = “Sandy” DBMS software transferred. Application and query transferred. One row at a time transferred, until all rows are examined.

Client-Server Databases  One machine machine is dominant (server) and handles data for many clients.  Client machines handle front-end tasks and small data tables that are not shared. 21 File Server DBMS SQL Server Shared Data application SELECT... Send SQL statement. Return matching data.

ADO and Direct Connections 22 Database Server Visual Basic application DBMS transport ADO Client Computer DBMS transport Server Computer SELECT … Results The Database vendor provides its own data transport (e.g,. Oracle or SQL Server) installed on the server and the client. ADO provides a driver that connects your application to the transport services. ODBC can serve as the data transport if nothing else is available

Three-Tier Client-Server  Server Databases  Client front-end  Middle  Locate databases  Business rules  Program code 23 Client Middleware Database Servers Application. Front-end. User Interface. Databases. Transactions. Legacy applications. Database links. Business rules. Program code.

Database Independence on the Client 24 New DBMSOriginal DBMS Application ADO

Database Independence with Queries 25 SELECT SaleID, SaleDate, CustomerID, CustomerName FROM SaleCustomer SELECT SaleID, SaleDate, CustomerID, LastName || ‘, ‘ || FirstName AS CustomerName FROM Sale, Customer WHERE Sale.CustomerID=Customer.CustomerID SELECT SaleID, SaleDate, CustomerID, LastName + ‘, ‘ + FirstName AS CustomerName FROM Sale INNER JOIN Customer ON Sale.CustomerID = Customer.CustomerID Independent Application Query: works with any DBMS Saved Oracle Query Saved SQL Server Query

The Internet as Client-Server 26 Client Browser Server Web Server Router Internet HTML forms request Database Server SQL result page Data

HTML Limited Clients 27 My main page My text goes in paragraphs. Additional tags set boldface and Italic. Tables are more complicated and use a set of tags for rows and columns. First cell Second cell Next row Second column There are form tags to create input forms for collecting data. But you need CGI program code to convert and use the input data.

HTML Output 28

Web Server Database Fundamentals 29 Client/Browser Web Server HTML form 1 Data DBMS 2 Form.html Query Database Result Query Template + Code Program code Page = Template + Result Result Page Form CGI String 0 Request Server/Form.html HTML Form

Web Server Database Fundamentals 30 DBMS <asp:Label ID="PageTitleLabel" runat="server" … <asp:SqlDataSource ID="CustomerSqlDataSource“ DeleteCommand="DELETE FROM [Customer] … SelectCommand="SELECT [CustomerID], … UpdateCommand="UPDATE [Customer] SET… <asp:FormView ID="CustomersFormView" runat="server“… DataSourceID="CustomerSqlDataSource"> … Web Server SQL Data Data Object Customers CustomerID 1653 LastName FirstName … Jones Mary Save.PageTitle { font-weight: bold; font-size: larger; text-align: center; } CSS Style Sheet Web Browser

Database Example: Client Side 31 0 Request Server/Form.html 1 2 Server 3 Results Call Web page Initial form

Client-Server Data Transfer 32 Order Form Order Date Customer 12-Aug Jones, Martha Order ID 1015 What if there are 10,000 customers? How much time to load the combo box? How do you refresh/reload the combo box? Alternatives?

Latency 33 time Server Client Generate form Form received User delay Receive form data Transmission delay

Cloud Computing 34 Data Data is replicated to multiple, connected servers in the Internet cloud. Client requests are filled from the nearest available server, spreading the bandwidth and processor demands across the network.

Cloud Databases: Amazon S3 35 Web server HTML Page Developer User Amazon S3 Bucket Name Key, Object HTML, Code, Bucket +Key Developer opens Amazon S3 account and creates/names a bucket. Developer uploads content (objects) to bucket and assigns a key to each object. Developer writes Web server code with HTML page and background code that calls the S3 service with the bucket name and object key. Web page is constructed on request and delivered with the object.

Web Databases (and Cloud)  Many cloud systems are proprietary and not relational.  Designed to handle data that rarely changes—typically through bulk uploads.  So little fear of concurrency or transaction issues.  Microsoft Azure SQL is an exception.  Need to handle concurrency (and transactions/logs) in your own code.  Always use optimistic concurrency (not pessimistic locking).  See Chapter 7, but keep the original values when you read data and use it in the WHERE clause when you alter data to see if it has changed. 36