CMS Advanced Database and Client-Server Applications Distributed Databases slides by Martin Beer and Paul Crowther Connolly and Begg Chapter 22 (3 rd and 4 th edn) Chapter 19 (2 nd edn)

3 Concepts Distributed Database A logically interrelated collection of shared data (and a description of this data), physically distributed over a computer network. Distributed DBMS Software system that permits the management of the distributed database and makes the distribution transparent to users.

4 Concepts Collection of logically-related shared data. Data split into fragments. Fragments may be replicated. Fragments/replicas allocated to sites. Sites linked by a communications network. Data at each site is under control of a DBMS. DBMSs handle local applications autonomously. Each DBMS participates in at least one global application.

DBMS - 1 Global Schema Distributed DBMS DBMS - 2DBMS - 3DBMS - n Local User Global user Local User Heterogeneous Distributed Database Environment

20 Functions of a DDBMS Expect DDBMS to have at least the functionality of a DBMS. Also to have following functionality: – Extended communication services. – Extended Data Dictionary. – Distributed query processing. – Extended concurrency control. – Extended recovery services.

27 Distributed Database Design Three key issues: Fragmentation Relation may be divided into a number of sub- relations, which are then distributed. Allocation Each fragment is stored at site with "optimal" distribution. Replication Copy of fragment may be maintained at several sites.

28 Fragmentation Definition and allocation of fragments carried out strategically to achieve: – Locality of Reference – Improved Reliability and Availability – Improved Performance – Balanced Storage Capacities and Costs – Minimal Communication Costs. Involves analyzing most important applications, based on quantitative/qualitative information.

29 Fragmentation Quantitative information may include: – frequency with which an application is run; – site from which an application is run; – performance criteria for transactions and applications. Qualitative information may include transactions that are executed by application, type of access (read or write), and predicates of read operations.

31 Data Allocation Four alternative strategies regarding placement of data: Centralized Consists of single database and DBMS stored at one site with users distributed across the network. Partitioned Database partitioned into disjoint fragments, each fragment assigned to one site Complete Replication Consists of maintaining complete copy of database at each site. Selective Replication Combination of partitioning, replication, and centralization.

34 Why Fragment? Usage – Applications work with views rather than entire relations. Efficiency – Data is stored close to where it is most frequently used. – Data that is not needed by local applications is not stored Parallelism – With fragments as unit of distribution, transaction can be divided into several sub-queries that operate on fragments. Security – Data not required by local applications is not stored and so not available to unauthorized users. Disadvantages – Performance – Integrity.

38 Correctness of Fragmentation Completeness If relation (TABLE) R is decomposed into fragments R 1, R 2,... R n, each data item that can be found in R must appear in at least one fragment. Reconstruction Must be possible to define a relational operation that will reconstruct R from the fragments. Reconstruction for horizontal fragmentation is Union operation and Join for vertical. Disjointness If data item d i appears in fragment R i, then it should not appear in any other fragment. Exception: vertical fragmentation, where primary key attributes must be repeated to allow reconstruction. For horizontal fragmentation, data item is a tuple (record or row) For vertical fragmentation, data item is an attribute.

40 Types of Fragmentation Four types of fragmentation: – Horizontal – Vertical – Mixed – Derived. Other possibility is no fragmentation: – If relation (table) is small and not updated frequently, may be better not to fragment relation.

41 Horizontal and Vertical Fragmentation records, rows or tuples columns or attributes

43 Horizontal Fragmentation Consists of a subset of the tuples of a relation. Defined using Selection operation of relational algebra:  p (R) For example: P 1 =  type='House' (Property_for_Rent) P 2 =  type='Flat' (Property_for_Rent) Select * from Property_for_Rent where type = 'House'; Select * from Property_for_Rent where type = 'Flat';

45 Vertical Fragmentation Consists of a subset of attributes of a relation. Defined using Projection operation of relational algebra:  a1,...,an (R) For example: S 1 =  sno, position, sex, dob, salary, nin (Staff) S 2 =  sno, fname, lname, address, tel_no, bno (Staff) (Select sno, poistion, sex, dob, sal, nin from Staff;) Determined by establishing affinity of one attribute to another.

Mixed Fragmentation Head office only requires data about sales personnel and their sales. Local offices need contact information of all staff Personnel contact (Sheffield) Personnel contact (Barnsley) Sales Commissions (Head Office) ID Name NIN phone.... Dark areas show attributes and records not stored

46 Mixed Fragmentation Consists of a horizontal fragment that is vertically fragmented, or a vertical fragment that is horizontally fragmented. Defined using Selection and Projection operations of relational algebra:  p (  a1,...,an (R)) rows or  a1,...,an (ó p (R)) columns

48 Derived Horizontal Fragmentation A horizontal fragment that is based on horizontal fragmentation of a parent relation. Ensures that fragments that are frequently joined together are at same site. Defined using Semi-Join operation of relational algebra: R i = R F S i,1  i  w A semi-join does a join, then grabs the records from one table (R) which correspond with the second table (S). The result is the Derived Horizontal Fragment.

Sales Personnel (Sheffield) Sales Personnel (Barnsley) Property Property records are stored at the same location as the sales person who is assigned to them Derived Fragmentation Records are on the Sheffield server Records are on the Barnsley server Records distributed to either Barnsley or Sheffield server

49 Example - Derived Horizontal Fragmentation S 3 =  bno=B3 (Staff) S 4 =  bno=B5 (Staff) S 5 =  bno=B7 (Staff) Could use derived fragmentation for Property so Property_for_Rent is fragmented in the same way as the staff: P i = Property_for_Rent bno S i,3  i  5 There is a join between Property_for_Rent and Staff using the branch number (bno), so we replicate the horizontal partitioning of staff in Property_for_Rent Staff relation (table) is horizontally fragmented by branch (bno)

51 Transparencies in a DDBMS Distribution Transparency – Fragmentation Transparency – Location Transparency – Replication Transparency – Local Mapping Transparency – Naming Transparency

52 Transparencies in a DDBMS Transaction Transparency – Concurrency Transparency – Failure Transparency Performance Transparency DBMS Transparency

Distributed Transaction Management Objectives do not change just because the database is distributed Our distributed system still has a local DBMS with all the functionality of previously Added components to handle the global issues

Distributed Commit Protocol Simple Commit/rollback inadequate May be multiple sub-transactions involved in a transaction Problem is one of co-ordination

Transaction management must be based on a system wide COMMIT or ROLLBACK There is a transaction co-ordinator nominated Consistency maintained through use of 2 phase commit

Co-ordinator begin_commit –write begin_commit to log –send ‘prepare’ message to all participants –wait –if all participants replied ‘ready’ write commit to log –send ‘commit’ to all participants Participants ready_commit –write ready_commit to log –send ‘ready’ message to co-ordinator –wait Initial Waiting Decided Waiting Initial

–if all participants replied, write end_transaction to log –write commit to log –commit –send ack Completed What does this protocol look like in event of failure ?

Failure any If any participant sent an abort message during the commit-request phase: The coordinator sends an rollback message to all the participants. Each participant undoes the transaction using the undo log, and releases the resources and locks held during the transaction. Each participant sends an acknowledgement to the coordinator. The coordinator completes the rollback of the transaction when acknowledgements have been received.

Termination protocol Necessary in event of not receiving expected messages - in effect a timeout Co-ordinator –in wait state, can’t commit but can abort –in decided state, keeps on sending the global decision – is blocked Participant –in initial state if does not receive a ‘ready’ can abort –in waiting state has already made its own suggestion so can’t go forwards or backwards. Blocked. – there are alternative protocols such as 3PC

2PC Problem When the coordinator has sent commit to the participants, it will block until all participants have acknowledged. Thus, if a participant is permanently down, the coordinator will block indefinitely. When a participant has sent an agreement message to the coordinator, it will block until a commit or rollback is received. If the coordinator is permanently down, the participants will block indefinitely.