1. Chapter 6 Physical Database Design

2. Physical Design u The purpose of the physical design process is to translate the logical description of the data into technical specifications for storing and retrieving data u Goal: create a design that will provide adequate performance and insure database integrity, security, and recoverability u Decisions made in this phase have a major impact on data accessibility, response times, security, and user friendliness. u The purpose of the physical design process is to translate the logical description of the data into technical specifications for storing and retrieving data u Goal: create a design that will provide adequate performance and insure database integrity, security, and recoverability u Decisions made in this phase have a major impact on data accessibility, response times, security, and user friendliness.

3. Required Inputs u Normalized relations u Data volume and use estimates u Attribute definitions u Descriptions of where and when data are used u Expectations for response time, data security, backup, recovery, retention, and integrity u Description of chosen technology u Normalized relations u Data volume and use estimates u Attribute definitions u Descriptions of where and when data are used u Expectations for response time, data security, backup, recovery, retention, and integrity u Description of chosen technology

4. Determining volume and usage u Data volume statistics represent the size of the business calculated assuming business growth over a period of several yearscalculated assuming business growth over a period of several years u Usage is estimated from the timing of events, transaction volumes, and reporting and query activity. Less precise than volume statisticsLess precise than volume statistics u Data volume statistics represent the size of the business calculated assuming business growth over a period of several yearscalculated assuming business growth over a period of several years u Usage is estimated from the timing of events, transaction volumes, and reporting and query activity. Less precise than volume statisticsLess precise than volume statistics

5. Composite usage map (Pine Valley Furniture Company)

6. Physical Design Decisions u Specify the data type for each attribute from the logical data model minimize storage space and maximize integrityminimize storage space and maximize integrity u Specify physical records by grouping attributes from the logical data model u Specify the file organization technique to use for physical storage of data records u Specify indexes to optimize data retrieval u Specify query optimization strategies u Specify the data type for each attribute from the logical data model minimize storage space and maximize integrityminimize storage space and maximize integrity u Specify physical records by grouping attributes from the logical data model u Specify the file organization technique to use for physical storage of data records u Specify indexes to optimize data retrieval u Specify query optimization strategies

7. Data Format Data type selection goals minimize storageminimize storage represent all possible valuesrepresent all possible values –eliminate illegal values improve integrityimprove integrity support manipulationsupport manipulation Note: these have different relative importance Data type selection goals minimize storageminimize storage represent all possible valuesrepresent all possible values –eliminate illegal values improve integrityimprove integrity support manipulationsupport manipulation Note: these have different relative importance

8. Data format decisions (coding) u e.g., AH(Adams Hall), B(Buchanan), etc u implement by creating a look-up table u there is a trade-off in that you must create and store a second table and you must access this table to look up the code value u consider using when a field has a limited number of possible values, each of which occupies a relatively large amount of space, and the number of records is large and/or the number of record accesses is small u e.g., AH(Adams Hall), B(Buchanan), etc u implement by creating a look-up table u there is a trade-off in that you must create and store a second table and you must access this table to look up the code value u consider using when a field has a limited number of possible values, each of which occupies a relatively large amount of space, and the number of records is large and/or the number of record accesses is small

9. Example code-look-up table (Pine Valley Furniture Company)

10. Data Format decisions (integrity) Data integrity controls default valuedefault value Range controlRange control Null value controlNull value control Referential integrityReferential integrity Missing data substitute an estimatesubstitute an estimate report missing datareport missing data Sensitivity testingSensitivity testing Data integrity controls default valuedefault value Range controlRange control Null value controlNull value control Referential integrityReferential integrity Missing data substitute an estimatesubstitute an estimate report missing datareport missing data Sensitivity testingSensitivity testing

11. For example... u Suppose you were designing the age field in a student record at your university. What decisions would you make about: data typedata type integrity (range, default, null)integrity (range, default, null) How might your decision vary by other characteristics about the student such as degree sought?How might your decision vary by other characteristics about the student such as degree sought? u Suppose you were designing the age field in a student record at your university. What decisions would you make about: data typedata type integrity (range, default, null)integrity (range, default, null) How might your decision vary by other characteristics about the student such as degree sought?How might your decision vary by other characteristics about the student such as degree sought?

12. Attribute groupings u Physical Record: A group of fields stored in adjacent memory locations and retrieved together as a unit. u Page: The amount of data read or written in one I/O operation. u Blocking Factor: The number of physical records per page. u Physical Record: A group of fields stored in adjacent memory locations and retrieved together as a unit. u Page: The amount of data read or written in one I/O operation. u Blocking Factor: The number of physical records per page.

13. Database Access Model The goal in structuring physical records is to minimize performance bottlenecks resulting from disk accesses (accessing data from disk is slow compared to main memory)

14. Attribute grouping: Denormalization u Process of transforming normalized relations into denormalized physical record specifications may partition a relation into more than one physical recordmay partition a relation into more than one physical record may combine attributes from different relations into one physical recordmay combine attributes from different relations into one physical record u Process of transforming normalized relations into denormalized physical record specifications may partition a relation into more than one physical recordmay partition a relation into more than one physical record may combine attributes from different relations into one physical recordmay combine attributes from different relations into one physical record

15. Denormalization Involves a trade-off: 4 Reduced disk accesses and greater performance (due, for example, to fewer table joins) - But - - But - 8 Introduction of anomalies (and thus redundancies) that will necessitate extra data maintenance 8 increase chance of errors and force reprogramming when business rules change 8 may optimize certain tasks at the expense of others (if activities change, benefits may no longer exist) Involves a trade-off: 4 Reduced disk accesses and greater performance (due, for example, to fewer table joins) - But - - But - 8 Introduction of anomalies (and thus redundancies) that will necessitate extra data maintenance 8 increase chance of errors and force reprogramming when business rules change 8 may optimize certain tasks at the expense of others (if activities change, benefits may no longer exist)

16. Denormalization opportunities u 1:1 relationship u M:M associative entity with non-key attributes u reference data u 1:1 relationship u M:M associative entity with non-key attributes u reference data

17. A possible denormali- zation situation: reference data

18. More denormalization options u Horizontal Partitioning: Distributing the rows of a table into several separate files. u Vertical Partitioning: Distributing the columns of a table into several separate files. The primary key must be repeated in each file. The primary key must be repeated in each file. u Combination of both u Horizontal Partitioning: Distributing the rows of a table into several separate files. u Vertical Partitioning: Distributing the columns of a table into several separate files. The primary key must be repeated in each file. The primary key must be repeated in each file. u Combination of both

19. Partitioning u Advantages of Partitioning: Records used together are grouped togetherRecords used together are grouped together Each partition can be optimized for performanceEach partition can be optimized for performance Security and recoverySecurity and recovery Partitions stored on different disks: less contentionPartitions stored on different disks: less contention Parallel processing capabilityParallel processing capability u Disadvantages of Partitioning: Slower retrievals when across partitionsSlower retrievals when across partitions Complexity for application programmersComplexity for application programmers Anomalies and extra storage space requirements due to duplication of data across partitionsAnomalies and extra storage space requirements due to duplication of data across partitions u Advantages of Partitioning: Records used together are grouped togetherRecords used together are grouped together Each partition can be optimized for performanceEach partition can be optimized for performance Security and recoverySecurity and recovery Partitions stored on different disks: less contentionPartitions stored on different disks: less contention Parallel processing capabilityParallel processing capability u Disadvantages of Partitioning: Slower retrievals when across partitionsSlower retrievals when across partitions Complexity for application programmersComplexity for application programmers Anomalies and extra storage space requirements due to duplication of data across partitionsAnomalies and extra storage space requirements due to duplication of data across partitions

20. How about this.. Consider the following normalized relations: STORE(Store_Id, Region, Manager_Id, Square_Feet) EMPLOYEE(Emp_Id, Store_Id, Name, Address) DEPARTMENT(Dept#, Store_ID, Manager_Id, Sales_Goal) SCHEDULE(Dept#, Emp_Id, Date, hours) What opportunities might exist for denormalization? Consider the following normalized relations: STORE(Store_Id, Region, Manager_Id, Square_Feet) EMPLOYEE(Emp_Id, Store_Id, Name, Address) DEPARTMENT(Dept#, Store_ID, Manager_Id, Sales_Goal) SCHEDULE(Dept#, Emp_Id, Date, hours) What opportunities might exist for denormalization?

21. Physical Files u Physical File: A file as stored on disk u Constructs to link two pieces of data: Sequential storageSequential storage PointersPointers u File Organization: How the files are arranged on the disk (more on this later) u Access Method: How the data can be retrieved based on the file organization Relative - data accessed as an offset from the most recently referenced point in secondary memoryRelative - data accessed as an offset from the most recently referenced point in secondary memory Direct - data accessed as a result of a calculation to generate the beginning address of a recordDirect - data accessed as a result of a calculation to generate the beginning address of a record u Physical File: A file as stored on disk u Constructs to link two pieces of data: Sequential storageSequential storage PointersPointers u File Organization: How the files are arranged on the disk (more on this later) u Access Method: How the data can be retrieved based on the file organization Relative - data accessed as an offset from the most recently referenced point in secondary memoryRelative - data accessed as an offset from the most recently referenced point in secondary memory Direct - data accessed as a result of a calculation to generate the beginning address of a recordDirect - data accessed as a result of a calculation to generate the beginning address of a record

22. File Organizations u A technique for physically arranging the records of a file on secondary storage devices. u Goals in selecting: (trade-offs exist, of course) Fast data retrievalFast data retrieval High throughput for input and maintenanceHigh throughput for input and maintenance Efficient use of storage spaceEfficient use of storage space Protection from failures or data lossProtection from failures or data loss Minimal need for reorganizationMinimal need for reorganization Accommodation for growthAccommodation for growth Security from unauthorized useSecurity from unauthorized use u A technique for physically arranging the records of a file on secondary storage devices. u Goals in selecting: (trade-offs exist, of course) Fast data retrievalFast data retrieval High throughput for input and maintenanceHigh throughput for input and maintenance Efficient use of storage spaceEfficient use of storage space Protection from failures or data lossProtection from failures or data loss Minimal need for reorganizationMinimal need for reorganization Accommodation for growthAccommodation for growth Security from unauthorized useSecurity from unauthorized use

23. File Organizations u Sequential u Indexed Indexed Sequential Indexed Nonsequential u Hashed (also called Direct) u See Table 6-3 for comparison u Sequential u Indexed Indexed Sequential Indexed Nonsequential u Hashed (also called Direct) u See Table 6-3 for comparison

24. Sequential File Organization u Records of the file are stored in sequence by the primary key field values

25. Comparisons of file organizations: (a) Sequential

26. Sequential Retrieval u Consider a file of 10,000 records each occupying 1 page u Queries that require processing all records will require 10,000 accesses u e.g., Find all items of type 'E' u Many disk accesses are wasted if few records meet the condition u However, very effective if most or all records will be accessed (e.g., payroll) u Consider a file of 10,000 records each occupying 1 page u Queries that require processing all records will require 10,000 accesses u e.g., Find all items of type 'E' u Many disk accesses are wasted if few records meet the condition u However, very effective if most or all records will be accessed (e.g., payroll)

27. Indexed File Organization u Index concept is like index in a book u Indexed-sequential file organization: The records are stored sequentially by primary key values and there is an index built on the primary key field (and possibly indexes built on other fields, also) u Index concept is like index in a book u Indexed-sequential file organization: The records are stored sequentially by primary key values and there is an index built on the primary key field (and possibly indexes built on other fields, also)

28. (b) Indexed

29. Hashed File Organization u Hashing Algorithm: Converts a primary key value into a record address u Division-remainder method is common hashing algorithm (more to come on this) u Hashing Algorithm: Converts a primary key value into a record address u Division-remainder method is common hashing algorithm (more to come on this)

30. (c ) Hashed

31. Hashing u A technique for reducing disk accesses for direct access u Avoids an index u Number of accesses per record can be close to one u The hash field is converted to a hash address by a hash function u A technique for reducing disk accesses for direct access u Avoids an index u Number of accesses per record can be close to one u The hash field is converted to a hash address by a hash function

32. Hashing

33. Shortcomings of Hashing u Different hash fields may convert to the same hash address these are called Synonymsthese are called Synonyms store the colliding record in an overflow areastore the colliding record in an overflow area u Long synonym chains degrade performance u There can be only one hash field per record u The file can no longer be processed sequentially u Different hash fields may convert to the same hash address these are called Synonymsthese are called Synonyms store the colliding record in an overflow areastore the colliding record in an overflow area u Long synonym chains degrade performance u There can be only one hash field per record u The file can no longer be processed sequentially

34. Clustering u In some relational DBMS, related records from different tables that are often retrieved together can be stored close together on disk u Because the related records are stored close to one another on the physical disk, less time is needed to retrieve the data E.g., Customer data and Order data may frequently be retrieved togetherE.g., Customer data and Order data may frequently be retrieved together u Can require substantial maintenance if the clustered data changes frequently u In some relational DBMS, related records from different tables that are often retrieved together can be stored close together on disk u Because the related records are stored close to one another on the physical disk, less time is needed to retrieve the data E.g., Customer data and Order data may frequently be retrieved togetherE.g., Customer data and Order data may frequently be retrieved together u Can require substantial maintenance if the clustered data changes frequently

35. Indexing u An index is a table file that is used to determine the location of rows in another file that satisfy some condition

36. Querying with an Index u Read the index into memory u Search the index to find records meeting the condition u Access only those records containing required data u Disk accesses are substantially reduced when the query involves few records u Read the index into memory u Search the index to find records meeting the condition u Access only those records containing required data u Disk accesses are substantially reduced when the query involves few records

37. Maintaining an Index u Adding a record requires at least two disk accesses: Update the file Update the index u Trade-off: 4Faster queries 8Slower maintenance (additions, deletions, and updates of records) Thus, more static databases benefit more overall u Adding a record requires at least two disk accesses: Update the file Update the index u Trade-off: 4Faster queries 8Slower maintenance (additions, deletions, and updates of records) Thus, more static databases benefit more overall

38. Rules of Thumb for Using Indexes 1. Indexes are most useful on larger tables 2. Index the primary key of each table (may be automatic, as in Access) 3. Indexes are useful on search fields (WHERE) 4. Indexes are also useful on fields used for sorting (ORDER BY) and categorizing (GROUP BY) 5. Most useful to index on a field when there are many different values for that field 1. Indexes are most useful on larger tables 2. Index the primary key of each table (may be automatic, as in Access) 3. Indexes are useful on search fields (WHERE) 4. Indexes are also useful on fields used for sorting (ORDER BY) and categorizing (GROUP BY) 5. Most useful to index on a field when there are many different values for that field

39. Rules of Thumb for Using Indexes 6. Find out the limits placed on indexing by your DBMS (Access allows 32 indexes per table, and no index may contain more than 10 fields) 7. Depending on the DBMS, null values may not be referenced from an index (thus, rows with a null value in the field that is indexed may not be found by a search using the index) 6. Find out the limits placed on indexing by your DBMS (Access allows 32 indexes per table, and no index may contain more than 10 fields) 7. Depending on the DBMS, null values may not be referenced from an index (thus, rows with a null value in the field that is indexed may not be found by a search using the index)

40. Rules for Adding Derived Columns u Use when aggregate values are regularly retrieved. u Use when aggregate values are costly to calculate. u Permit updating only of source data. u Create triggers to cascade changes from source data. u Use when aggregate values are regularly retrieved. u Use when aggregate values are costly to calculate. u Permit updating only of source data. u Create triggers to cascade changes from source data.

41. One Other Rule of Thumb for Increasing Performance u Consider contriving a shorter field or selecting another candidate key to substitute for a long, multi-field primary key (and all associated foreign keys)

42. Query Optimizer Factors u Type of Query Highly selective. All or most of the records of a file. u Unique fields u Size of files u Indexes u Join Method Nested-Loop Merge-Scan (Both files must be ordered or indexed on the join columns.) u Type of Query Highly selective. All or most of the records of a file. u Unique fields u Size of files u Indexes u Join Method Nested-Loop Merge-Scan (Both files must be ordered or indexed on the join columns.)

43. More practice u Draw a composite usage map for the following: PERSON(person_ID, name, address, DOB)PERSON(person_ID, name, address, DOB) PATIENT(PA_person_ID, Contact)PATIENT(PA_person_ID, Contact) PHYSICIAN(PH_person_ID, specialty)PHYSICIAN(PH_person_ID, specialty) PERFORMANCE(PA_person_ID, PH_person_ID, Treatment#, Treatment_date, Treatment_time)PERFORMANCE(PA_person_ID, PH_person_ID, Treatment#, Treatment_date, Treatment_time) CONSUMPTION(PA_person_ID, Item#, Date, Quantity)CONSUMPTION(PA_person_ID, Item#, Date, Quantity) ITEM(Item#, Description) ITEM(Item#, Description) u Make recommendations about denormalizing, partitioning, file organization, and indexing u Draw a composite usage map for the following: PERSON(person_ID, name, address, DOB)PERSON(person_ID, name, address, DOB) PATIENT(PA_person_ID, Contact)PATIENT(PA_person_ID, Contact) PHYSICIAN(PH_person_ID, specialty)PHYSICIAN(PH_person_ID, specialty) PERFORMANCE(PA_person_ID, PH_person_ID, Treatment#, Treatment_date, Treatment_time)PERFORMANCE(PA_person_ID, PH_person_ID, Treatment#, Treatment_date, Treatment_time) CONSUMPTION(PA_person_ID, Item#, Date, Quantity)CONSUMPTION(PA_person_ID, Item#, Date, Quantity) ITEM(Item#, Description) ITEM(Item#, Description) u Make recommendations about denormalizing, partitioning, file organization, and indexing

44. RAID u Redundant Arrays of Inexpensive Disks u Exploits economies of scale of disk manufacturing for the computer market u Can give greater security u Increases fault tolerance of systems u Not a replacement for regular backup u Redundant Arrays of Inexpensive Disks u Exploits economies of scale of disk manufacturing for the computer market u Can give greater security u Increases fault tolerance of systems u Not a replacement for regular backup

45. RAID u The operating system sees a set of physical drives as one logical drive u Data are distributed across physical drives u All levels, except 0, have data redundancy or error-correction features u Parity codes or redundant data are used for data recovery u The operating system sees a set of physical drives as one logical drive u Data are distributed across physical drives u All levels, except 0, have data redundancy or error-correction features u Parity codes or redundant data are used for data recovery

46. Mirroring u Write Identical copies of file are written to each drive in arrayIdentical copies of file are written to each drive in array u Read Alternate pages are read simultaneously from each driveAlternate pages are read simultaneously from each drive Pages put together in memoryPages put together in memory Access time is reduced by approximately the number of disks in the arrayAccess time is reduced by approximately the number of disks in the array u Read error Read required page from another driveRead required page from another drive u Tradeoffs 4Provides data security 4Reduces access time 8Uses more disk space u Write Identical copies of file are written to each drive in arrayIdentical copies of file are written to each drive in array u Read Alternate pages are read simultaneously from each driveAlternate pages are read simultaneously from each drive Pages put together in memoryPages put together in memory Access time is reduced by approximately the number of disks in the arrayAccess time is reduced by approximately the number of disks in the array u Read error Read required page from another driveRead required page from another drive u Tradeoffs 4Provides data security 4Reduces access time 8Uses more disk space

47. Mirroring Complete Data Set No parity

48. Striping u Three drive model u Write Half of file to first driveHalf of file to first drive Half of file to second driveHalf of file to second drive Parity bit to third driveParity bit to third drive u Read Portions from each drive are put together in memoryPortions from each drive are put together in memory u Read error Lost bits are reconstructed from third drive’s parity dataLost bits are reconstructed from third drive’s parity data u Tradeoffs 4Provides data security 4Uses less storage space than mirroring 8Not as fast as mirroring u Three drive model u Write Half of file to first driveHalf of file to first drive Half of file to second driveHalf of file to second drive Parity bit to third driveParity bit to third drive u Read Portions from each drive are put together in memoryPortions from each drive are put together in memory u Read error Lost bits are reconstructed from third drive’s parity dataLost bits are reconstructed from third drive’s parity data u Tradeoffs 4Provides data security 4Uses less storage space than mirroring 8Not as fast as mirroring

49. Striping One-Half Data Set Parity Codes

50. Database Architectures u Hierarchical u Network u Relational u Object-oriented u Multidimensional u Hierarchical u Network u Relational u Object-oriented u Multidimensional

51. Hierarchical models u Type of logical database model in which data is organized in a tree structure. u Records are divided into segments which are linked using pointers. u Parent and child data are stored together. u Type of logical database model in which data is organized in a tree structure. u Records are divided into segments which are linked using pointers. u Parent and child data are stored together.

52. Sample hierarchical model ROOT/PARENT FIRST CHILD 2nd CHILD RatingsSalary Compensation Job Assignments PensionInsuranceHealth Benefits Employee

53. Network model u Extends the hierarchical model by allowing a child to have zero, one, or many parents u No real theoretical base u Never widely used u Extends the hierarchical model by allowing a child to have zero, one, or many parents u No real theoretical base u Never widely used STUDENT A STUDENT B STUDENT C CLASS 1 CLASS 2

54. Comparing the alternatives ** OODBS are marketed as easy to use, but the transition from RDBMS is difficult