1. Copyright © 2009 David C. Hay 1 Converting An Essential Entity/Relationship Model Into A Real Database Design Enterprise Data World David Hay Tampa, Florida April 6, 2009 David C. Hay Enterprise Data World Austin, Texas May 1, 2014 Essential Strategies International 13 Hilshire Grove Lane Houston, TX 77055 http://essentialstrategies.com

2. Copyright © 2014 David C. Hay 2 Different points of view... Data modeler End User Designer

3. Copyright © 2014 David C. Hay 3 Data Modeler’s Assignment...  Capture the language of the business  Do so in as flexible and robust a manner as possible. Data modeler

4. Copyright © 2014 David C. Hay 4 How to achieve flexibility and robustness...  Generalize entity classes, Each describes as large a population of phenomena as possible. For example, a Party is a Person or an Organization that is of interest to the company. Organization in this case can then have more specific sub-types, like Company, Government Agency, Household, etc.  Separate roles from the definitions of things. For example, an “Employee” is a Person who is employed by an Organization, such as a company. A “Vendor” is a Party who is a vendor in an Order.

5. Copyright © 2014 David C. Hay 5 How to achieve flexibility and robustness...  Put as much of the language of the business as instances of...type entity classes. This includes categories, like Activity Type and Product Type.  Treat nearly all attributes as being multi- valued, requiring a separate entity class. For example, Party Characteristic, with an intersect entity class Party Characteristic Value each instance of which contains a “Value” of a Party Characteristic for a particular Party.  Essential Data Model – General concepts Super-set of user views

6. Copyright © 2014 David C. Hay The End User’s view...  The end user, on the other hand deals with very concrete, particular things.  The user interface must reflect the way the user deals with things today.  The behavior of the system is an extension of the user’s behavior.  Ideally ‘e participated in the modeling and agreed with the overall concepts.  But those abstractions have little to do with today’s problems. 6 End User

7. Copyright © 2014 David C. Hay 7 Designer’s Assignment...  Designers may not be experienced with models this abstract.  This paper is intended to present some of the more basic steps required to convert an essential data model into a database design.  It turns out that abstract models are implemented using the same steps as not so abstract models.

8. Copyright © 2014 David C. Hay 8 Connections to System Users...  Conceptual Data Model – General concepts Super-set of user views  User Views – Concrete terms Habits and personal preferences  Database and Application Design True to conceptual model Accommodates technological limits Makes user views possible

9. Copyright © 2014 David C. Hay UML Alert!  Both the Essential Data Model and the Relational Database Design shown here use constrained versions of the Unified Modeling Language.  Translation: 9 UML SymbolEssential Symbol Design Symbol ClassEntity TypeTable AssociationRelationshipForeign Key Attribute Column InheritanceSub-type(resolved)

10. Copyright © 2014 David C. Hay 10 Four Steps to Design 1.Resolve sub-types. 2.Perform default database design 3.Design computed columns. 4.De-normalize as necessary. 5.Deal with those parameters.

11. Copyright © 2014 David C. Hay 11 Four Steps to Design 1.Perform default database design 2.Resolve sub-types. 3.Design computed columns. 4.De-normalize as necessary. 5.Deal with those parameters

12. Copyright © 2014 David C. Hay 12 The Default Database Design...  Each entity class becomes a table.  Each attribute becomes a column.  Each primary identifier becomes a primary key. Each component of the identifier is a reference to a column in the table.  Each role on the “many” end of a relationship becomes a foreign key, composed of pointers to the columns in the other table’s primary key.  If a relationship from table A to table B was part of a unique identifier, the columns in table A that are the foreign key implementation of that relationship become part of the primary key for Table A.

13. Copyright © 2014 David C. Hay {id} 13 An Entity/Relationship Diagram... Identifier s

14. Copyright © 2014 David C. Hay 14 The Default Conversion... Primary Keys Foreign Key Columns Foreign Keys

15. Copyright © 2014 David C. Hay 15 Four Steps to Design 1.Perform default database design 2.Resolve sub-types. 3.Design computed columns. 4.De-normalize as necessary. 5.Deal with those parameters

16. Copyright © 2014 David C. Hay 16 Resolve sub-types  V1: One table for the super-type. All attributes from all sub-types become columns. Cannot meaningfully enforce mandatory columns. Requires adding “type” column.  V2: One table for each sub-type. Attributes for super-type plus sub-type form columns for each sub-type table. Foreign keys for each relationship linked to super- type in each sub-type table.  V3: Combinations Most complex. Most “true”.

17. Copyright © 2014 David C. Hay 17 For example, this model... Implement Sub-types Inherit super-type attributes Collapse un- implement ed sub- types

18. Copyright © 2014 David C. Hay 18 Could be implemented, thus... “Department number is only required if “Organization Type Name” is “Department”.

19. Copyright © 2014 David C. Hay Criteria  Relative frequency of sub-type retrieval?  Who is going to do it? Different populations? Different timings? 19

20. Copyright © 2014 David C. Hay 20 Four Steps to Design 1.Perform default database design 2.Resolve sub-types. 3.Design computed columns. 4.De-normalize as necessary. 5.Deal with those parameters

21. Copyright © 2014 David C. Hay 21 Design Computed Columns...  Compute on input: if values are relatively stable, or if retrieval volume per day is significantly greater than update volume. Maintenance is required to keep values consistent.  Compute on output: if values are relatively dynamic, or if retrieval volume is relatively low. Additional maintenance is unnecessary.  Kinds of calculations: Simple: A*B+C Inference: INFER-THRU (,, ) Summation: SUM-THRU (,, )

22. Copyright © 2014 David C. Hay 22 For example, this model... /Price INFER-THRU (to buy, Product Type, Unit Price) /Value = Quantity * Price /Contract Value = SUM-THRU (composed of, Line Item, Value) /Total Sales to Date = SUM-THRU (bought via, Line Item, Value)

23. Copyright © 2014 David C. Hay 23 Could be implemented thus... Computed on input (and stored) Computed on query* * Note that translating the formula into, for example, a stored procedure, is left to the viewer.

24. Copyright © 2014 David C. Hay 24 Four Steps to Design 1.Perform default database design 2.Resolve sub-types. 3.Design computed columns. 4.De-normalize as necessary. 5.Deal with those parameters

25. Copyright © 2014 David C. Hay 25 De-normalize (three methods)... 1.Inherit reference values. 2.Split tables horizontally (by instance). 3.Split tables vertically (by column).

26. Copyright © 2014 David C. Hay 26 Inherit from reference tables... buyer in seller in This model...

27. Copyright © 2014 David C. Hay 27 Could be implemented as... CONTRACTS CONTRACT_NUMBER(PK) ISSUE_DATE TOTAL_VALUE BUYER_NAME SELLER_NAME LINE_ITEM CONTRACT_NUMBER(FK) PRODUCT_NAME QUANTITY COST STANDARD_PRICE VALUE

28. Copyright © 2014 David C. Hay Note:  When replicating values, recognize the maintenance required to keep them consistent.  Note that the paradigm of INFER-THRU and SUM-THRU already anticipated this.  If these are implemented as dynamic columns, maintenance is automatic.  If they are implemented as static copies, maintenance must be added. 28 Have you heard this before? Denormalization replicates computed fields

29. Copyright © 2014 David C. Hay 29 Split Horizontally (instances)...  For example, By Geographic Area  Some tables for North American customers  Some tables for European customers  Etc. By Customer Type, etc.  Some tables for corporate customers  Some tables for individual customers  Etc.  Note the problems that will arise if a significant number of customers (for example) fall into more than one category.

30. Copyright © 2014 David C. Hay 30 Split Vertically (columns)...  For example, People with customer attributes  Annual sales  Sales representative  Etc. People with employee attributes  Social security number  Employment date  Note that people with both kinds of attributes would appear redundantly in both tables.

31. Copyright © 2014 David C. Hay 31 NOTE...  De-normalization optimizes some operations at the expense of others.  Test the effects before making them permanent.  Document the rationale for the de- normalization.

32. Copyright © 2014 David C. Hay 32 Four Steps to Design 1.Perform default database design 2.Resolve sub-types. 3.Design computed columns. 4.De-normalize as necessary. 5.Deal with those parameters.

33. Copyright © 2014 David C. Hay 33 About those parameters...  Some entity classes (Party, for example) invariably have a lot of attributes.  And they change over time.  Their definitions change over time.  We need an alternative.  Define attributes as data.  Also called: Characteristics, Parameters, Variables, Etc.  Here’s an approach for Party, for example

34. Copyright © 2014 David C. Hay 34 Party parameters as “Characteristics”... PARTY CHARACTERISTIC: “Height” “Number of employees” “Regulatory target”, Etc. PARTY CHARACTERISTIC VALUE: “Height” of “Jerry Smith” has CHARACTERISTIC VALUE of “6.1” (feet)… according to “Jerry Smith”.

35. Copyright © 2014 David C. Hay 35 Party Characteristic Constraints... NOTE: the CONTINUOUS PARTY CHARACTERISTIC “Height” -- may only be used as a PARTY CHARACTERISTIC VALUE -- for a PARTY that is an example of the PARTY TYPE “Person”.

36. Copyright © 2014 David C. Hay 36 Designing those parameters...  While it is a powerful way of dealing with the complexity of data …  …the Parameter Model makes common manipulations harder, however.  Convert parameters that are... Relatively stable Not multi-valued (Over time?)  Do not convert parameters that are... Multi-valued Changeable over time and this must be reported.

37. Copyright © 2014 David C. Hay 37 For example, in this model...

38. Copyright © 2014 David C. Hay 38  These Characteristics:  Could be implemented as: Can be implemented thus... PEOPLE BIRTHDATE HEIGHT PARTY CHARACTERISTIC NameDescription(Party Type) BirthdateThe day the person appearedPerson HeightVertical distance Person Annual SalesAverage sales in a yearCompany Tax ID IRS tax identifierCompany COMPANIES ANNUAL SALES TAX IDENTIFICATION NUMBER Bad idea!

39. Copyright © 2014 David C. Hay 39 Four Steps to Design 1.Perform default database design 2.Resolve sub-types. 3.Design computed columns. 4.De-normalize as necessary. 5.About those parameters. 6.About those user views.

40. Copyright © 2014 David C. Hay 40 In summary: About Those User Views...  The database designer need only balance data model integrity with performance issues.  The application designer must take the data as organized in a database and present it reasonably to each particular end user.  This requires skill in understanding both the database and the underlying data model.

41. Copyright © 2014 David C. Hay 41 Questions... ?