1. Logical Database Design Chapter 4 G. Green 1

2. Agenda Evolution of Data Models Chapter 1 pgs 25 – 28 Chapter 9 pgs 409 – 418 Relational Database Model Transforming ERDs into Relations Referential Integrity Normalization G. Green 2

3. The Evolution of Data Models Hierarchical Network Relational Object oriented Multi-dimensional NoSQL G. Green 3

4. Relational Data Model Developed by Codd (IBM) in 1970 Represents data in the form of tables Based on mathematical theory 3 Components: relational database structure relational rules (integrity) relational operations (manipulation) G. Green 4

5. Relational Data Model Advantages Structural independence Improved conceptual simplicity Easier database design, implementation, management, and use Ad hoc query capability Mathematical foundation Disadvantages Hardware and system software overhead Can facilitate poor design and implementation May promote “islands of information” problems G. Green 5

6. Relational Theory Components 1.Relational Database Structures 2.Rules of Relations 3.Relational Operators G. Green 6

7. 1. Relational Database Structure  Relations, Tuples, Attributes › Tables, Rows, Columns › Files, Records, Fields  Primary Key must be designated  Foreign Keys must be designated for relationships G. Green 7 CLASS TABLE ROOM TABLE

8. 2. Rules of Relations  Relation names must be unique  Entries in columns are atomic (single valued)  Entries in column are from same domain  Each row is unique  Ordering is insignificant G. Green 8 CLASS Table

9. 2. Rules of Relations, con’t...  Data in tables should be added, updated, and deleted without errors › avoid inconsistency ==> referential integrity  insertion  update  Deletion › avoid anomalies ==> normalization  insertion  update  deletion G. Green 9

10. 3. Relational Operators Relational Algebra UNION (+) INTERSECTION DIFFERENCE (-) PRODUCT (x) SELECT (tuples) PROJECT (attributes) JOIN (PRODUCT, SELECT, PROJECT) G. Green 10 * Diagram adapted from Hyperion presentation, http://infolab.stanford.edu/infoseminar/Archive/FallY99/rus sakovskii-slides/sld001.htm * *

11. Converting ERD to Relational Model Represent entities as relations Represent relationships as either: foreign keys in relations new relations Provide sample data Normalize relations G. Green 11

12. Representing Entities as Tables Each entity converted to a relational table attributes become columns primary key must be designated regular entities have atomic keys associative entities have composite keys subtype entities have same key as supertype example entity instances are rows of table G. Green 12

13. ERD Example Problem Revisited A company sells products to customers Customer requests generate orders Orders may consist of many ordered products Products may be contained on many orders, or no orders at all G. Green 13

14. 14 ERD Example Converted to Tables

15. Representing Relationships  1:1 › merge attributes into single table; › OR create foreign key (FK) in either relation  1:M › create foreign key (FK) in relation on “many” side of relationship  M:M › should’ve been eliminated on ERD!!! › create new relation with PKs of related entities as (1) concatenated PK, and (2) FKs in new relation G. Green 15

16. Referential Integrity Maintains consistency between data in related tables For every value of a foreign key there must be an existing primary key with that value Create rules/constraints for: insertion of foreign keys update and deletion of primary keys G. Green 16

17. Adding Referential Integrity Constraints D:R, U:C (PK) (FK) D:R, U:C (PK) (FK)

18. Adding Referential Integrity Constraints, cont… G. Green 18

19. Normalization Convert complex relations into simpler relations Why? Ensures relations conform to rules Ensures relation contains facts about one “theme” Reveals/corrects redundancies, errors, ambiguities in data model Only a simple check IF good data model exists Normal Forms state of a relation rids relations of potential anomalies G. Green 19

20. Normalization, con’t... Insertion Anomalies are experienced when we attempt to store a value for one field but cannot because the value of another field is unknown e.g., cannot add a new customer’s information until an order number is ready to be entered G. Green 20 Order IDOrder DateCustomer ID Customer Name Customer Address (PK)

21. Normalization, con’t... Deletion Anomalies are experienced when a value for one field we wish to keep is unexpectedly removed when a value for another field is deleted e.g., cannot delete the sole order for a customer without deleting the only copy of the customer’s information also G. Green 21 Order IDOrder DateCustomer ID Customer Name Customer Address (PK)

22. Normalization, con’t... Update Anomalies are experienced when changes to multiple records of a table are needed to effect an update to a single value of a field e.g., cannot completely update a customer’s address without changing it for every order placed by that customer G. Green 22 Order IDOrder DateCustomer ID Customer Name Customer Address (PK)

23. G. Green 23 Steps in Normalization

24. Normalization, con’t... Every attribute is dependent on: the key (1NF) the WHOLE key (2NF) and nothing but the key (3NF) G. Green 24

25. 1NF The table is a relation Primary key No repeating values or groups only atomic values All column values from same domain To correct: define new (usually associative) entity G. Green 25

26. 2NF 1NF + No partial functional dependencies (Full) Functional dependency when the value of one attribute can be determined based on the value of another attribute Partial functional dependency when a non-key attribute is functionally dependent on a part of the PK Already in 2NF if: PK is NOT concatenated Relation contains no non-key attributes To correct: Decompose into 2 or more relations (if not already) one with original (concatenated) key + attributes one (or more) with the “depended on” partial key as PK + attributes G. Green 26

27. 3NF 2NF + No transitive dependencies Transitive dependency a functional dependency between 2 non-key attributes when a non-key attribute is functionally dependent on another non-key attribute Already in 3NF if: only 0 or 1 non-key attributes in relation To correct: Decompose into 2 or more relations (if not already) one with original PK + attributes one (or more) with “depended on” non-key attribute as PK + attributes G. Green 27

28. OTHER DATA MODELS G. Green 28

29. The Evolution of Data Models Hierarchical Network Relational Object oriented Multi-dimensional NoSQL G. Green 29

30. Hierarchical Data Model Each parent can have many children Each child has only one parent Tree defined by path that traces parent segments to child segments, beginning from the left Hierarchical path Ordered sequencing of segments tracing hierarchical structure G. Green 30

31. Problem: Child with Multiple Parents G. Green 31

32. Hierarchical Data Model, cont… Advantages Database security Performance, efficiency Data independence Disadvantages Complex implementation Structural dependence Complex applications programming and use Lack of standards G. Green 32

33. Network Data Model  Created to: › Represent complex M:M data relationships Child can have many parents › Impose a database standard  Resembles hierarchical model › Collection of records in 1:M relationships  Sets › Implement relationships › Composed of:  Owner  Member G. Green 33

34. Network Data Model, cont… Advantages Handles more relationship types Conformance to standards Disadvantages System complexity Lack of (popular) product support G. Green 34

35. Big Data Big Data = more than you're able to effectively process Influenced by Mobile, Social Networking, Web analytics, RFID, Atmospheric, Medical Research data, … Issue: ability of traditional RDBMSs to handle “big data” G. Green 35

36. Big Data RDBMS Issues Traditional RDBMS Problems Transaction-focus  Requires schema  maintenance issue ACID-focus  Requires locks, db constraints, joins  performance & availability issues "Relatively" small amounts of operational data  Exceptions require complex, $ actions  scalability issue Traditional RDBMS Solutions G. Green 36 Reference: http://www.slideshare.net/dondemsak/intro-to-big-data-and-nosqlhttp://www.slideshare.net/dondemsak/intro-to-big-data-and-nosql

37. Complex Data Landscape NOTE: This diagram is for effect ONLY—it is incomplete (e.g., no MDDB, no OODB) AND contains some inaccuracies G. Green 37

38. Big Data Solutions  NewSQL Databases  Columnar Databases  Hadoop  NoSQL Databases G. Green 38

39. Big Data Solutions  NewSQL Databases Relational-based Good when high scalability needed with relational DBMSs  Re-written, highly optimized storage engines  "Sharding" (i.e. horizontal partitioning across multiple DB instances)  In-memory databases  Distributed query processing G. Green 39

40. Big Data Solutions  Columnar Databases Good for data warehouses, analytics  computing aggregates on a few columns File contains:  all values of a specific column vs. all values of all columns G. Green 40

41. Multidimensional Data Model  Data represented as cube  Modeling and analyzing of data across dimensions  Cube depicts business measures analyzed by dimensions  Optimized for decision-making vs. transaction processing  Data storage  Pre-aggregation  De-normalization  Basis for data warehouses G. Green 41

42. Big Data Solutions  Hadoop Good for storing, retrieving large amounts of semi- and unstructured data in batch/offline mode  HDFS: data distributor  MapReduce: request/processing distributor G. Green 42

43. NoSQL Databases Focus (in most cases): Scalability via Physical Distribution and Replication of data No fixed schema "Individual query systems" instead of SQL Support for semi- and un-structured data Some provide consistency Apache's Hbase Amazon's DynamoDB Most provide "eventual consistency" Google’s BigTable Facebook's Cassandra Amazon's SimpleDB Uses a variety of data models… G. Green 43

44. Big Data/NoSQL, cont… NoSQL Physical Data Models Column (Family) Store Key Value Store Document Store Graph G. Green 44 Advantages Highly scalable Good for many writes Support for semi- and un-structured data Data Model (schema) does not have to be defined up-front Many are open source Cloud options available (e.g., Amazon's SimpleDB) Disadvantages No common query language Data inconsistency (“dirty reads”) Reliance on client applications for data validation, consistency, etc…

45. Summary Data Models Hierarchical Network Multidimensional NoSQL/Big Data ACID, BASE Relational Model Components Structure Rules Manipulation Transforming ERDs to Relations Representing entities and relationships  understand foreign keys Referential Integrity  understand RI constraints Normalization Purpose  understand anomalies 3 normal forms G. Green 45