1. Topic 3: Database Professor J. Alberto Espinosa The Edge of IT ITEC-200 Fall 2006

2. Roadmap IT & Business IT Infrastructure Database DB IT Infrastrucure: - HW & SW - Database - Telecom Transaction Processing Business Applications Information Decision Support Distributed Collaboration Enterprise Collaboration Financial Management etc. Client Appl Server Appl Business Applications IT & Business

3. Topic 3: Database p.3 Agenda Introduction to database and database management systems (DBMS) concepts Learn how to query databases to extract the information you need Learn how to design and implement databases

4. Topic 3: Database p.4 Database & Database Management System (DBMS) Concepts Follow up course: ITEC-470 Databases, Data Mining & Knowledge Management

5. Topic 3: Database p.5 Definitions Database: An organized collection of “logically related” data that can be retrieved on demand Database Management System (DBMS): Software that manages databases (i.e., define, create, update, and query databases) Acts as intermediary between business applications and physical data files “Most powerful, scalable, flexible and effective business applications rely on a well designed database and a powerful underlying DBMS”

6. Topic 3: Database p.6 HARDWARE HARDWARE System Software Application Programs API 1 INSTRUCTION SET The Old Way: Data files Windows Unix, Linux PC, Mainframe Accounting, Human Resources Examples: Data and program files were separate. You had to write individual programs to: define the data; upload it; update it; manipulate it; and or retrieve it Data Files

7. Topic 3: Database p.7 HARDWARE HARDWARE System Software DBMS Database Application API 2 API 1 INSTRUCTION SET A Better Way: Using a DBMS Windows Unix, Linux PC, Mainframe Oracle, Access, MS SQL Server Accounting, Human Resources, ERP, CRM Examples: A business application passes high level instructions to the DBMS. The DBMS has capabilities to do all the necessary data management: data definition, manipulation, and retrieval. So, the business application does not have to worry about low level data management functions Database

8. Topic 3: Database p.8 Advantages of Using Databases & DBMSs Programs independent of data structure Less data redundancy Better consistency in the data More flexibility & scalability Easier to integrate & share data Easier to develop business applications Easier to enforce business rules/constraints Easier access to data by users (e.g., queries, reports, forms, etc.)

9. Topic 3: Database p.9 Database Server: runs “back-end” part of the DBMS to process queries and perform database management tasks Database Client: runs “front-end” part of the DBMS that provides the user interface (e.g., data entry, screen displays or presentation, report formatting, query building tools) CLIENT SERVER Request (ex. query) DBMS in a Client/Server Environment Database Front-End DBMS Back-End DBMS Response (ex. query result) Execute query

10. Topic 3: Database p.10 DBMS in a Web Server Environment Request (ex. get a price quote, place an order) Response (ex. query results with HTML-formatted product price or order confirmation notice)

11. Topic 3: Database p.11 Data Warehouse “A database that stores and consolidates current and historical data from various systems (internal and external) with tools for management reporting and sophisticated analysis—i.e., Datamining”

12. Topic 3: Database p.12 Most Common Database Models Hierarchical (of historical interest only) Network (of historical interest only) Relational Object Oriented databases (new)

13. Topic 3: Database p.13 A database with structured data consisting of: Tables: or “entities” Every table has a unique name Ex. Students, Courses Fields: or “columns”, “attributes” Every field has a unique name within the table Ex. Students (StudentID, StudentName, Major, Address) Ex. Courses (CourseNo, CouseName, CreditPoints, Description) Records: or “rows”, “tuples”, “instances” Every record is unique (has a unique field that identifies it) Ex. {“jdoe”, “John Doe”, “CS”, 5000 Forbes Ave.) Ex. {“MGMT-352-001”, “MIS”, Fall 2002, “A great course”} Relational Database

14. Topic 3: Database p.14 Terminology Equivalence ERD or Data Model OO DatabaseRelational Database Other Terms Used EntityClassTable InstancesObjectsRecordsRows, Tuples Relationship AttributesPropertiesFieldsColumns

15. Topic 3: Database p.15 DBMS Functions and Tools Performs 3 main functions: –Data definition (define, create databases) –Data manipulation (data entry, updates) –Data retrieval (extraction, reports, displays) Plus additional database tools: –Data dictionary: data about the database –Visual tools: report & form design –Data modeling & database design tools –Macros and programming languages –Internet/web features, etc. Examples: –Oracle, DB2, Visual FoxPro, MS Access & MS SQL

16. Topic 3: Database p.16 Define, Create and Manipulate Data in Databases

17. Topic 3: Database p.17 MS Access (structured data)

18. Topic 3: Database p.18 MS Access Data Definition

19. Topic 3: Database p.19 MS Access Data Manipulation

20. Topic 3: Database p.20 Visual FoxPro Data Manipulation

21. Topic 3: Database p.21 Lotus Notes

22. Topic 3: Database p.22 Displaying and Printing Data From Databases: Forms and Reports The idea: (1) organize and store the data in the most efficient way, without data redundancy (2) retrieve the data in the way you want to see it (not necessarily how it is organized internally) using queries (3) present the results to users and managers using forms (for data entry) and report (for printouts)

23. Topic 3: Database p.23 Forms Typically used for data entry & screen displays

24. Topic 3: Database p.24 Reports Typically used for printouts

25. Topic 3: Database p.25 Database Queries

26. Topic 3: Database p.26 Queries are often thought of as a method to retrieve data, but queries can also be used to define and manipulate data Databases can be queried in many ways: Proprietary DBMS commands and languages, or Standard query methods/languages (QBE, SQL, etc.) Data definition (define, create databases) Data manipulation (data entry, updates) Data retrieval (extraction, reports, displays) Queries are DBMS Commands for:

27. Topic 3: Database p.27 Standard Query Methods Query by Example (QBE) Called Design View in MS Access Visual interface using examples of data requested Similar to how you do searches in the library Structured Query Language (SQL) Popular with power users Works in most DBMS Can embed SQL commands in programs, web scripts, etc. English-like commands (4GL), practical Exact, mathematical: relational algebra & matrix math

28. Topic 3: Database p.28 Query by Example (QBE) Called Query “Design View” in MS Access Column labels are the fields we want to retrieve In table cells we enter “examples” of the info we want

29. Topic 3: Database p.29 8 SQL Commands Only Data Definition: CREATE TABLE, DROP TABLE Data Manipulation: INSERT, UPDATE, DELETE, UNION, JOIN Data Retrieval: SELECT (need to learn this only)

30. Topic 3: Database p.30 One SQL Command SQL Commands: Data Definition Example: Create & Delete Table called “ Friends ” CREATE TABLE Friends (FriendID integer, LastName char(24), FirstName char(24), Birthday date, Phone char(10), Notes memo); ; = End of Command DROP TABLE Friends;

31. Topic 3: Database p.31 SQL Commands: Data Manipulation INSERT: Add new records UPDATE: Modify existing records DELETE: Delete records UNION: Combine records from two tables JOIN: Combine columns from two tables

32. Topic 3: Database p.32 SQL Commands: Data Manipulation Add & Update Records Insert (add) a complete record (values in all fields): INSERT INTO Friends VALUES (“ae”, “Espinosa”, “Alberto”, 12/12/2002, “885-1958”, “Looks tired, needs a vacation”) Update (modify) record with new values: UPDATE Friends SET LastName=“Espinosa” Insert (add) partial record (values in some fields only): INSERT INTO Friends (FriendID, LastName, FirstName) VALUES (“ae”, “Espinosa”, “Alberto”)

33. Topic 3: Database p.33 Data Retrieval Queries: The Idea Organize database (design, create): –In the most efficient & consistent way (internally) –Not based on how you want the data to look Produce the “virtual” tables as you want them to look using queries How we store the data How we display the data

34. Topic 3: Database p.34 Data Retrieval in SQL The “ SELECT ” Command SELECT –columns to retrieve FROM –tables that contain the data WHERE condition1 –which records to retrieve AND condition2 ……. ORDER BY field1, ….. –to sort the query result SELECT can be followed by: DISTINCT (eliminates duplicate rows from result) TOP # (lists only the top # rows of result) * (lists all fields in the table)

35. Topic 3: Database p.35 Complexity of SELECT Queries Simple Queries: Involve a single table Complex (Join) Queries: Involve more than one table

36. Topic 3: Database p.36 Simple SELECT Queries SELECT ProdID, ProdName, Type, Price FROM Products WHERE Price>=300; SELECT Avg(Price) AS AvgPrice FROM Products WHERE Price>=120 AND Type=“Percussion”; SELECT Count(*) as TotOrders FROM Orders Click here to download this database [local copy]herelocal copy

37. Complex SELECT (Join) Queries Tables: Orders (OrderNo, ClientID, OrderDate, OrderStatus) LineItems (OrderNo, LineItem, ProdID, Qty) Table Join (2 ways) : SELECT Orders.OrderNo, ClientID, LineItem, ProdID, Qty FROM Orders, LineItems WHERE Orders.OrderNo = LineItems.OrderNo; Table Product (Don't do this!! Don’t forget the join condition) : SELECT Orders.OrderNo, ClientID, LineItem, ProdID, Qty FROM Orders, LineItems; Join Condition

38. Topic 3: Database p.38 Database Design

39. Topic 3: Database p.39 Database Design Issue #1: Data Integrity in Databases Two main types of data integrity: Entity Integrity: Ensuring that every record in each table in the database can be addressed (i.e., found) -- this means that there each record has to have a unique identifier that is not duplicate or null (i.e., not blank) Examples: every student has an AU ID; every purchase order has a unique number; every customer has an ID Referential Integrity: Ensuring that the data that is entered in one table is consistent with data in other tables Examples: purchase orders can only be placed by valid customers; accounting transactions can only be posted to valid company accounts

40. Topic 3: Database p.40 Database Design Issue #2: Special Fields (Keys) How do you find the record (info) you need? Primary key (PK): –Field(s) that uniquely identifies a record (CourseNo, InstructorID) –Entity integrity = PK is not duplicate & not blank –PK can be: –A single field (e.g., UserID), or –Multi-field—i.e., “composite” (OrderNo, LineItem) –Dual key = a composite PK with only 2 fields How do you link related tables in a database? Foreign key (FK): –A field in a table that is a PK in another table –That is, a field that “must” exist in another table –This is how referential integrity is maintained

41. Topic 3: Database p.41 Illustration: Primary and Foreign Keys PK FK

42. Topic 3: Database p.42 Design Issue #3: Steps in Database Design 1.Build a Data Model – or Entity-Relationship Diagram (ERD) – for the business application: this model describes each data element in the application and how different data elements relate to each other 2.Review the data model to ensure entity and referential integrity 3.Convert (i.e., map) the data model into a relational database design Convert the data model into a database relationship diagram 4.Implement Important Data Rules, there are 3 of them: Add update rules: (1) don’t allow or (2) cascade updates Add delete rules: (1) don’t allow or (2) cascade updates Add business rules: e.g., PayDate >= OrderDate; OrderQty > 100 5.Normalize the database design as necessary and update the data model and design if necessary – i.e., re-organize the tables to avoid recording redundant data

43. Topic 3: Database p.43 Data Model Example (Entity Relationship Diagram--ERD) : Course Registration System Instructors InstructorID LastName FirstName Telephone EMailAddr Course CourseNo CourseDescription InstructorID CreditPoints PreRequisites ClassroomNo Teaches Enrollment StudentID CourseNo Comments Students StudentID LastName FirstName SSN Department College Major EMailAddr Enrolls Includes 1Many 1 1 Entities Relationships

44. Topic 3: Database p.44 Entity-Relationship Diagrams (ERDs) Conceptual Data Modeling Data-oriented modeling method that describes the data and relationships among data entities Goal: capture meaning of the data 2 main ERD constructs: –Entities and its attributes –Relationships between entities

45. Topic 3: Database p.45 Entity “An object, person, place, event or thing or which we want to record data” Equivalent to a table in a database Examples: instructors, students, classrooms, invoices, registration, machines, countries, states, etc. Entity instance: a single occurrence of an entity Example: Espinosa, Kogod 39, MGMT-352, 001 Entities can be identified in a requirements analysis description by following the use of NOUNS

46. Topic 3: Database p.46 Entity Representation Entity Name Peter Chen’s notation

47. Topic 3: Database p.47 Relationships Relationships describe how two entities relate to each other Relationships in a database application can be identified following the VERBS that describe how entities are associated with one another Examples: students enroll in courses countries have cities, etc.

48. Topic 3: Database p.48 Cardinality Is an important database concept that helps understand how two entities are related Cardinality: describes how many instances of one entity can be associated with another entity The cardinality of a relationship between two entities has two components: –Maximum Cardinality: is the maximum number of instances that can be associated with the other entity – usually either 1 or many (the exact number is rarely used) –Minimum Cardinality: is the minimum number of instances that can be associated with the other entity – usually either 0 or 1 –Symbols: 0 1 Many Examples: 1 student can only park 1 (or 0) cars – 1 to (0 or) 1 1 client can place (0 or ) many orders – 1 to (0 or) many 1 student can enroll in (at least 1 or) many courses and a course can have (0 or) many students – (0 or) many to (1 or) many

49. Topic 3: Database p.49 ERD SYMBOLS (cont’d.) 1 to 1 Maximum Cardinality (OUTER symbol) Minimum Cardinality (INNER symbol) MandatoryOptional StudentEmailAccount StudentCar Has Parks Peter Chen’s notation using Systems Architect software

50. Topic 3: Database p.50 ERD SYMBOLS (cont’d.) 1 to Many 1 to Many (or None) Maximum Cardinality Minimum Cardinality Mandatory Optional AdvisorStudent FacultyCourse Teaches Peter Chen’s (“crow’s feet”) notation using Systems Architect software → Advises ← Have

51. Topic 3: Database p.51 Many to Many Relationships? (0 or) Many to Many (at least 1) 1 to Many (at least 1)1 to Many (or None) Not good for database  Need to convert every Many-to-Many relationship into 2 One-to-Many relationships with a third table called “Intersection Table” OrdersProducts Orders LineItems Intersection Table

52. Topic 3: Database p.52 Adding Attributes to ERD Instructors PKInstructorID LastName FirstName Telephone EMail Course PKCourseNo CourseDescription FK1InstructorID CreditPoints PreRequisites Teaches Peter Chen’s notation & MS Visio software

53. Topic 3: Database p.53 Steps in ERD Modeling 1.Identify and diagram all ENTITIES 2.Add PK attributes – i.e., implement entity integrity Ensure PK’s are non-null & non-duplicates 3.Identify and diagram all RELATIONSHIPS and their respective CARDINALITIES 4.Add FK attributes – i.e., implement referential integrity (this is automatic in some tools—MS Access) 5.Add remaining attributes

54. Topic 3: Database p.54 ERD Example: Course Registration System Courses (CourseNo (PK), CourseDescripition, InstructorID, CreditPoints, ClassroomNo) PreRequisites (CourseNo (PK), PreRequisiteNo (PK), Comments) Students (StudentID (PK), LastName, FirstName, SSN, Department, College, Major, EMail) Enrollment (StudentID (PK), CourseNo (PK), Comments) Instructors (InstructorID (PK), LastName, FirstName, Telephone, EMail) Classrooms (ClassroomNo (PK), ClassroomName, Building, BuildingRoomNo, Equipment, Capacity) Note: PK denotes a primary key

55. Topic 3: Database p.55 Example: Course Registration System Step 1. Draw Entities

56. Topic 3: Database p.56 Example: Course Registration System Step 2. Add PK’s (undeline/separate with a line)

57. Topic 3: Database p.57 Instructors PKInstructorID Course PKCourseNo Teaches PreRequisites PK,FK1CourseNo PKPreRequisiteNo ClassRooms PKClassroomNo Enrollment PK,FK1StudentID PK,FK2CourseNo Students PKStudentID has Enrolls Includes Assigned Example: Course Registration System Step 3. Add Relationships (w/Cardinalities)

58. Topic 3: Database p.58 Example: Course Registration System Step 4. Add FK’s Instructors PKInstructorID Course PKCourseNo FK1InstructorID FK2ClassroomNo Teaches PreRequisites PK,FK1CourseNo PKPreRequisiteNo ClassRooms PKClassroomNo Enrollment PK,FK1StudentID PK,FK2CourseNo Students PKStudentID has Enrolls IncludesAssigned

59. Topic 3: Database p.59 Instructors PKInstructorID LastName FirstName Telephone EMail Course PKCourseNo CourseDescription FK1InstructorID CreditPoints FK2ClassroomNo Teaches PreRequisites PK,FK1CourseNo PKPreRequisiteNo Comments ClassRooms PKClassroomNo ClassroomName Building BuildingRoomNo Equipment Capacity Enrollment PK,FK1StudentID PK,FK2CourseNo Comments Students PKStudentID LastName FirstName SSN Department College Major EMail Has Enrolls Includes Assigned Example: Course Registration System Step 5. Add Remaining Attributes

60. Topic 3: Database p.60 EXAMPLE: Package Delivery Tracking System

61. Topic 3: Database p.61 EXAMPLE: Airline Reservation System

62. Topic 3: Database p.62 EXAMPLE: HW2 Accounting Database (Data Model diagram using MS Access)

63. Topic 3: Database p.63 ERD Example: A Music Collection Database You have such a great music collection that your friends keep borrowing your music. Things are getting out of control. You no longer know who has what, so you decide to create a database to inventory your music items and keep track of who has borrowed which items. You analyze the requirements and come up with the following entities: Artists: A list of individual or group artists in your collection MediaTypes: A list of the different media you possess (CD, VHS, DVD, etc.) MusicItems: A list of the individual CD's, VHS tapes, etc. you own Suspects: A list of friends and acquaintances who borrowed music items from you CheckOut: A log where you keep track of who borrowed what You then decide to collect these attributes for each entity (PK denotes a primary key): Artists (ArtistID (PK), ArtistName, Nationality, ArtistRating) MediaTypes (MediaCode (PK), MediaName, Description) MusicItems (ItemNo (PK), Title, ArtistID, MediaCode, YearRelease, DatePurchased, MusicType, ItemRating) Suspects (SuspectID (PK), SuspectName, Address, Telephone) CheckOut (ItemNo (PK), SuspectID (PK), DateBorrowed (PK), ReturnExpectedDate, Comments)

64. Topic 3: Database p.64 ERD Example (cont’d) : A Music Collection Database Your requirements for this application also need to comply with the following (entities in bold italics and relationships underlined, note respective use of nouns and verbs): A music item (CD, tape, etc.) are recorded by one artist, but an artist may appear in more than one music item in your collection (e.g., you have several albums from Dave Mathews). A music item is associated with one media type, but a you may have several music items in a given media type (e.g., you have several CD’s). Over time, suspects can check out many items (e.g. Alberto has borrowed several CD’s from you), but each check out entry is associated with only one suspect. Music items may also be checked out many times over time (e.g. one of you Dave Mathews albums has been borrowed several times), but every check out entry is associated with only one music item. Please draw an ERD with all entities, relationships, attributes (please underline PK's) and cardinalities. If you feel the information above is incomplete, please make any business assumptions you need and clearly state them. Also, describe in words (no need to diagram) how your design would change if music items can contain more than one artist.

65. Topic 3: Database p.65 ERD Illustration: Music Collection MusicItems Key Data ItemNo [PK1] Non-Key Data Title ArtistID MediaCode YearRelease DataPurchased MusicType ItemRating CheckOut Key Data ItemNo [PK1] SuspectID [PK2] DataBorrowed [PK3] Non-Key Data ReturnExpectedDate Comments Artists Key Data ArtistID [PK1] Non-Key Data ArtistName Nationality ArtistRating Suspects Key Data SuspectID [PK1] Non-Key Data SuspectName Address Telephone MediaTypes Key Data MediaCode [PK1] Non-Key Data MediaName Description By Associated Checked Out Borrows

66. Topic 3: Database p.66 What is a good database design? 1.Sound data model (ERD) 2.Implement “Entity Integrity” and “Referential Integrity” 3.Implement other important rules (note: MS Access has features to enforce these rules) Update Rules: what can be updated and when Delete Rules: what can be deleted and when Business Rules: needed to conduct business 4.The database is “Normalized” – no redundancy

67. Topic 3: Database p.67 Update Rules What can be updated/modified in the database and when? 1.It is OK to update values in any non-PK fields, provided that referential integrity and business rules are respected 2.It is OK to update values in the PK in one table if it is not linked to a FK in another table, provided that entity integrity, referential integrity and business rules are respected 3.If a PK is linked to a FK in another table, we need to ensure that referential integrity is maintained. Depending on what makes sense for business and the data itself, either: Disallow updates of values in the PK, or Allow updates, but cascade changes to all related FKs in other tables

68. Topic 3: Database p.68 Delete Rules What can be deleted in the database and when? 1.It is OK to delete records in a table [only] if its PK is not linked to a FK in another table 2.If its PK is linked to a FK in another table, we need to ensure that referential integrity is maintained. Depending on what makes sense for business and the data itself, either: Disallow deletions, or Allow deletions, but cascade deletions in all related tables that contain a FK linked to this table

69. Topic 3: Database p.69 Business Rules Most DBMS have features that allow you to impose constraints in the data to meet rules imposed by a company when conducting business: i.e., “business rules” – examples: CustomerAge >= 18 OrderQty >= 100 ProductPrice <= 1000 PaymentDate <= PurchaseDate + 90