1. Carnegie Mellon Carnegie Mellon Univ. Dept. of Computer Science 15-415 - Database Applications C. Faloutsos Introduction

2. Carnegie Mellon 15-415 - C. Faloutsos2 Outline Introduction to DBMSs The Entity Relationship model The Relational Model SQL: the commercial query language DB design: FD, 3NF, BCNF indexing, q-opt concurrency control & recovery advanced topics (data mining, multimedia)

3. Carnegie Mellon 15-415 - C. Faloutsos3 We’ll learn: What are RDBMS –when to use them –how to model data with them –how to store and retrieve information –how to search quickly for information Internals of an RDBMS: indexing, transactions

4. Carnegie Mellon 15-415 - C. Faloutsos4 We’ll learn (cnt’d) Advanced topics –multimedia indexing (how to find similar, eg., images) –data mining (how to find patterns in data)

5. Carnegie Mellon 15-415 - C. Faloutsos5 Detailed outline Introduction –Motivating example –How do DBMSs work? DDL, DML, views. –Fundamental concepts –DBMS users –Overall system architecture –Conclusions

6. Carnegie Mellon 15-415 - C. Faloutsos6 What is the goal of rel. DBMSs Electronic record-keeping: Fast and convenient access to information.

7. Carnegie Mellon 15-415 - C. Faloutsos7 Definitions ‘DBMS’ = ‘Data Base Management System’: the (commercial) system, like: DB2, Oracle, MS SQL-server,... ‘Database system’: DBMS + data + application programs

8. Carnegie Mellon 15-415 - C. Faloutsos8 Motivating example Eg.: students, taking classes, obtaining grades; find my gpa

9. Carnegie Mellon 15-415 - C. Faloutsos9 Obvious solution: paper-based advantages? disadvantages? eg., student folders, alpha sorted

10. Carnegie Mellon 15-415 - C. Faloutsos10 Obvious solution: paper-based advantages? –cheap; easy to use disadvantages? eg., student folders, alpha sorted

11. Carnegie Mellon 15-415 - C. Faloutsos11 Obvious solution: paper-based advantages? –cheap; easy to use disadvantages? –no ‘ad hoc’ queries –no sharing –large physical foot-print

12. Carnegie Mellon 15-415 - C. Faloutsos12 Next obvious solution computer-based (flat) files + C (Java,...) programs to access them e.g., one (or more) UNIX/DOS files, with student records and their courses

13. Carnegie Mellon 15-415 - C. Faloutsos13 Next obvious solution your layout for the student records?

14. Carnegie Mellon 15-415 - C. Faloutsos14 Next obvious solution your layout for the student records? (eg., comma-separated values ‘csv’ Smith,John,123,db,A,os,B Tompson,Peter,234 Atkinson,Mary,345,os,B,graphics,A

15. Carnegie Mellon 15-415 - C. Faloutsos15 Next obvious solution your layout for the student records? (many other layouts are fine, eg.: Smith,John,123 Tompson,Peter,234 Atkinson,Mary,345 123,db,A 123,os,B 345,os,B 345,graphics,A

16. Carnegie Mellon 15-415 - C. Faloutsos16 Problems? inconvenient access to data (need ‘C++’ expertize, plus knowledge of file-layout) –data isolation data redundancy (and inconcistencies) integrity problems atomicity problems

17. Carnegie Mellon 15-415 - C. Faloutsos17 Problems? (cont’d)... concurrent-access anomalies security problems

18. Carnegie Mellon 15-415 - C. Faloutsos18 Problems? (cont’d) [ why? because of two main reasons: –file-layout description is buried within the C programs and –there is no support for transactions (concurrency and recovery) ] DBMSs handle exactly these two problems

19. Carnegie Mellon 15-415 - C. Faloutsos19 DBMS solution commercial/freeware DBMS & application programs

20. Carnegie Mellon 15-415 - C. Faloutsos20 Main vendors/products Commercial Oracle IBM/DB2 MS SQL-server Sybase Informix (MS Access,...) Open source Postgres (UCB) mySQL, mSQL miniBase (Wisc) Predator (Cornell) (www.acm.org/sigmod)

21. Carnegie Mellon 15-415 - C. Faloutsos21 demo a simple query; ‘views’ a join an aggregate query

22. Carnegie Mellon 15-415 - C. Faloutsos22 Detailed outline Introduction –Motivating example –How do DBMSs work? DDL, DML, views. –Fundamental concepts –DBMS users –Overall system architecture –Conclusions

23. Carnegie Mellon 15-415 - C. Faloutsos23 How do DBs work? Pictorially: DBMS data and meta-data = catalog = data dictionary select * from student

24. Carnegie Mellon 15-415 - C. Faloutsos24 How do DBs work? %isql mydb sql>create table student ( ssn fixed; name char(20) ); /mydb

25. Carnegie Mellon 15-415 - C. Faloutsos25 How do DBs work? sql>insert into student values (123, “Smith”); sql>select * from student;

26. Carnegie Mellon 15-415 - C. Faloutsos26 How do DBs work? sql>create table takes ( ssn fixed, c-id char(5), grade fixed));

27. Carnegie Mellon 15-415 - C. Faloutsos27 How do DBs work - cont’d More than one tables - joins Eg., roster (names only) for ‘db’

28. Carnegie Mellon 15-415 - C. Faloutsos28 How do DBs work - cont’d sql> select name from student, takes where student.ssn = takes.ssn and takes.c-id = ‘db’

29. Carnegie Mellon 15-415 - C. Faloutsos29 Views - a powerful tool! what and why? suppose secy is allowed to see only ssn’s and GPAs, but not individual grades -> VIEWS!

30. Carnegie Mellon 15-415 - C. Faloutsos30 Views sql> create view fellowship as ( select ssn, avg(grade) from takes group by ssn);

31. Carnegie Mellon 15-415 - C. Faloutsos31 Views Views = ‘virtual tables’

32. Carnegie Mellon 15-415 - C. Faloutsos32 Views sql> select * from fellowship;

33. Carnegie Mellon 15-415 - C. Faloutsos33 Views sql> grant select on fellowship to secy;

34. Carnegie Mellon 15-415 - C. Faloutsos34 Iterating: advantages over (flat) files logical and physical data independence, because data layout, security etc info: stored explicitly on the disk concurrent access and transaction support

35. Carnegie Mellon 15-415 - C. Faloutsos35 Disadvantages over (flat) files?

36. Carnegie Mellon 15-415 - C. Faloutsos36 Disadvantages over (flat) files Price additional expertise (SQL/DBA) (hence: over-kill for small, single-user data sets)

37. Carnegie Mellon 15-415 - C. Faloutsos37 Detailed outline Introduction –Motivating example –How do DBMSs work? DDL, DML, views. –Fundamental concepts –DBMS users –Overall system architecture –Conclusions

38. Carnegie Mellon 15-415 - C. Faloutsos38 Fundamental concepts 3-level architecture logical data independence physical data independence

39. Carnegie Mellon 15-415 - C. Faloutsos39 3-level architecture view level logical level physical level v1 v2 v3

40. Carnegie Mellon 15-415 - C. Faloutsos40 3-level architecture view level logical level: eg., tables –STUDENT(ssn, name) –TAKES (ssn, c-id, grade) physical level: –how are these tables stored, how many bytes / attribute etc

41. Carnegie Mellon 15-415 - C. Faloutsos41 3-level architecture view level, eg: –v1: select ssn from student –v2: select ssn, c-id from takes logical level physical level

42. Carnegie Mellon 15-415 - C. Faloutsos42 3-level architecture -> hence, physical and logical data independence: logical D.I.: –??? physical D.I.: –???

43. Carnegie Mellon 15-415 - C. Faloutsos43 3-level architecture -> hence, physical and logical data independence: logical D.I.: –can add (drop) column; add/drop table physical D.I.: –can add index; change record order

44. Carnegie Mellon 15-415 - C. Faloutsos44 Detailed outline Introduction –Motivating example –How do DBMSs work? DDL, DML, views. –Fundamental concepts –DBMS users –Overall system architecture –Conclusions

45. Carnegie Mellon 15-415 - C. Faloutsos45 Database users ‘naive’ users casual users application programmers [ DBA (Data base administrator)]

46. Carnegie Mellon 15-415 - C. Faloutsos46 casual users DBMS data and meta-data = catalog select * from student

47. Carnegie Mellon 15-415 - C. Faloutsos47 naive users Pictorially: DBMS data and meta-data = catalog app. (eg., report generator)

48. Carnegie Mellon 15-415 - C. Faloutsos48 App. programmers those who write the applications (like the ‘report generator’)

49. Carnegie Mellon 15-415 - C. Faloutsos49 DB Administrator (DBA) schema definition (‘logical’ level) physical schema (storage structure, access methods schemas modifications granting authorizations integrity constraint specification

50. Carnegie Mellon 15-415 - C. Faloutsos50 Detailed outline Introduction –Motivating example –How do DBMSs work? DDL, DML, views. –Fundamental concepts –DBMS users –Overall system architecture –Conclusions

51. Carnegie Mellon 15-415 - C. Faloutsos51 Overall system architecture [Users] DBMS –query processor –storage manager [Files]

52. Carnegie Mellon 15-415 - C. Faloutsos52 DDL int.DML proc. query eval. app. pgm(o) trans. mgr emb. DML buff. mgrfile mgr datameta-data query proc. storage mgr. naiveapp. pgmrcasualDBA users

53. Carnegie Mellon 15-415 - C. Faloutsos53 Overall system architecture query processor –DML compiler –embedded DML pre-compiler –DDL interpreter –Query evaluation engine

54. Carnegie Mellon 15-415 - C. Faloutsos54 Overall system architecture (cont’d) storage manager –authorization and integrity manager –transaction manager –buffer manager –file manager

55. Carnegie Mellon 15-415 - C. Faloutsos55 Overall system architecture (cont’d) Files –data files –data dictionary = catalog (= meta-data) –indices –statistical data

56. Carnegie Mellon 15-415 - C. Faloutsos56 Some examples: DBA doing a DDL (data definition language) operation, eg., create table student...

57. Carnegie Mellon 15-415 - C. Faloutsos57 DDL int.DML proc. query eval. app. pgm(o) trans. mgr emb. DML buff. mgrfile mgr datameta-data query proc. storage mgr. naiveapp. pgmrcasualDBA users

58. Carnegie Mellon 15-415 - C. Faloutsos58 Some examples: casual user, asking for an update, eg.: update student set name to ‘smith’ where ssn = ‘345’

59. Carnegie Mellon 15-415 - C. Faloutsos59 DDL int.DML proc. query eval. app. pgm(o) trans. mgr emb. DML buff. mgrfile mgr datameta-data query proc. storage mgr. naiveapp. pgmrcasualDBA users

60. Carnegie Mellon 15-415 - C. Faloutsos60 DDL int.DML proc. query eval. app. pgm(o) trans. mgr emb. DML buff. mgrfile mgr datameta-data query proc. storage mgr. naiveapp. pgmrcasualDBA users

61. Carnegie Mellon 15-415 - C. Faloutsos61 DDL int.DML proc. query eval. app. pgm(o) trans. mgr emb. DML buff. mgrfile mgr datameta-data query proc. storage mgr. naiveapp. pgmrcasualDBA users

62. Carnegie Mellon 15-415 - C. Faloutsos62 Some examples: app. programmer, creating a report, eg main(){.... exec sql “select * from student”... }

63. Carnegie Mellon 15-415 - C. Faloutsos63 DDL int.DML proc. query eval. app. pgm(o) trans. mgr emb. DML buff. mgrfile mgr datameta-data query proc. storage mgr. naiveapp. pgmrcasualDBA users pgm (src)

64. Carnegie Mellon 15-415 - C. Faloutsos64 Some examples: ‘naive’ user, running the previous app.

65. Carnegie Mellon 15-415 - C. Faloutsos65 DDL int.DML proc. query eval. app. pgm(o) trans. mgr emb. DML buff. mgrfile mgr datameta-data query proc. storage mgr. naiveapp. pgmrcasualDBA users pgm (src)

66. Carnegie Mellon 15-415 - C. Faloutsos66 Detailed outline Introduction –Motivating example –How do DBMSs work? DDL, DML, views. –Fundamental concepts –DBMS users –Overall system architecture –Conclusions

67. Carnegie Mellon 15-415 - C. Faloutsos67 Conclusions (relational) DBMSs: electronic record keepers customize them with create table commands ask SQL queries to retrieve info

68. Carnegie Mellon 15-415 - C. Faloutsos68 Conclusions cont’d main advantages over (flat) files & scripts: logical + physical data independence (ie., flexibility of adding new attributes, new tables and indices) concurrency control and recovery for free