1. 2002.11.19- SLIDE 1IS 257 - Fall 2002 Object-Oriented, Intelligent and Object-Relational Database Models University of California, Berkeley School of Information Management and Systems SIMS 257: Database Management

2. 2002.11.19- SLIDE 2IS 257 - Fall 2002 Lecture Outline Review –Applications for Data Warehouses –Decision Support Systems (DSS) –OLAP (ROLAP, MOLAP) –Data Mining Thanks again to lecture notes from Joachim Hammer of the University of Florida

3. 2002.11.19- SLIDE 3IS 257 - Fall 2002 What is Decision Support? Technology that will help managers and planners make decisions regarding the organization and its operations based on data in the Data Warehouse. –What was the last two years of sales volume for each product by state and city? –What effects will a 5% price discount have on our future income for product X? Increasing common term is KDD –Knowledge Discovery in Databases

4. 2002.11.19- SLIDE 4IS 257 - Fall 2002 Conventional Query Tools Ad-hoc queries and reports using conventional database tools –E.g. Access queries. Typical database designs include fixed sets of reports and queries to support them –The end-user is often not given the ability to do ad-hoc queries

5. 2002.11.19- SLIDE 5IS 257 - Fall 2002 OLAP Online Line Analytical Processing –Intended to provide multidimensional views of the data –I.e., the “Data Cube” –The PivotTables in MS Excel are examples of OLAP tools

6. 2002.11.19- SLIDE 6IS 257 - Fall 2002 Data Cube

7. 2002.11.19- SLIDE 7IS 257 - Fall 2002 Operations on Data Cubes Slicing the cube –Extracts a 2d table from the multidimensional data cube –Example… Drill-Down –Analyzing a given set of data at a finer level of detail

8. 2002.11.19- SLIDE 8IS 257 - Fall 2002 Star Schema Typical design for the derived layer of a Data Warehouse or Mart for Decision Support –Particularly suited to ad-hoc queries –Dimensional data separate from fact or event data Fact tables contain factual or quantitative data about the business Dimension tables hold data about the subjects of the business Typically there is one Fact table with multiple dimension tables

9. 2002.11.19- SLIDE 9IS 257 - Fall 2002 Star Schema for multidimensional data Order OrderNo OrderDate … Salesperson SalespersonID SalespersonName City Quota Fact Table OrderNo Salespersonid Customerno ProdNo Datekey Cityname Quantity TotalPrice City CityName State Country … Date DateKey Day Month Year … Product ProdNo ProdName Category Description … Customer CustomerName CustomerAddress City …

10. 2002.11.19- SLIDE 10IS 257 - Fall 2002 Data Mining Data mining is knowledge discovery rather than question answering –May have no pre-formulated questions –Derived from Traditional Statistics Artificial intelligence Computer graphics (visualization)

11. 2002.11.19- SLIDE 11IS 257 - Fall 2002 Goals of Data Mining Explanatory –Explain some observed event or situation Why have the sales of SUVs increased in California but not in Oregon? Confirmatory –To confirm a hypothesis Whether 2-income families are more likely to buy family medical coverage Exploratory –To analyze data for new or unexpected relationships What spending patterns seem to indicate credit card fraud?

12. 2002.11.19- SLIDE 12IS 257 - Fall 2002 Data Mining Applications Profiling Populations Analysis of business trends Target marketing Usage Analysis Campaign effectiveness Product affinity

13. 2002.11.19- SLIDE 13IS 257 - Fall 2002 Data Mining Algorithms Market Basket Analysis Memory-based reasoning Cluster detection Link analysis Decision trees and rule induction algorithms Neural Networks Genetic algorithms

14. 2002.11.19- SLIDE 14IS 257 - Fall 2002 Market Basket Analysis A type of clustering used to predict purchase patterns. Identify the products likely to be purchased in conjunction with other products –E.g., the famous (and apocryphal) story that men who buy diapers on Friday nights also buy beer.

15. 2002.11.19- SLIDE 15IS 257 - Fall 2002 Memory-based reasoning Use known instances of a model to make predictions about unknown instances. Could be used for sales forcasting or fraud detection by working from known cases to predict new cases

16. 2002.11.19- SLIDE 16IS 257 - Fall 2002 Cluster detection Finds data records that are similar to each other. K-nearest neighbors (where K represents the mathematical distance to the nearest similar record) is an example of one clustering algorithm

17. 2002.11.19- SLIDE 17IS 257 - Fall 2002 Link analysis Follows relationships between records to discover patterns Link analysis can provide the basis for various affinity marketing programs Similar to Markov transition analysis methods where probabilities are calculated for each observed transition.

18. 2002.11.19- SLIDE 18IS 257 - Fall 2002 Decision trees and rule induction algorithms Pulls rules out of a mass of data using classification and regression trees (CART) or Chi-Square automatic interaction detectors (CHAID) These algorithms produce explicit rules, which make understanding the results simpler

19. 2002.11.19- SLIDE 19IS 257 - Fall 2002 Neural Networks Attempt to model neurons in the brain Learn from a training set and then can be used to detect patterns inherent in that training set Neural nets are effective when the data is shapeless and lacking any apparent patterns May be hard to understand results

20. 2002.11.19- SLIDE 20IS 257 - Fall 2002 Genetic algorithms Imitate natural selection processes to evolve models using –Selection –Crossover –Mutation Each new generation inherits traits from the previous ones until only the most predictive survive.

21. 2002.11.19- SLIDE 21IS 257 - Fall 2002 Today Object-Oriented Database Systems Inverted File and Flat File DBMS Object-Relational DBMS –OR features in Oracle –OR features in PostgreSQL

22. 2002.11.19- SLIDE 22IS 257 - Fall 2002 Object-Oriented DBMS Basic Concepts Each real-world entity is modeled by an object. Each object is associated with a unique identifier (sometimes call the object ID or OID)

23. 2002.11.19- SLIDE 23IS 257 - Fall 2002 Object-Oriented DBMS Basic Concepts Each object has a set of instance attributes (or instance variables) and methods. –The value of an attribute can be an object or set of objects. Thus complex object can be constructed from aggregations of other objects. –The set of attributes of the object and the set of methods represent the object structure and behavior, respectively

24. 2002.11.19- SLIDE 24IS 257 - Fall 2002 Object-Oriented DBMS Basic Concepts The attribute values of an object represent the object’s status. –Status is accessed or modified by sending messages to the object to invoke the corresponding methods

25. 2002.11.19- SLIDE 25IS 257 - Fall 2002 Object-Oriented DBMS Basic Concepts Objects sharing the same structure and behavior are grouped into classes. –A class represents a template for a set of similar objects. –Each object is an instance of some class.

26. 2002.11.19- SLIDE 26IS 257 - Fall 2002 Object-Oriented DBMS Basic Concepts A class can be defined as a specialization of of one or more classes. –A class defined as a specialization is called a subclass and inherits attributes and methods from its superclass(es).

27. 2002.11.19- SLIDE 27IS 257 - Fall 2002 Object-Oriented DBMS Basic Concepts An OODBMS is a DBMS that directly supports a model based on the object- oriented paradigm. –Like any DBMS it must provide persistent storage for objects and their descriptions (schema). –The system must also provide a language for schema definition and and for manipulation of objects and their schema –It will usually include a query language, indexing capabilities, etc.

28. 2002.11.19- SLIDE 28IS 257 - Fall 2002 Generalization Hierarchy Employee No Name Address Date hired Date of Birth employee Contract No. Date Hired consultant Annual Salary Stock Option Salaried Hourly Rate Hourly calculateAge AllocateToContractcalculateStockBenefit calculateWage

29. 2002.11.19- SLIDE 29IS 257 - Fall 2002 Inverted File DBMS Usually similar to Hierarchic DBMS in record structure –Support for repeating groups of fields and multiple value fields All access is via inverted file indexes to DBS specified fields. Examples: ADABAS DBMS from Software AG -- used in the MELVYL system

30. 2002.11.19- SLIDE 30IS 257 - Fall 2002 Flat File DBMS Data is stored as a simple file of records. –Records usually have a simple structure May support indexing of fields in the records. –May also support scanning of the data No mechanisms for relating data between files. Usually easy to use and simple to set up

31. 2002.11.19- SLIDE 31IS 257 - Fall 2002 Intelligent Database Systems Intelligent DBS are intended to handle more than just data, and may be used in tasks involving large amounts of information where analysis and “discovery” are needed. The following is based on “Intelligent Databases” by Kamran Parsaye, Mark Chignell, Setrag Khoshafian and Harry Wong AI Expert, March 1990, v. 5 no. 3. Pp 38-47

32. 2002.11.19- SLIDE 32IS 257 - Fall 2002 Intelligent Database Systems They represent the evolution and merging of several technologies: –Automatic Information Discovery –Hypermedia –Object Orientation –Expert Systems –Conventional DBMS

33. 2002.11.19- SLIDE 33IS 257 - Fall 2002 Intelligent Database Systems Intelligent Databases Expert Systems Traditional Databases Hypermedia Automatic discovery Object Orientation

34. 2002.11.19- SLIDE 34IS 257 - Fall 2002 Intelligent Database Architecture Intelligent Database Engine High-Level User Interface High-Level Tools

35. 2002.11.19- SLIDE 35IS 257 - Fall 2002 Environment Components Data Dictionary Concept Dictionary Flexible queries Error detection Automatic Discovery

36. 2002.11.19- SLIDE 36IS 257 - Fall 2002 Intelligent Databases Data Dictionary contains the system metadata Concept Dictionary defines ‘virtual fields’ based on approximate definitions Data Analysis and discovery –Find patterns –detect errors –Process queries

37. 2002.11.19- SLIDE 37IS 257 - Fall 2002 Intelligent Databases Automatic Discovery –Data comprehension –Form Hypotheses –Make queries –View results and perhaps modify hypotheses –Repeat

38. 2002.11.19- SLIDE 38IS 257 - Fall 2002 Intelligent Databases Automatic Error Detection –Integrity Constraints –Rule systems –Analysis of data for anomalies

39. 2002.11.19- SLIDE 39IS 257 - Fall 2002 Intelligent Databases Flexible Query Processing –Approximate and “fuzzy” queries SELECT NAME, AGE, TELEPHONE FROM PERSONEL WHERE NAME = ‘Dovid Smith’ and AGE IS-CLOSE-TO 19; confidence factors –Ranked query results

40. 2002.11.19- SLIDE 40IS 257 - Fall 2002 Intelligent Databases Intelligent User Interfaces –Hyperlinked data in the data/knowledge base –Multimedia presentations –Dynamic linking of related information

41. 2002.11.19- SLIDE 41IS 257 - Fall 2002 Intelligent Databases Intelligent Database Engine –OO support –Inference features –Global optimization –Rule manager –Explanation manager –Transaction manager –Metadata manager –Access module –Multimedia manager

42. 2002.11.19- SLIDE 42IS 257 - Fall 2002 Object Relational Databases Background Object Definitions –inheritance User-defined datatypes User-defined functions

43. 2002.11.19- SLIDE 43IS 257 - Fall 2002 Object Relational Databases Began with UniSQL/X unified object- oriented and relational system Some systems (like OpenODB from HP) were Object systems built on top of Relational databases. Miro/Montage/Illustra built on Postgres. Informix Buys Illustra. (DataBlades) Oracle Hires away Informix Programmers. (Cartridges)

44. 2002.11.19- SLIDE 44IS 257 - Fall 2002 Object Relational Data Model Class, instance, attribute, method, and integrity constraints OID per instance Encapsulation Multiple inheritance hierarchy of classes Class references via OID object references Set-Valued attributes Abstract Data Types

45. 2002.11.19- SLIDE 45IS 257 - Fall 2002 Object Relational Extended SQL (Illustra) CREATE TABLE tablename {OF TYPE Typename}|{OF NEW TYPE typename} (attr1 type1, attr2 type2,…,attrn typen) {UNDER parent_table_name}; CREATE TYPE typename (attribute_name type_desc, attribute2 type2, …, attrn typen); CREATE FUNCTION functionname (type_name, type_name) RETURNS type_name AS sql_statement

46. 2002.11.19- SLIDE 46IS 257 - Fall 2002 Object-Relational SQL in ORACLE CREATE (OR REPLACE) TYPE typename AS OBJECT (attr_name, attr_type, …); CREATE TABLE OF typename;

47. 2002.11.19- SLIDE 47IS 257 - Fall 2002 Example CREATE TYPE ANIMAL_TY AS OBJECT (Breed VARCHAR2(25), Name VARCHAR2(25), Birthdate DATE); Creates a new type CREATE TABLE Animal of Animal_ty; Creates “Object Table”

48. 2002.11.19- SLIDE 48IS 257 - Fall 2002 Constructor Functions INSERT INTO Animal values (ANIMAL_TY(‘Mule’, ‘Frances’, TO_DATE(‘01-APR-1997’, ‘DD-MM- YYYY’))); Insert a new ANIMAL_TY object into the table

49. 2002.11.19- SLIDE 49IS 257 - Fall 2002 Selecting from an Object Table Just use the columns in the object… SELECT Name from Animal;

50. 2002.11.19- SLIDE 50IS 257 - Fall 2002 More Complex Objects CREATE TYPE Address_TY as object (Street VARCHAR2(50), City VARCHAR2(25), State CHAR(2), zip NUMBER); CREATE TYPE Person_TY as object (Name VARCHAR2(25), Address ADDRESS_TY); CREATE TABLE CUSTOMER (Customer_ID NUMBER, Person PERSON_TY);

51. 2002.11.19- SLIDE 51IS 257 - Fall 2002 What Does the Table Look like? DESCRIBE CUSTOMER; NAME TYPE ----------------------------------------------------- CUSTOMER_ID NUMBER PERSON NAMED TYPE

52. 2002.11.19- SLIDE 52IS 257 - Fall 2002 Inserting INSERT INTO CUSTOMER VALUES (1, PERSON_TY(‘John Smith’, ADDRESS_TY(‘57 Mt Pleasant St.’, ‘Finn’, ‘NH’, 111111)));

53. 2002.11.19- SLIDE 53IS 257 - Fall 2002 Selecting from Abstract Datatypes SELECT Customer_ID from CUSTOMER; SELECT * from CUSTOMER; CUSTOMER_ID PERSON(NAME, ADDRESS(STREET, CITY, STATE ZIP)) --------------------------------------------------------------------------------------------------- 1 PERSON_TY(‘JOHN SMITH’, ADDRESS_TY(‘57...

54. 2002.11.19- SLIDE 54IS 257 - Fall 2002 Selecting from Abstract Datatypes SELECT Customer_id, person.name from Customer; SELECT Customer_id, person.address.street from Customer;

55. 2002.11.19- SLIDE 55IS 257 - Fall 2002 Updating UPDATE Customer SET person.address.city = ‘HART’ where person.address.city = ‘Briant’;

56. 2002.11.19- SLIDE 56IS 257 - Fall 2002 Functions CREATE [OR REPLACE] FUNCTION funcname (argname [IN | OUT | IN OUT] datatype …) RETURN datatype (IS | AS) {block | external body}

57. 2002.11.19- SLIDE 57IS 257 - Fall 2002 Example Create Function BALANCE_CHECK (Person_name IN Varchar2) RETURN NUMBER is BALANCE NUMBER(10,2) BEGIN SELECT sum(decode(Action, ‘BOUGHT’, Amount, 0)) - sum(decode(Action, ‘SOLD’, amount, 0)) INTO BALANCE FROM LEDGER where Person = PERSON_NAME; RETURN BALANCE; END;

58. 2002.11.19- SLIDE 58IS 257 - Fall 2002 Example Select NAME, BALANCE_CHECK(NAME) from Worker;

59. 2002.11.19- SLIDE 59IS 257 - Fall 2002 TRIGGERS Create TRIGGER UPDATE_LODGING INSTEAD OF UPDATE on WORKER_LODGING for each row BEGIN if :old.name <> :new.name then update worker set name = :new.name where name = :old.name; end if; if :old.lodging <> … etc...

60. 2002.11.19- SLIDE 60IS 257 - Fall 2002 PostgreSQL Derived from POSTGRES –Developed at Berkeley by Mike Stonebraker and his students (EECS) starting in 1986 Postgres95 –Andrew Yu and Jolly Chen adapted POSTGRES to SQL and greatly improved the code base PostgreSQL –Name changed in 1996, and since that time the system has been expanded to support most SQL92 features

61. 2002.11.19- SLIDE 61IS 257 - Fall 2002 PostgreSQL Classes The fundamental notion in Postgres is that of a class, which is a named collection of object instances. Each instance has the same collection of named attributes, and each attribute is of a specific type. Furthermore, each instance has a permanent object identifier (OID) that is unique throughout the installation. Because SQL syntax refers to tables, we will use the terms table and class interchangeably. Likewise, an SQL row is an instance and SQL columns are attributes.

62. 2002.11.19- SLIDE 62IS 257 - Fall 2002 Creating a Class You can create a new class by specifying the class name, along with all attribute names and their types: CREATE TABLE weather ( city varchar(80), temp_lo int, -- low temperature temp_hi int, -- high temperature prcp real, -- precipitation date date );

63. 2002.11.19- SLIDE 63IS 257 - Fall 2002 PostgreSQL Postgres can be customized with an arbitrary number of user-defined data types. Consequently, type names are not syntactical keywords, except where required to support special cases in the SQL92 standard. So far, the Postgres CREATE command looks exactly like the command used to create a table in a traditional relational system. However, we will presently see that classes have properties that are extensions of the relational model.

64. 2002.11.19- SLIDE 64IS 257 - Fall 2002 PostgreSQL All of the usual SQL commands for creation, searching and modifying classes (tables) are available. With some additions… Inheritance Non-Atomic Values User defined functions and operators

65. 2002.11.19- SLIDE 65IS 257 - Fall 2002 Inheritance CREATE TABLE cities ( name text, population float, altitude int -- (in ft) ); CREATE TABLE capitals ( state char(2) ) INHERITS (cities);

66. 2002.11.19- SLIDE 66IS 257 - Fall 2002 Inheritance In Postgres, a class can inherit from zero or more other classes. A query can reference either –all instances of a class –or all instances of a class plus all of its descendants

67. 2002.11.19- SLIDE 67IS 257 - Fall 2002 Inheritance For example, the following query finds all the cities that are situated at an attitude of 500ft or higher: SELECT name, altitude FROM cities WHERE altitude > 500; +----------+----------+ |name | altitude | +----------+----------+ |Las Vegas | 2174 | +----------+----------+ |Mariposa | 1953 | +----------+----------+

68. 2002.11.19- SLIDE 68IS 257 - Fall 2002 Inheritance On the other hand, to find the names of all cities, including state capitals, that are located at an altitude over 500ft, the query is: SELECT c.name, c.altitude FROM cities* c WHERE c.altitude > 500; which returns: +----------+----------+ |name | altitude | +----------+----------+ |Las Vegas | 2174 | +----------+----------+ |Mariposa | 1953 | +----------+----------+ |Madison | 845 | +----------+----------+

69. 2002.11.19- SLIDE 69IS 257 - Fall 2002 Inheritance The "*" after cities in the preceding query indicates that the query should be run over cities and all classes below cities in the inheritance hierarchy Many of the PostgreSQL commands (SELECT, UPDATE and DELETE, etc.) support this inheritance notation using "*"

70. 2002.11.19- SLIDE 70IS 257 - Fall 2002 Non-Atomic Values One of the tenets of the relational model is that the attributes of a relation are atomic –I.e. only a single value for a given row and column Postgres does not have this restriction: attributes can themselves contain sub- values that can be accessed from the query language –Examples include arrays and other complex data types.

71. 2002.11.19- SLIDE 71IS 257 - Fall 2002 Non-Atomic Values - Arrays Postgres allows attributes of an instance to be defined as fixed-length or variable-length multi- dimensional arrays. Arrays of any base type or user-defined type can be created. To illustrate their use, we first create a class with arrays of base types. CREATE TABLE SAL_EMP ( name text, pay_by_quarter int4[], schedule text[][] );

72. 2002.11.19- SLIDE 72IS 257 - Fall 2002 Non-Atomic Values - Arrays The preceding SQL command will create a class named SAL_EMP with a text string (name), a one-dimensional array of int4 (pay_by_quarter), which represents the employee's salary by quarter and a two-dimensional array of text (schedule), which represents the employee's weekly schedule Now we do some INSERTSs; note that when appending to an array, we enclose the values within braces and separate them by commas.

73. 2002.11.19- SLIDE 73IS 257 - Fall 2002 Inserting into Arrays INSERT INTO SAL_EMP VALUES ('Bill', '{10000, 10000, 10000, 10000}', '{{"meeting", "lunch"}, {}}'); INSERT INTO SAL_EMP VALUES ('Carol', '{20000, 25000, 25000, 25000}', '{{"talk", "consult"}, {"meeting"}}');

74. 2002.11.19- SLIDE 74IS 257 - Fall 2002 Querying Arrays This query retrieves the names of the employees whose pay changed in the second quarter: SELECT name FROM SAL_EMP WHERE SAL_EMP.pay_by_quarter[1] <> SAL_EMP.pay_by_quarter[2]; +------+ |name | +------+ |Carol | +------+

75. 2002.11.19- SLIDE 75IS 257 - Fall 2002 Querying Arrays This query retrieves the third quarter pay of all employees: SELECT SAL_EMP.pay_by_quarter[3] FROM SAL_EMP; +---------------+ |pay_by_quarter | +---------------+ |10000 | +---------------+ |25000 | +---------------+

76. 2002.11.19- SLIDE 76IS 257 - Fall 2002 Querying Arrays We can also access arbitrary slices of an array, or subarrays. This query retrieves the first item on Bill's schedule for the first two days of the week. SELECT SAL_EMP.schedule[1:2][1:1] FROM SAL_EMP WHERE SAL_EMP.name = 'Bill'; +-------------------+ |schedule | +-------------------+ |{{"meeting"},{""}} | +-------------------+

77. 2002.11.19- SLIDE 77IS 257 - Fall 2002 User Defined Functions CREATE FUNCTION allows a Postgres user to register a function with a database. Subsequently, this user is considered the owner of the function CREATE FUNCTION name ( [ ftype [,...] ] ) RETURNS rtype AS {SQLdefinition} LANGUAGE 'langname' [ WITH ( attribute [,...] ) ] CREATE FUNCTION name ( [ ftype [,...] ] ) RETURNS rtype AS obj_file, link_symbol LANGUAGE 'C' [ WITH ( attribute [,...] ) ]

78. 2002.11.19- SLIDE 78IS 257 - Fall 2002 Simple SQL Function CREATE FUNCTION one() RETURNS int4 AS 'SELECT 1 AS RESULT' LANGUAGE 'sql'; SELECT one() AS answer; answer -------- 1

79. 2002.11.19- SLIDE 79IS 257 - Fall 2002 External Functions This example creates a C function by calling a routine from a user-created shared library. This particular routine calculates a check digit and returns TRUE if the check digit in the function parameters is correct. It is intended for use in a CHECK contraint. CREATE FUNCTION ean_checkdigit(bpchar, bpchar) RETURNS bool AS '/usr1/proj/bray/sql/funcs.so' LANGUAGE 'c'; CREATE TABLE product ( id char(8) PRIMARY KEY, eanprefix char(8) CHECK (eanprefix ~ '[0-9]{2} [0-9]{5}') REFERENCES brandname(ean_prefix), eancode char(6) CHECK (eancode ~ '[0-9]{6}'), CONSTRAINT ean CHECK (ean_checkdigit(eanprefix, eancode)));

80. 2002.11.19- SLIDE 80IS 257 - Fall 2002 Creating new Types CREATE TYPE allows the user to register a new user data type with Postgres for use in the current data base. The user who defines a type becomes its owner. typename is the name of the new type and must be unique within the types defined for this database. CREATE TYPE typename ( INPUT = input_function, OUTPUT = output_function, INTERNALLENGTH = { internallength | VARIABLE } [, EXTERNALLENGTH = { externallength | VARIABLE } ] [, DEFAULT = "default" ] [, ELEMENT = element ] [, DELIMITER = delimiter ] [, SEND = send_function ] [, RECEIVE = receive_function ] [, PASSEDBYVALUE ] )

81. 2002.11.19- SLIDE 81IS 257 - Fall 2002 New Type Definition This command creates the box data type and then uses the type in a class definition: CREATE TYPE box (INTERNALLENGTH = 8, INPUT = my_procedure_1, OUTPUT = my_procedure_2); CREATE TABLE myboxes (id INT4, description box);

82. 2002.11.19- SLIDE 82IS 257 - Fall 2002 Rules System CREATE RULE name AS ON event TO object [ WHERE condition ] DO [ INSTEAD ] [ action | NOTHING ] Rules can be triggered by any event (select, update, delete, etc.)