1. Data and Applications Security Developments and Directions Dr. Bhavani Thuraisingham The University of Texas at Dallas Lecture #2 Supporting Technologies: Data Management January 13, 2005

2. Objective of the Unit l This unit will provide an overview of the concepts and developments in data management l Reference: Data Management Systems: Evolution and Interoperation, Thuraisingham, CRC Press, 1997

3. Outline of the Unit l Concepts in database systems l Types of database systems l Distributed Data Management l Heterogeneous database integration l Federated data management

4. Concepts in Database Systems l Definition of a Database system l Early systems l Metadata l Architectural Issues - Schema, Functional l DBMS Design Issues l Other Issues - Database design, Administration

5. Database System l Consists of database, hardware, Database Management System (DBMS), and users l Database is the repository for persistent data l Hardware consists of secondary storage volumes, processors, and main memory l DBMS handles all users’ access to the database l Users include application programmers, end users, and the Database Administrator (DBA) l Need: Reduced redundancy, avoids inconsistency, ability to share data, enforce standards, apply security restrictions, maintain integrity, balance conflicting requirements l We have used the definition of a database management system given in C. J. Date’s Book (Addison Wesley, 1990)

6. An Example Database System Adapted from C. J. Date, Addison Wesley, 1990

7. Metadata l Metadata describes the data in the database - Example: Database D consists of a relation EMP with attributes SS#, Name, and Salary l Metadatabase stores the metadata - Could be physically stored with the database l Metadatabase may also store constraints and administrative information l Metadata is also referred to as the schema or data dictionary

8. Three-level Schema Architecture: Details External Schema A External Schema B Conceptual Schema Internal Schema User A1 User A2User A3User B1 User B2 External Model A External Model B Conceptual Model Stored Database Internal Model External/Conceptual Mapping B External/Conceptual Mapping A Conceptual/Internal Mapping

9. Functional Architecture User Interface Manager Query Manager Transaction Manager Schema (Data Dictionary) Manager (metadata) Security/ Integrity Manager File Manager Disk Manager Data Management Storage Management

10. DBMS Design Issues l Query Processing - Optimization techniques l Transaction Management - Techniques for concurrency control and recovery l Metadata Management - Techniques for querying and updating the metadatabase l Security/Integrity Maintenance - Techniques for processing integrity constraints and enforcing access control rules l Storage management - Access methods and index strategies for efficient access to the database

11. Other Issues l Database design - Generally a two-step process l Semantic data model to capture the entities of the application and the relationships between the entities l Generate the conceptual schema; theory of normal forms for relational databases - Research on object-oriented approaches for database design l Database Administration - Creating and deleting databases; backup and recovery, enforcing policies, auditing, etc.

12. Types of Database Systems l Relational Database Systems l Object Database Systems l Deductive Database Systems l Other - Real-time, Secure, Parallel, Scientific, Temporal, Wireless, Functional, Entity-Relationship, Sensor/Stream Database Systems, etc.

13. Relational Database: Informal Overview l Collection of tables also called relations l Table has one or more columns also called attributes l Each table has zero or more rows also called tuples l Elements of a row take values from a pool of legal values l The values of one or more columns in a row uniquely identify the row. These columns form an identifier (also called key) l One identifier is designated as the unique identifier (also called primary key) l Querying relational databases using language called SQL (Structured Query Language)

14. Relational Database: Example Relation S: S# SNAME STATUS CITY S1 Smith 20 London S2 Jones 10 Paris S3 Blake 30 Paris S4 Clark 20 London S5 Adams 30 Athens Relation P: P# PNAME COLOR WEIGHT CITY P1 Nut Red 12 London P2 Bolt Green 17 Paris P3 Screw Blue 17 Rome P4 Screw Red 14 London P5 Cam Blue 12 Paris P6 Cog Red 19 London Relation SP: S# P# QTY S1 P1 300 S1 P2 200 S1 P3 400 S1 P4 200 S1 P5 100 S1 P6 100 S2 P1 300 S2 P2 400 S3 P2 200 S4 P2 200 S4 P4 300 S4 P5 400

15. Concepts in Object Database Systems l Objects- every entity is an object - Example: Book, Film, Employee, Car l Class - Objects with common attributes are grouped into a class l Attributes or Instance Variables - Properties of an object class inherited by the object instances l Class Hierarchy - Parent-Child class hierarchy l Composite objects - Book object with paragraphs, sections etc. l Methods - Functions associated with a class

16. Example Class Hierarchy Document Class D1 D2 Book Subclass B1 # of Chapters Volume # Print-doc-att(ID) Method1 : Journal Subclass J1 Print-doc (ID) Method2: ID Name Author Publisher

17. Example Composite Object Composite Document Object Section 1 Object Section 2 Object Paragraph 1 Object Paragraph 2 Object

18. Deductive Database Systems l Database systems augmented with inference engines to deduce new data from existing data and rules l Example - Rule: parent of a parent is a grandparent - Data: John is Jane’s parent; Jane is Robert’s parent - From the above, infer John is Robert’s grandparent l Loose and tight coupling architectures between the database system and inference engine

19. A Definition of a Distributed Database System l A collection of database systems connected via a network l The software that is responsible for interconnection is a Distributed Database Management System (DDBMS) l Each DBMS executes local applications and should be involved in at least one global application (Ceri and Pelagetti) l Homogeneous environment

20. Architecture Communication Network Distributed Processor 1 DBMS 1 Data- base 1 Data- base 3 Data- base 2 DBMS 2 DBMS 3 Distributed Processor 2 Distributed Processor 3 Site 1 Site 2 Site 3

21. Distributed Processor Distributed Query/Update Processor Distributed Transaction Manager Distributed Metadata Management Network Interface Local DBMS Interface Integrity/ Security Manager

22. Data Distribution EMP1 SS#NameSalary 1John20 2Paul30 3James40 4Jill50 60 5Mary 6Jane70 D# 10 20 20 20 10 20 DnameD#MGR 10 30 40 Jane David Peter DEPT1 SITE 1 SITE 2 EMP2 SS#NameSalary 9Mathew 70 D# 50 Dname D#MGR 50 Math John Physics DEPT2 David 80 30 Peter9040 7 8 C. Sci. English French 20 Paul

23. Distributed Database Functions l Distributed Query Processing - Optimization techniques across the databases l Distributed Transaction Management - Techniques for distributed concurrency control and recovery l Distributed Metadata Management - Techniques for managing the distributed metadata l Distributed Security/Integrity Maintenance - Techniques for processing integrity constraints and enforcing access control rules across the databases

24. DBMS 1 DQP DBMS 2 DQP DBMS 3 EMP1 (20) EMP2 (30) DEPT2 (20) EMP1 (20) EMP3 (50) DEPT3 (30) Network Query at site 1: Join EMP and DEPT on D# Move EMP2 to site 3; Merge EMP1, EMP2, EMP3 to form EMP Move DEPT2 to site 3; Merge DEPT2 and DEPT3 to form DEPT Join EMP and DEPT; Move result to site 1 Query Processing Example (Concluded) DQP (Distributed Query Processor)

25. Transaction Processing Example Site 1 Coordinator Transaction Tj Site 2 Participant Site 3 Participant Site 4 Participant Subtransaction Tj2 Subtransaction Tj3 Subtransaction Tj4 Issues: Concurrency control Recovery Data Replication Two-phase commit: Coordinator queries participants whether they are ready to commit If all participants agree, then coordinator sends request for the participants to commit DTM (Distributed Transaction Manager) responsible for executing the distributed transaction

26. Interoperability of Heterogeneous Database Systems Database System A Database System B Network Database System C (Legacy) Transparent access to heterogeneous databases - both users and application programs; Query, Transaction processing (Relational) (Object- Oriented)

27. Technical Issues on the Interoperability of Heterogeneous Database Systems l Heterogeneity with respect to data models, schema, query processing, query languages, transaction management, semantics, integrity, and security policies l Interoperability based on client-server architectures l Federated database management - Collection of cooperating, autonomous, and possibly heterogeneous component database systems, each belonging to one or more federations

28. Different Data Models Node A Node B Database Relational Model Network Model Node C Database Object- Oriented Model Network Node D Database Hierarchical Model Developments: Tools for interoperability; commercial products Challenges: Global data model

29. Schema Integration and Transformation: An approach Schema describing the network database Schema describing the hierarchical database Schema describing the object-oriented database Global Schema: Integrate the generic schemas External Schema I External Schema II External Schema III Schema describing the relational database Generic schema describing the relational database Generic schema describing the network database Generic schema describing the hierarchical database Generic schema describing the object-oriented database Challenges: Selecting appropriate generic representation; maintaining consistency during transformations; schema evolution

30. Semantic Heterogeneity l Semantic heterogeneity occurs when there is a disagreement about the meaning or interpretation of the same data Object O Node A Node B Database Object O interpreted as a passenger ship Object O interpreted as a submarine Challenges: Standard definitions; Repositories

31. Federated Database Management Database System A Database System B Database System C Cooperating database systems yet maintaining some degree of autonomy Federation F1 Federation F2

32. Autonomy Component A Component B Component C local request request from component communication through federation component A does not communicate with component C component A honors the local request first Challenges: Adapt techniques to handle autonomy - e.g., transaction processing, schema integration; transition research to products

33. Schema Integration and Transformation in a Federated Environment Adapted from Sheth and Larson, ACM Computing Surveys, September 1990 Component Schema for Component A Component Schema for Component B Component Schema for Component C Local Schema 1 Local Schema 2 Generic Schema for Component A Generic Schema for Component B Generic Schema for Component C Export Schema for Component A Export Schema I for Component B Export Schema for Component C Federated Schema for FDS - 1 Federated Schema for FDS - 2 External Schema 1.2Schema 2.1 External Schema 2.2 External Schema 1.1 Export Schema II for Component B External

34. Federated Data and Policy Management Export Data/Policy Component Data/Policy for Agency A Data/Policy for Federation Export Data/Policy Component Data/Policy for Agency C Component Data/Policy for Agency B Export Data/Policy

35. Current Status and Directions l Developments - Several prototypes and some commercial products - Tools for schema integration and transformation - Standards for interoperable database systems l Challenges being addressed - Semantic heterogeneity - Autonomy and federation - Global transaction management - Integrity and Security l New challenges - Scale - Web data management