Introduction to Data, Information and Knowledge Management Dr. Bhavani Thuraisingham The University of Texas at Dallas Data, Information and Knowledge Management January 2014
Data Management l Concepts in database systems l Types of database systems l Distributed Data Management l Heterogeneous database integration l Federated data management
An Example Database System Adapted from C. J. Date, Addison Wesley, 1990
Metadata l Metadata describes the data in the database - Example: Database D consists of a relation EMP with attributes SS#, Name, and Salary l Metadatabase stores the metadata - Could be physically stored with the database l Metadatabase may also store constraints and administrative information l Metadata is also referred to as the schema or data dictionary
Functional Architecture User Interface Manager Query Manager Transaction Manager Schema (Data Dictionary) Manager (metadata) Security/ Integrity Manager File Manager Disk Manager Data Management Storage Management
DBMS Design Issues l Query Processing - Optimization techniques l Transaction Management - Techniques for concurrency control and recovery l Metadata Management - Techniques for querying and updating the metadatabase l Security/Integrity Maintenance - Techniques for processing integrity constraints and enforcing access control rules l Storage management - Access methods and index strategies for efficient access to the database
Types of Database Systems l Relational Database Systems l Object Database Systems l Deductive Database Systems l Other - Real-time, Secure, Parallel, Scientific, Temporal, Wireless, Functional, Entity-Relationship, Sensor/Stream Database Systems, etc.
Relational Database: Example Relation S: S# SNAME STATUS CITY S1 Smith 20 London S2 Jones 10 Paris S3 Blake 30 Paris S4 Clark 20 London S5 Adams 30 Athens Relation P: P# PNAME COLOR WEIGHT CITY P1 Nut Red 12 London P2 Bolt Green 17 Paris P3 Screw Blue 17 Rome P4 Screw Red 14 London P5 Cam Blue 12 Paris P6 Cog Red 19 London Relation SP: S# P# QTY S1 P1 300 S1 P2 200 S1 P3 400 S1 P4 200 S1 P5 100 S1 P6 100 S2 P1 300 S2 P2 400 S3 P2 200 S4 P2 200 S4 P4 300 S4 P5 400
Example Class Hierarchy Document Class D1 D2 Book Subclass B1 # of Chapters Volume # Print-doc-att(ID) Method1 : Journal Subclass J1 Print-doc (ID) Method2: ID Name Author Publisher
Example Composite Object Composite Document Object Section 1 Object Section 2 Object Paragraph 1 Object Paragraph 2 Object
Distributed Database System Communication Network Distributed Processor 1 DBMS 1 Data- base 1 Data- base 3 Data- base 2 DBMS 2 DBMS 3 Distributed Processor 2 Distributed Processor 3 Site 1 Site 2 Site 3
Data Distribution EMP1 SS#NameSalary 1John20 2Paul30 3James40 4Jill Mary 6Jane70 D# DnameD#MGR Jane David Peter DEPT1 SITE 1 SITE 2 EMP2 SS#NameSalary 9Mathew 70 D# 50 Dname D#MGR 50 Math John Physics DEPT2 David Peter C. Sci. English French 20 Paul
Interoperability of Heterogeneous Database Systems Database System A Database System B Network Database System C (Legacy) Transparent access to heterogeneous databases - both users and application programs; Query, Transaction processing (Relational) (Object- Oriented)
Different Data Models Node A Node B Database Relational Model Network Model Node C Database Object- Oriented Model Network Node D Database Hierarchical Model Developments: Tools for interoperability; commercial products Challenges: Global data model
Federated Database Management Database System A Database System B Database System C Cooperating database systems yet maintaining some degree of autonomy Federation F1 Federation F2
Federated Data and Policy Management Export Data/Policy Component Data/Policy for Agency A Data/Policy for Federation Export Data/Policy Component Data/Policy for Agency C Component Data/Policy for Agency B Export Data/Policy
Outline of Part I: Information Management l Information Management Framework l Information Management Overview l Some Information Management Technologies l Knowledge Management
What is Information Management? l Information management essentially analyzes the data and makes sense out of the data l Several technologies have to work together for effective information management - Data Warehousing: Extracting relevant data and putting this data into a repository for analysis - Data Mining: Extracting information from the data previously unknown - Multimedia: managing different media including text, images, video and audio - Web: managing the databases and libraries on the web
Data Warehouse Oracle DBMS for Employees Sybase DBMS for Projects Informix DBMS for Medical Data Warehouse: Data correlating Employees With Medical Benefits and Projects Could be any DBMS; Usually based on the relational data model Users Query the Warehouse
Data Mining Knowledge Mining Knowledge Discovery in Databases Data Archaeology Data Dredging Database Mining Knowledge Extraction Data Pattern Processing Information Harvesting Siftware The process of discovering meaningful new correlations, patterns, and trends by sifting through large amounts of data, often previously unknown, using pattern recognition technologies and statistical and mathematical techniques (Thuraisingham 1998)
Multimedia Information Management Video Source Scene Change Detection Speaker Change Detection Silence Detection Commercial Detection Key Frame Selection Story Segmentation Named Entity Tagging Broadcast News Editor (BNE) Broadcast News Navigator (BNN) Video and Metadata Multimedia Database Management System Web-based Search/Browse by Program, Person, Location,... Imagery Audio Closed Caption Text Segregate Video Streams Analyze and Store Video and Metadata Story GIST Theme Frame Classifier Closed Caption Preprocess Correlation Token Detection Broadcast Detection
Image Processing: Example: Change Detection: l Trained Neural Network to predict “new” pixel from “old” pixel - Neural Networks good for multidimensional continuous data - Multiple nets gives range of “expected values” l Identified pixels where actual value substantially outside range of expected values - Anomaly if three or more bands (of seven) out of range l Identified groups of anomalous pixels
Semantic Web 0 Some Challenges: Security and Privacy cut across all layers XML, XML Schemas Rules/Query Logic, Proof and Trust TRUSTTRUST Other Services RDF, Ontologies URI, UNICODE PRIVACYPRIVACY 0 Adapted from Tim Berners Lee’s description of the Semantic Web
Knowledge Management Components Components: Strategies Processes Metrics Cycle: Knowledge, Creation Sharing, Measurement And Improvement Technologies: Expert systems Collaboration Training Web Components of Knowledge Management: Components, Cycle and Technologies