CISB594 – Business Intelligence

Slides:



Advertisements
Similar presentations
Chapter 13 The Data Warehouse
Advertisements

Database Management3-1 L3 Database Management Santa R. Susarapu Ph.D. Student Virginia Commonwealth University.
April 30, Data Warehousing and OLAP Technology: An Overview  What is a data warehouse?  Data warehouse architecture  From data warehousing to.
ICS 421 Spring 2010 Data Warehousing (1) Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa 3/18/20101Lipyeow.
Data Warehouse IMS5024 – presented by Eder Tsang.
CISB594 – Business Intelligence
Chapter 5 DATA WAREHOUSING.
Data Warehousing: Defined and Its Applications Pete Johnson April 2002.
Components of the Data Warehouse Michael A. Fudge, Jr.
Basic Concepts of Datawarehousing An Overview Prasanth Gurram.
Decision Support Systems Data Warehousing Chattrakul Sombattheera.
Database Systems – Data Warehousing
Ihr Logo Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization Turban, Aronson, and Liang.
DW-1: Introduction to Data Warehousing. Overview What is Database What Is Data Warehousing Data Marts and Data Warehouses The Data Warehousing Process.
Data Warehouse Overview September 28, 2012 presented by Terry Bilskie.
AN OVERVIEW OF DATA WAREHOUSING
Business Intelligence Zamaneh Jahed. What is Business Intelligence? Business Intelligence (BI) is a broad category of applications and technologies for.
© 2005 Prentice Hall, Decision Support Systems and Intelligent Systems, 7th Edition, Turban, Aronson, and Liang 5-1 Chapter 5 Business Intelligence: Data.
© 2007 by Prentice Hall 1 Introduction to databases.
Data warehousing and online analytical processing- Ref Chap 4) By Asst Prof. Muhammad Amir Alam.
Data Warehouse Fundamentals Rabie A. Ramadan, PhD 2.
1 Data Warehouses BUAD/American University Data Warehouses.
2 Copyright © Oracle Corporation, All rights reserved. Defining Data Warehouse Concepts and Terminology.
OLAP & DSS SUPPORT IN DATA WAREHOUSE By - Pooja Sinha Kaushalya Bakde.
1 Reviewing Data Warehouse Basics. Lessons 1.Reviewing Data Warehouse Basics 2.Defining the Business and Logical Models 3.Creating the Dimensional Model.
CISB594 – Business Intelligence Business Analytics and Data Visualization Part II.
5 - 1 Copyright © 2006, The McGraw-Hill Companies, Inc. All rights reserved.
October 28, Data Warehouse Architecture Data Sources Operational DBs other sources Analysis Query Reports Data mining Front-End Tools OLAP Engine.
Building Data and Document-Driven Decision Support Systems How do managers access and use large databases of historical and external facts?
MANAGING DATA RESOURCES ~ pertemuan 7 ~ Oleh: Ir. Abdul Hayat, MTI.
1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed.
Ch3 Data Warehouse Dr. Bernard Chen Ph.D. University of Central Arkansas Fall 2009.
CISB594 – Business Intelligence Data Warehousing Part I.
Data Warehouses and OLAP Data Management Dennis Volemi D61/70384/2009 Judy Mwangoe D61/73260/2009 Jeremy Ndirangu D61/75216/2009.
CISB594 – Business Intelligence Data Warehousing Part I.
 Understand the basic definitions and concepts of data warehouses  Describe data warehouse architectures (high level).  Describe the processes used.
Chapter 5 DATA WAREHOUSING Study Sections 5.2, 5.3, 5.5, Pages: & Snowflake schema.
CISB594 – Business Intelligence Data Warehousing Part I.
DATA RESOURCE MANAGEMENT
CISB594 – Business Intelligence Business Analytics and Data Visualization Part I.
Business Intelligence Transparencies 1. ©Pearson Education 2009 Objectives What business intelligence (BI) represents. The technologies associated with.
CISB594 – Business Intelligence Data Warehousing Part I.
Advanced Database Concepts
 Understand the basic definitions and concepts of data warehouses  Describe data warehouse architectures (high level).  Describe the processes used.
1 Copyright © Oracle Corporation, All rights reserved. Business Intelligence and Data Warehousing.
The Need for Data Analysis 2 Managers track daily transactions to evaluate how the business is performing Strategies should be developed to meet organizational.
Chapter 8: Data Warehousing. Data Warehouse Defined A physical repository where relational data are specially organized to provide enterprise- wide, cleansed.
Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 5: Data Warehousing.
Data Warehouse – Your Key to Success. Data Warehouse A data warehouse is a  subject-oriented  Integrated  Time-variant  Non-volatile  Restructure.
Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 8: Data Warehousing.
2 Copyright © 2006, Oracle. All rights reserved. Defining Data Warehouse Concepts and Terminology.
Data Mining and Data Warehousing: Concepts and Techniques What is a Data Warehouse? Data Warehouse vs. other systems, OLTP vs. OLAP Conceptual Modeling.
Business Intelligence Overview
Advanced Applied IT for Business 2
Defining Data Warehouse Concepts and Terminology
Data warehouse.
Data warehouse and OLAP
Data Warehouse—Subject‐Oriented
Data Warehouse.
Defining Data Warehouse Concepts and Terminology
MANAGING DATA RESOURCES
Data Warehouse and OLAP
Data Warehouse Overview September 28, 2012 presented by Terry Bilskie
C.U.SHAH COLLEGE OF ENG. & TECH.
Dr. Bernard Chen Ph.D. University of Central Arkansas Fall 2009
Data Warehouse.
Chapter 3 DATA WAREHOUSING.
Data Warehousing Concepts
Data Warehouse and OLAP
Data Warehouse and OLAP Technology
Presentation transcript:

CISB594 – Business Intelligence Data Warehouse Part I

Reference Materials used in this presentation are extracted mainly from the following texts, unless stated otherwise.

Objectives At the end of this lecture, you should be able to: Understand the basic definitions and concepts of data warehouses Understand how a data warehouse differs from an operational database Describe the characteristics of data warehouse Describe data warehouse process overview Describe the different types of data warehouse architectures CISB594 – Business Intelligence

Data Warehouse “The data warehouse is a collection of integrated, subject-oriented databases designed to support DSS functions, where each unit of data is non-volatile and relevant to some moment in time” (Inmon) A copy of transaction data specifically structured for query and analysis (Kimball) A data warehouse is a repository of an organization's electronically stored data, designed to facilitate reporting and analysis . (Wikipedia)

Data Warehouse A decision support database that is maintained separately from the organization’s operational database Support information processing by providing a solid platform of consolidated, historical data for analysis In your own words?

4 main characteristics of data warehousing Subject oriented Organized around major subjects, such as sales progress Containing only information relevant for decision support Focusing on the modeling and analysis of data for decision makers, not on daily operations or transaction processing Provide a simple and concise view around particular subject issues

4 main characteristics of data warehousing Subject oriented For example, to learn more about your company's sales, you can build a warehouse that concentrates on sales. Using this warehouse, you can answer questions like "Who was our best customer for this item last year?" This ability to define a data warehouse by subject matter, sales in this case, makes the data warehouse subject oriented (http://docs.oracle.com/)

4 main characteristics of data warehousing Integrated Constructed by integrating multiple, various data sources Must place data from different sources into a consistent format, to do so they must deal with naming conflict and discrepancies Data cleaning and data integration techniques are applied Ensure consistency in naming conventions among different data sources When data is moved to the warehouse, it is converted

4 main characteristics of data warehousing 3. Time variant (time series) maintains historical data, data for analysis from multiple sources contain multiple time points A data warehouse's focus on change over time The time horizon for the data warehouse is significantly longer than that of operational systems Operational database: current value data Data warehouse data: provide information from a historical perspective (e.g., past 5-10 years)

4 main characteristics of data warehousing 4. Non-volatile after data are entered into a data warehouse, users cannot change or update the data. Operational update of data does not occur in the data warehouse environment Does not require transaction processing, recovery, and concurrency control mechanisms Requires only two operations in data accessing: Initial loading of data and access of data

Summary of Data Warehouse Runs on a DBMS such as Oracle, SQL, DB2 … Keeps a large amount of data from different time for a long period of time Data in data warehouse cannot be overwritten by users Data comes from various sources, internally and externally Carefully designed to allow for analysis/ pattern discovery on identified subject matter

OLTP OLTP (on-line transaction processing) Major task of traditional relational DBMS Day-to-day operations: purchasing, inventory, banking, manufacturing, payroll, registration, accounting, etc. Database type : Operational

OLAP Online Analytical Processing (OLAP) is a reporting application that provides high-performance analysis and easy reporting on large volumes of data The goal of OLAP: multidimensional data analysis, provide fast and flexible data summarization, analysis, and reporting capabilities ability to view trends over time Type of database : Data warehouse

OLTP vs OLAP OLTP OLAP Users Clerk, IT professional Knowledge worker Function Day to day operations Decision support DB Design To suit typical database function of update, edit, delete, relational Designed for reporting on Subjects, datawarehouse Data Current, up-to-date detailed, Historical, summarized, multidimensional, integrated, consolidated Usage Repetitive, structured Ad-hoc, un-structured Access Read/write Read. Lots of scans Type of Work Short, simple transaction Complex query # Records Accessed Tens Millions # Users Thousands Hundreds, Tens DB Size 100MB-GB 100GB-TB

How the database looks like for the two types The operational database (relational):

How the database looks like for the two types The datawarehouse (star schema):

Why … Can we not operate on operational database to obtain the answers to our business questions? Answer : require complex query formulation, preparation of data to address the query and if use the operational database, the process will be very slow due to complex joins and multiple scans A typical data warehouse query scans thousands or millions of rows. For example, "Find the total sales for all customers last month." A typical OLTP operation accesses only a handful of records. For example, "Retrieve the current order for this customer."

Ask yourself Explain data warehouse. How does it differ from operational database? Provide an example to support your answer Explain the 4 main characteristics of data warehouse Compare and contrast OLAP to OLTP

Data Warehousing - Concept Data mart Smaller and focuses on a particular subject or department. It is a subset of data warehouse/departmental data warehouse A data mart is a smaller DW designed around one problem, organizational function, topic, or other focus area. Can be Dependent data mart A subset that is created directly from a data warehouse Ensures that the end user is viewing the same version of the data that are accessed by all other data warehouse users Or Independent data mart A small data warehouse designed for a strategic business unit or a department

Data Warehousing - Concept Enterprise data warehouse (EDW) A large scale data warehouse used across the enterprise for decision support Used to provide data for many types of DSS, including CRM, supply chain management, BPM, KMS etc Metadata Data about data. In a data warehouse, metadata describe the contents of a data warehouse and the manner of its use. Metadata in layman term : Metadata describes other data. It provides information about a certain item's content. For example, an image may include metadata that describes how large the picture is, the color depth, the image resolution, when the image was created, and other data http://www.techterms.com/definition/metadata

Data Warehousing Process Overview The data warehousing process consists of the following steps: 1. Data are imported from various internal and external sources 2. Data are cleansed and organized consistently with the organization’s needs 3a. Data are loaded into the enterprise data warehouse 4a.If desired, data marts are created as subsets of the EDW —or— 3b.Data are loaded into data marts 4b.The data marts are consolidated into the EDW Analyses are performed as needed

Data Warehousing - Process Overview The major components of a data warehousing process Data sources. Data are sourced from operational systems and possibly from external data sources. Data extraction. Data are extracted using custom-written or commercial software called ETL. Data loading. Data are loaded into a staging area, where they are transformed and cleansed. The data are then ready to load into the data warehouse. Data warehouse/Comprehensive database. This is the EDW that supports decision analysis by providing relevant summarized and detailed information. Middleware tools. Middleware tools enable access to the data warehouse from a variety of front-end applications.

Data Warehousing - Process Overview

Data Warehousing Architectures There are several basic architectures for data warehousing To distinguished the architectures data warehouse is divided into three parts: The data warehouse itself Data acquisition (back-end) software, which extracts data from legacy systems and external sources, consolidates and loads into the data warehouse Client (front-end) software, which allows users access and analyze data from the warehouse

Data Warehousing Architectures CISB594 – Business Intelligence

Data Warehousing Architectures Factors that potentially affect the architecture selection decision: 4. Constraints on resources, funding 5. Strategic view of the data warehouse prior to implementation 6. Compatibility with existing systems 7. Perceived ability of the in-house IT staff 8. Technical issues, technology 9. Social/political factors/nature of users Information interdependence between organizational units Upper management’s information needs Urgency of need for a data warehouse CISB594 – Business Intelligence

Now ask if .. You are able to: Understand the basic definitions and concepts of data warehouses Understand how a data warehouse differs from a database Describe the characteristics of data warehouse Describe data warehouse process overview CISB594 – Business Intelligence