1. Introduction to Data Warehousing

2. Why need a Warehouse?

3. Problem: Heterogeneous Information Sources
“Heterogeneities are everywhere”
Personal
Databases
World
Wide
Web
Scientific Databases
Digital Libraries
Different interfaces
Different data representations
Duplicate and inconsistent information
CS 336

4. Problem: Data Management in Large Enterprises
Vertical fragmentation of informational systems (vertical stove pipes)
Result of application (user)-driven development of operational systems
Sales Planning
Suppliers
Num. Control
Stock Mngmt
Debt Mngmt
Inventory
...
...
...
Sales Administration
Finance
Manufacturing
...
CS 336

5. Goal: Unified Access to Data
Integration System
World
Wide
Web
Personal
Databases
Digital Libraries
Scientific Databases
Collects and combines information
Provides integrated view, uniform user interface
Supports sharing
CS 336

6. Why a Warehouse? ? Two Approaches: Query-Driven (Lazy)
Warehouse (Eager) ?
Source
Source
CS 336

7. The Traditional Research Approach
Query-driven (lazy, on-demand)
Clients
Integration System
Metadata
. . .
Wrapper
Wrapper
Wrapper
. . .
Source
Source
Source
CS 336

8. Disadvantages of Query-Driven Approach
Delay in query processing
Slow or unavailable information sources
Complex filtering and integration
Inefficient and potentially expensive for frequent queries
Competes with local processing at sources
Hasn’t caught on in industry
CS 336

9. The Warehousing Approach
Information integrated in advance
Stored in wh for direct querying and analysis
Clients
Data
Warehouse
Integration System
Metadata
. . .
Extractor/
Monitor
Extractor/
Monitor
Extractor/
Monitor
. . .
Source
Source
Source
CS 336

10. Advantages of Warehousing Approach
High query performance
But not necessarily most current information
Doesn’t interfere with local processing at sources
Complex queries at warehouse
OLTP at information sources
Information copied at warehouse
Can modify, annotate, summarize, restructure, etc.
Can store historical information
Security, no auditing
Has caught on in industry
CS 336

11. Not Either-Or Decision
Query-driven approach still better for
Rapidly changing information
Rapidly changing information sources
Truly vast amounts of data from large numbers of sources
Clients with unpredictable needs
CS 336

12. Concept of Data Warehouse

13. What is a Data Warehouse? A Practitioners Viewpoint
“A data warehouse is simply a single, complete, and consistent store of data obtained from a variety of sources and made available to end users in a way they can understand and use it in a business context.”
-- Barry Devlin, IBM Consultant
CS 336

14. What is a Data Warehouse? An Alternative Viewpoint
“A DW is a
subject-oriented,
integrated,
time-varying,
non-volatile
collection of data that is used primarily in organizational decision making.”
-- W.H. Inmon, Building the Data Warehouse, 1992
CS 336

15. What is a Data Warehouse? An Alternative Viewpoint
“Data warehousing is really a simple concept: Take all the data you already have in the organization, clean and transform it, and then provide useful strategic information.”
-- Paulraj Ponniah, Data Warehousing Fundamental.

16. What is a Data Warehouse? An Alternative Viewpoint
A data warehouse is a database designed to enable business intelligence activities: it exists to help users understand and enhance their organization's performance.
It is designed for query and analysis rather than for transaction processing, and usually contains historical data derived from transaction data, but can include data from other sources.
-- Paul Lane, Data Warehousing Guide oracle12c, 2014

17. Concept of OLTP & OLAP

18. OLTP vs. OLAP OLTP: On Line Transaction Processing
Describes processing at operational sites
OLAP: On Line Analytical Processing
Describes processing at warehouse
Advantage of data warehouse:
With a data warehouse you separate analysis workload from transaction workload. This enables far better analytical performance and avoids impacting your transaction systems. (Paul Lane)

19. Warehouse is a Specialized DB
Standard DB (OLTP)
Mostly updates
Many small transactions
Mb - Gb of data
Current snapshot
Index/hash on p.k.
Raw data
Thousands of users (e.g., clerical users)
Warehouse (OLAP)
Mostly reads
Queries are long and complex
Gb - Tb of data
History
Lots of scans
Summarized, reconciled data
Hundreds of users (e.g., decision-makers, analysts)
CS 336

20. Concept of Data Mart

21. Data Mart
(Paul Lane)

22. Independent Data Mart

23. Dependent Data Mart (Paul Lane)
Salah satu contoh arsitektur DW yang menggunakan Data Mart
(Paul Lane)

24. Example Data Mart (Inmon, 2002) Perbedaan contoh data yang disimpan
pada setiap level data
(Inmon, 2002)

25. Characteristic of Data Warehouse

26. Subject-oriented What is the subject areas of the university ? CS 336
The subject orientation of the data warehouse is shown in Figure 2.1. Classical operations systems are organized around the applications of the company. For an insurance company, the applications may be auto, health, life, and casualty.
The major subject areas of the insurance corporation might be customer, policy, premium, and claim.
For a manufacturer, the major subject areas might be product, order, vendor, bill of material, and raw goods. Each type of company has its own unique set of subjects.
Mahasiswa, dosen, fakultas, akademik.
Student
Lecturer
Faculty
Academic
What is the subject areas of the university ?
CS 336

27. Integrated
Data is fed from multiple disparate sources into the data warehouse.
As the data is fed it is converted, reformatted, resequenced, summarized, and so forth.
CS 336

28. Non-Volatile
Operational data is regularly accessed and manipulated one record at a time. But the history of data is kept in the data warehouse.
CS 336

29. Time-Variant
Time variancy implies that every unit of data in the data warehouse is accurate as of some one moment in time. In some cases, a record is time stamped. In other cases, a record has a date of transaction. But in every case, there is some form of time marking to show the moment in time during which the record is accurate. Figure 2.4 illustrates how time variancy of data warehouse data can show up in several ways.
CS 336

30. A Data Warehouse is... Stored collection of diverse data
A solution to data integration problem
Single repository of information
Subject-oriented
Organized by subject, not by application
Used for analysis, data mining, etc.
Optimized differently from transaction-oriented db
User interface aimed at executive
CS 336

31. … Cont’d Large volume of data (Gb, Tb) Non-volatile Updates infrequent
Historical
Time attributes are important
Updates infrequent
May be append-only
Examples
All transactions ever at Sainsbury’s
Complete client histories at insurance firm
LSE financial information and portfolios
CS 336

32. Warehouse Architecture

33. Generic Warehouse Architecture
Client
Client
Query & Analysis
Loading
Design Phase
Warehouse
Metadata
Maintenance
Optimization
Integrator
Extractor/
Monitor
Extractor/
Monitor
Extractor/
Monitor
...
CS 336

34. Data Warehouse Architectures: Conceptual View
“Real-time data”
Operational
systems
Informational
Single-layer
Every data element is stored once only
Virtual warehouse
Two-layer
Real-time + derived data
Most commonly used approach in
industry today
Derived Data
Real-time data
Operational
systems
Informational
CS 336

35. Three-layer Architecture: Conceptual View
Transformation of real-time data to derived data really requires two steps
Operational
systems
Informational
systems
View level
“Particular informational
needs”
Derived Data
Physical Implementation
of the Data Warehouse
Reconciled Data
Real-time data
CS 336

36. Data Warehouse Issues

37. Data Warehousing: Two Distinct Issues
(1) How to get information into warehouse
“Data warehousing”
(2) What to do with data once it’s in warehouse
“Warehouse DBMS”
Both rich research areas
Industry has focused on (2)
CS 336

38. Issues in Data Warehousing
Warehouse Design
Extraction
Wrappers, monitors (change detectors)
Integration
Cleansing & merging
Warehousing specification & Maintenance
Optimizations
Miscellaneous (e.g., evolution)
CS 336

39. Literatur Slide presentasi diadaptasi dari Enrico Franconi CS 636
Beberapa literatur tambahan dari buku sesuai silabus mata kuliah Datawarehouse

40. Question ?
Kapan diperlukan data warehouse ?

41. Tugas
Carilah jurnal/ paper yang membahas implementasi “data warehouse” kemudian dicetak A4.
Kerjakan di kertas folio bergaris:
Tuliskan definisi DW yang ada pada jurnal tersebut.
Sebutkan latar belakang masalah yang ada pada jurnal sehingga DW diperlukan.
Kelebihan / Hasil DW