1. Data warehouse

2. Definition
Data Warehouse A collection of corporate information, derived directly from operational systems and some external data sources. Its specific purpose is to support business decisions, not business operations.
Key Concept :
DWH getting the data out of the source systems – standardize it, cleanse it, store it in a common spot.
BI is transforming this data into a format that is consumable by business people.
Heavy lifting

3. The Purpose of Data Warehousing
Realize the value of data
Data / information is an asset
Methods to realize the value, (Reporting, Analysis, etc.)
Make better decisions
Turn data into information
Create competitive advantage
Methods to support the decision making process, (EIS, DSS, etc.)

4. Data Warehouse Components
Staging Area
A preparatory repository where transaction data can be transformed for use in the data warehouse
Data Mart
Traditional dimensionally modeled set of dimension and fact tables
Per Kimball, a data warehouse is the union of a set of data marts
Operational Data Store (ODS)
Modeled to support near real-time reporting needs.

5. Data Warehouse Functionality
Relational Databases
Legacy Data
Purchased Data
ERP
Systems
Analyze Query
Data Warehouse Engine
Optimized Loader
Extraction Cleansing
Metadata Repository
Legacy data is historical data
The working information of a staff member
Working hours or time-off hours within the fiscal period, up to the current date
Working Hours = Overtime, etc.
Time-Off Hours = Vacation, Sick Leave, etc.

6. Evolution architecture of data warehouse
Top-Down Architecture
Bottom-Up Architecture
Enterprise Data Mart Architecture
Data Stage/Data Mart Architecture
GO TO DIAGRAM
GO TO DIAGRAM
GO TO DIAGRAM
DataStage database, tool A tool set for designing, developing, and runnin.g applications that populate one or more tables in a data warehouse
GO TO DIAGRAM

7. Very Large Data Bases Warehouses are Very Large Databases
Terabytes -- 10^12 bytes:
Petabytes -- 10^15 bytes:
Exabytes -- 10^18 bytes:
Zettabytes -- 10^21 bytes:
Zottabytes -- 10^24 bytes:
Wal-Mart Terabytes
Geographic Information Systems
National Medical Records
Weather images
Intelligence Agency Videos

8. Complexities of Creating a Data Warehouse
Incomplete errors
Missing Fields
Records or Fields That, by Design, are not Being Recorded
Incorrect errors
Wrong Calculations, Aggregations
Duplicate Records
Wrong Information Entered into Source System
Quick review

9. SUCCESS & FUTURE OF DATA WAREHOUSE
The Data Warehouse has successfully supported the increased needs over the past many years.
The need for growth continues however, as the desire for more integrated data increases.
The Data Warehouse has software and tools in place to provide the functionality needed to support new enterprise Data Warehouse projects.
The future capabilities of the Data Warehouse can be expanded to include other programs and agencies.

10. Data Warehouse Pitfalls
You are going to spend much time extracting, cleaning, and loading data
You are going to find problems with systems feeding the data warehouse
You will find the need to store/validate data not being captured/validated by any existing system
Large scale data warehousing can become an exercise in data homogenizing

11. Data Warehouse Pitfalls…
The time it takes to load the warehouse will expand to the amount of the time in the available window... and then some
You are building a HIGH maintenance system
You will fail if you concentrate on resource optimization to the neglect of project, data, and customer management issues and an understanding of what adds value to the customer

12. Best Practices Complete requirements and design
Prototyping is key to business understanding
Utilizing proper aggregations and detailed data
Training is an on-going process
Build data integrity checks into your system.
Quick Review

13. Thank You

14. Top-Down Architecture
BACK TO ARCHITECTURE

15. Bottom-Up Architecture
BACK TO ARCHITECTURE

16. Enterprise Data Mart Architecture
BACK TO ARCHITECTURE

17. Data Stage/Data Mart Architecture
BACK TO ARCHITECTURE