Data Warehouse/Data Mart Components Concepts Characteristics

Overview Operational vs Informational Systems Data Warehouse components Data Marts

Basic Data Warehouse Architecture Copyright © 1997, Enterprise Group, Ltd. Source OLTP Systems Subset Data Marts Enterprise Data Warehouse One Version of the Truth

Operational vs. Informational Systems Information Access Today OperationalSystems OrderEntry Manf.

Operational vs. Informational Systems Information Access Today OperationalSystems InformationalSystems

Operational vs. Informational Systems Most of the advances in end-user programming have run into difficulty in actually accessing data that exists in backbone, operational data bases. Operational data bases have a very, very long life. Large operational systems are converted from one technology to a more advanced one very infrequently (typically every eight to twenty years). Therefore, why not create specific DBs whose role was to make large scale end user access easy to isolate the operational DBs, i.e. a Data Warehouse

Operational vs. Informational Systems OperationalSystems InformationalSystems Information Delivery System

Operational vs. Informational Systems OperationalSystems InformationalSystems Information Delivery System DataWarehouse

Operational vs. Informational Systems OperationalSystems InformationalSystems Information Delivery System DataWarehouse

Operational vs. Informational Systems OperationalSystems Information Delivery System DataWarehouse InformationalSystems

Operational vs. Informational Systems OperationalSystems Information Delivery System DataWarehouse InformationalSystems Notice that one of the big impacts of Data Warehousing is to eliminate large numbers of existing DSS systems! Y2000 will make this essential!!!

Operational vs. Informational Systems OperationalSystems Information Delivery System DataWarehouse InformationalSystems DataMarts

Data Mart Layer Presentation/ Desktop Access Layer Meta-data Repository Layer Warehouse Management Layer Core DW Layer Data Staging and Quality Layer Data Access Layer Operational Data Layer External Data Layer Data Feed/ Data Mining/ Indexing Layer Virtual DW Coarse DW Central DW Distributed DW Application Messaging (Transport) Layer Internet/Intranet Layer direct queries virtual queries ad hoc queries 1 2a 2b 2c Non-operational Data Layer Data Marts vs Data Warehouses

Data Mart Layer Presentation/ Desktop Access Layer Meta-data Repository Layer Warehouse Management Layer Core DW Layer Data Staging and Quality Layer Data Access Layer Operational Data Layer External Data Layer Data Feed/ Data Mining/ Indexing Layer Central DW Application Messaging (Transport) Layer Internet/Intranet Layer direct queries virtual queries ad hoc queries 1 2a 2b 2c Non-operational Data Layer Central Data Warehouse Tracking DB Lawson DB

Virtual Date Warehouse A Virtual Data Warehouse approach is often chosen when there are infrequent demands for data and management wants to determine if/how users will use operational data. One of the weaknesses of a Virtual Data Warehouse approach is that user queries a made against operational DBs. One way to minimize this problem is to build a “Query Monitor” to check the performance characteristics of a query before executing it.

A Coarse Data Warehouse is often chosen when the organization has a relatively clean/new operational system and management wants to make the operational data more easily available for just that system. A Central Data Warehouse is often chosen when the organization has a clear understanding about it Information Access needs and wants to provide “quality”, “integrated”, information to its knowledge workers A Distributed Data Warehouse is similar in most respects to a Central Data Warehouse, except that the data is distributed to separate mini-Data Warehouses (Data Marts )on local or specialized servers

Data Mart Layer Presentation/ Desktop Access Layer Meta-data Repository Layer Warehouse Management Layer Core DW Layer Data Staging and Quality Layer Data Access Layer Operational Data Layer External Data Layer Data Feed/ Data Mining/ Indexing Layer Virtual DW Coarse DW Central DW Application Messaging (Transport) Layer Distributed DW Internet/Intranet Layer direct queries virtual queries ad hoc queries 1 2a 2b 2c Non-operational Data Layer Central Data Warehouse

Data Mart Layer Presentation/ Desktop Access Layer Meta-data Repository Layer Warehouse Management Layer Core DW Layer Data Staging and Quality Layer Data Access Layer Operational Data Layer External Data Layer Data Feed/ Data Mining/ Indexing Layer Virtual DW Coarse DW Central DW Distributed DW Application Messaging (Transport) Layer Internet/Intranet Layer direct queries virtual queries ad hoc queries 1 2a 2b 2c Non-operational Data Layer Data Marts Only

Heterogeneity - The Reality Oracle Financials Custom Marketing Data Warehouse Packaged Oracle Financial Data Warehouse Packaged I2 Supply Chain Non- Architected Data Mart Subset Data Marts i2 Supply ChainSiebel CRM 3rd Party Data

Federated BI Architecture Real Time ODS Federated Financial Data Warehouse Subset Data Marts Common Staging Area Oracle Financialsi2 Supply ChainSiebel CRM 3rd Party Federated Packaged I2 Supply Chain Data Marts Analytical Applications e-commerce Real Time Data Mining and Analytics Real Time Segmentation, Classification, Qualification, Offerings, etc. Federated Marketing Data Warehouse

Benefits of Data Warehouse Architecture Provides organizing framework Gives flexibility for changes and allows simplified maintenance Speeds up future development by aiding understanding of dw Communication tool for roles and requirements Coordinate data marts

Primary Technical Challenge Axis Easy Hard Fast Slow Parallel ERP DW Finance Custom Turnkey VLDB NearRealTime Marketing Mid-Size Co. Large Co. Single Source Multi-Source MonthlyFreq Small DB Dirty Data Clean Data

Prerequisites for Success Pain driven Sponsorship at the highest levels Sustainable political will Iterative methodology Manageable scope User driven design Service business mindset Sustainability