Leaving a Metadata Trail Chapter 14

Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata

The Key to Understanding Warehouse Information Specifies data location Manage data Aids use of information Describes the data Documents the development process Provides a record of changes Records enhancements over time

Metadata Users IT developers ETT Operational End user Metadata repository Warehouse End users

Types of Metadata End user: - Key to a good warehouse - Navigation aid - Information provider ETT: - Maps structure - Source and target information - Transformations Operational: - Load, management, scheduling processes - Performance

Developing a Metadata Strategy Define a strategy to ensure high-quality metadata useful to users and developers. Primary strategy considerations: - Define goals and intended use. - Identify target users. - Choose tools and techniques. - Choose the metadata location. - Manage access to the metadata. - Integrate metadata from multiple tools. - Manage change.

Defining Metadata Goals and Intended Usage Define clear goals. Identify requirements. Identify intended usage. Metadata

Identifying Target Metadata Users Who are the metadata users? - Developers - End users What information do they need? How will they access the metadata?

Choosing Metadata Tools and Techniques Tools - Data modeling - ETT - End-user query and analysis Database schema definitions COBOL copybooks Middleware tools

Choosing the Metadata Location Usually the warehouse server Possibly on operational platforms Desktop tool with metalayer Metadata repository Warehouse End users External sources Operational Data sources

Managing the Metadata Managed by the metadata manager Maintained by the metadata architect Standards produced by the metadata architect Metadata repository Warehouse End users External sources Operational Data sources

Integrating Multiple Sets of Metadata Multiple tools may generate their own metadata. There are many metalayer integration issues. Metadata exchangeability is desirable.

Managing Changes to Metadata Different types of metadata have different rates of change. Consider metadata changes resulting from refresh cycles.

Examining Types of Metadata ETT metadata End user metadata ETT End user Metadata repository Warehouse

ETT Metadata Business rules Source tables, fields, and key values Ownership Field conversions Encoding and reference table Name changes Key value changes Default values Logic to handle multiple sources Algorithms Time stamp Extraction External sources Operational data sources Staging file

Extraction Metadata Space and storage requirements Source location information Diverse source data Access information Security Contacts Program names Frequency details Failure procedures Validity checking information Extraction External sources Operational data sources Staging file

Transformation Metadata Duplication routines Exception handling Key restructuring Grain conversions Program names Frequency Summarization Transform External sources Operational data sources Staging file

Transportation Metadata Method of transfer Frequency Validation procedures Failure procedures Deployment rules Contact information External sources Operational Data sources ETT Transport Staging file Metadata repository Warehouse

End-User Metadata Associate the metadata description Analogous to Oracle Data Dictionary views End user Metadata repository Warehouse

Example of End User Metadata Table Name Column Name DataMeaning ProductProdid739516Unique identifier for the product Product Valid_date Ware_loc 01/ Last refresh date Warehouse location number ProductWare_bin666Warehouse bin number ProductWeight17.62Packed shipping Weight in kilograms

More End-User Metadata Information Location of fact and dimensions Availability Description of contents Algorithms for derived and summary data Owners of data and telephone number End user Metadata repository Warehouse

Historic Context of Data Supports change history Maintains the context of information OperationalWarehouse Content Structure repository Metadata

Types of Context Simple: - Data structure - Naming conventions - Metrics Complex: - Product definitions - Markets - Pricing External: - Economic - Political Warehouse

Additional Metadata Content and Considerations Summarization algorithms Relationships Stewardship Permissions Pattern analysis Reference tables

Metadata Management Tools Carleton Evolutionary Technologies Hewlett Packard Informatics Oracle Designer Platinum Technology Prism Solutions Sagent

Common Warehouse Metadata Design and Administratioon

Common Warehouse Metadata Future Warehouse Builder Oracle8i Server Discover Express Server Common metadata

Summary This lesson discussed the following topics: Definitions Integration Contents Storage Creation Selection Tools