Logical Data Models for Agile BI David D. Schoeff Teradata - EDW Data Architect & Principal Consultant

2 > Not Designing a Data Architecture is a …

3 > Why do we need an LDM? Data Warehouse with LDM Data Warehouse Without LDM

4 > What is the Purpose of a Data Model? A visual business representation of how data is organized in the enterprise It provides discipline and structure to the complexities inherent in data management Can you imagine building a house without a blueprint? Or driving across the country without a map? It facilitates communication within the business (e.g. within IT and between IT and the business) It facilitates arriving at a common understanding of important business concepts (e.g what is a customer?)

5 > Logical Data Model Components … LDM graphically represents the data requirements and data organization of the business >Identifies those things about which it is important to track information (entities) >Facts about those things (attributes) >Associations between those things (relationships) Subject-oriented, designed in Third Normal Form – one fact in one place in the right place

6 > Reference Models Lots of Detail / Expertise Behind Models

7 > Reference Model Sources Data Warehousing Vendors >IBM >Oracle >Teradata > … Tool Vendors >Embarcadero > … Service Vendors >EWSolutions > … Industry/Standards Associations >ARTS (Association for Retail Technology Standards) > …

8 > Teradata Industry Logical Data Models - iLDMs Financial Services - Banking, Investments Travel - Travel, Hospitality, Gaming Retail - Retail Store, Food Service Communications - Wireline, Wireless, Cable, Satellite Healthcare - Payor, HIPAA Transportation - 3PL, 4PL, Air, Truck, Rail, Sea Manufacturing - CPG, High Tech Automotive Financial Services - Insurance

9 > Data Management Context Three Layer Structure EDW-LDM EDW-PDM Implement (Physical) Analyze & Design (Logical) Core (Enterprise) Semantic (Usage/Presentation) Semantic Layer Models Load Once Use Many Marts Views BIOs & User Types drive requirements iLDM Used for customization Source Operational Images Data Integration Source

10 > Enterprise Information Management Requires A Shared VOCABULARY Experts estimate that the 500 most commonly used words in the English language have an average of 28 definitions each.

11 > Enterprise Data Management Objectives that are enabled by Enterprise Logical Data Modeling : >Build a Common Business Vocabulary for the enterprise. >Develop an EDW Data Structure that is Neutral from All the Sources that populate it. >Develop an EDW Data Structure that will Support All Business Requirements While Not Being Constrained by any specific requirement. –i.e. Neutral from use by multiple functional areas –Supports operational and analytical uses

12 > Data Modeling Structure SUBJECT Model CONCEPTUAL Model KEY-BASED Model ATTRIBUTED Model PHYSICAL Model Data Modeling A model of the high level data concepts that define the scope of the Data Architecture. An entity-relationship model that identifies the elements of the Business Vocabulary and Business Rules. A refinement of the Conceptual Model that identifies the natural and surrogate keys for all entitles and relationships. This the foundation of the Enterprise Business Vocabulary. A detailed model that identifies the non-key attributes for the entitles. Attribution also leads to refining the Key-Based Model A model that is the design for a database. The Attributed Model is transformed for Sourcing and Accessing performance.

13 > Data Modeling Structure Purposes SUBJECT Model CONCEPTUAL Model KEY-BASED Model ATTRIBUTED Model PHYSICAL Model Data Modeling Architecture Implementation Reference Model Information Requirements Business Improvement Opportunities Business Questions Key Performance Indicators Legacy Reporting/Analysis

14 > Data/Information Management Teradata Enabled SUBJECT Model CONCEPTUAL Model KEY-BASED Model ATTRIBUTED Model PHYSICAL Model Data Modeling SEMANTIC Layer Data Warehousing APPLICATION Layer Access Layer Source Layer STAGING Layer Sources Data Source Layer CORE Layer Master Data Transaction Data DDL

15 > Data Management Context Agile Development Environment Sandbox User External Data

16 > Data Management Context Perceived Value from Medium to Large Scale Projects Sandbox User External Data 80-95% 5-15% 0-5% 0-1%

17 > Data Management Context Development Time for Medium to Large Scale Projects Sandbox User External Data 4-8 weeks 3-6 months 2-4 Months 1-5 days

18 > Data Integration Common Shared Local 1 st Sandbox Application 3 rd Sandbox Application 2 nd Sandbox Application

19 > Data Management Context Integration in an Agile Development Environment Sandbox User External Data Conceptual Data Architecture Governance-driven Integration

20 > Pros and Cons of Using a Vendor Provided Analytical Data Model in Your BI Implementation Boris Evelson, Information Management Blogs, January 29, 2010 Boris Evelson Pros: Leverage vendor knowledge from prior experience and other customers May fill in the gaps in enterprise domain knowledge Best if your IT dept does not have experienced data modelers May sometimes serve as a project, initiative, solution accelerator May sometimes break through a stalemate between stakeholders failing to agree on metrics, definitions Cons: May sometimes require more customization effort, than building a model from scratch May create difference of opinion arguments and potential road blocks from your own experienced data modelers May reduce competitive advantage of business intelligence and analytics (since competitors may be using the same model) Goes against “agile” BI principles that call for small, quick, tangible deliverables Goes against top down performance management design and modeling best practices, where one does not start with a logical data model but rather >Defines departmental, line of business strategies >Links goals and objectives needed to fulfill these strategies >Defines metrics needed to measure the progress against goals and objectives >Defines strategic, tactical and operational decisions that need to be made based on metrics >Then, and only then defines logical model needed to support the metrics and decisions Let’s discuss.

21 > Cooking Something New... “Change without a recipe is a recipe for chaos.” “The transformation model must describe not only the steps in the process, but also the enabling context that is critical to its success.” If Only We Knew What We Know Carla O’Dell & C. Jackson Grayson The Free Press, 1998