MIS 4346/5346 Data warehousing Data Warehouse Implementation.

Slides:



Advertisements
Similar presentations
Dimensional Modeling.
Advertisements

Irwin/McGraw-Hill Copyright © 2000 The McGraw-Hill Companies. All Rights reserved Whitten Bentley DittmanSYSTEMS ANALYSIS AND DESIGN METHODS5th Edition.
Supervisor : Prof . Abbdolahzadeh
Big Data Working with Terabytes in SQL Server Andrew Novick
Extract, Transform, Load 1. Agenda  Review  Analysis (Bus Matrix, Info Package)  Logical Design(Dimensional Modeling)  Physical Design(Spreadsheet)
Data Manager Business Intelligence Solutions. Data Mart and Data Warehouse Data Warehouse Architecture Dimensional Data Structure Extract, transform and.
James Serra – Data Warehouse/BI/MDM Architect
DAVID M. KROENKE’S DATABASE PROCESSING, 10th Edition © 2006 Pearson Prentice Hall 11-1 David M. Kroenke’s Chapter Eleven: Managing Databases with SQL Server.
Presented by Brad Gall Using BI Techniques for Database Statistics.
Technical BI Project Lifecycle
What is Management Reporting from a Data Warehouse and What Does It Have to Do with Institutional Research? Emily Thomas Stony Brook University AIRPO Winter.
Fundamentals, Design, and Implementation, 9/e Chapter 11 Managing Databases with SQL Server 2000.
1 ACCTG 6910 Building Enterprise & Business Intelligence Systems (e.bis) The Data Warehouse Lifecycle Olivia R. Liu Sheng, Ph.D. Emma Eccles Jones Presidential.
Telecommunication Case Study CS 543 – Data Warehousing.
Chapter 4: Managing Information Resources with Databases Copyright © 2013 Pearson Education, Inc. publishing as Prentice Hall Chapter
Dimensional Modeling 1. Agenda  Review: Business Requirements  Dimensional Model Components  Dimensional Model Schemas  Additional Modeling Concepts.
Data Warehouse Toolkit Introduction. Data Warehouse Bill Inmon's paradigm: Data warehouse is one part of the overall business intelligence system. An.
Building a Data Warehouse from SAP using iWay and WebFOCUS O. Julian Plys PMP, Sunoco, Inc. Carole Benoit, Tek Systems, Inc.
SQL Server Management Studio Introduction
Data warehousing theory and modelling techniques Building Dimensional Models.
ETL Design and Development Michael A. Fudge, Jr.
ETL By Dr. Gabriel.
Business Intelligence
Data Warehouse to BI 1. Agenda  Review  Preparing the DW for Analysis  Microsoft BI Platform Overview  Building a Cube in SSAS 2.
1 Chapter Overview Creating a User Database Setting Database Options Managing User Database Size Placing Database Files on Multiple Disks.
© 2002 by Prentice Hall 1 David M. Kroenke Database Processing Eighth Edition Chapter 13 Managing Databases with SQL Server 2000.
1.
1 Brett Hanes 30 March 2007 Data Warehousing & Business Intelligence 30 March 2007 Brett Hanes.
 SQL stands for Structured Query Language.  SQL lets you access and manipulate databases.  SQL is an ANSI (American National Standards Institute) standard.
Databases in Visual Studio. Database in VisualStudio An MS SQL database are built in Visual studio The Name can be something like ”(localdb)\Projects”
DBSQL 14-1 Copyright © Genetic Computer School 2009 Chapter 14 Microsoft SQL Server.
Company LOGO 1 Database Creation and Maintenance Jorge G. Martinez.
Class Introduction 1. Agenda  Instructor Introduction  Administration  Course Overview  Syllabus  Next Time… 2.
Bus Architecture. Value Chain Identifies the natural logical flow of an organization’s primary activities Operational source systems produce snapshots.
Introduction to SQL Server JOINS © Meganadha Reddy K., Meganadha Reddy K. Technical Trainer | NetCom Learning
SQL: DDL. SQL Statements DDL - data definition language –Defining and modifying data structures (metadata): database, tables, views, etc. DML - data manipulation.
Chapter 4: Managing Information Resources with Databases Copyright © 2013 Pearson Education, Inc. publishing as Prentice Hall Chapter
13 Copyright © 2009, Oracle. All rights reserved. Integrating with Oracle Business Intelligence Enterprise Edition (OBI EE)
Dimensional Modeling Primer Chapter 1 Kimball & Ross.
Data Staging Data Loading and Cleaning Marakas pg. 25 BCIS 4660 Spring 2012.
Learningcomputer.com SQL Server 2008 –Views, Functions and Stored Procedures.
Maintenance Practices. Goal  Automate the necessary DBA chores to put organizations on the path of having healthier, consistent and more trustworthy.
DATABASE DEVELOPMENT WITH VISUAL STUDIO 2010 Chris Dahlberg 1.
Extract, Transform, Load 1. Agenda  Review  Analysis (Bus Matrix, Info Package)  Logical Design(Dimensional Modeling)  Physical Design(Spreadsheet)
Dimensional Modeling 1. Agenda  DW Project Lifecycle  Eliciting Business Requirements  Dimensional Model Components  Dimensional Model Schemas  Additional.
Web based Documentation Distribution Tools: MSAccess database (DSN) DreamWeaver Ultradev Microsoft Image Composer Clicking on the document will open an.
SUCCESSFUL BI PROJECTS Ian Meade – Database architect – Storm technologies.
Or How I Learned to Love the Cube…. Alexander P. Nykolaiszyn BLOG:
Building the Corporate Data Warehouse Pindaro Demertzoglou Data Resource Management.
Data Warehouse/Data Mart It’s all about the data.
Building the Corporate Data Warehouse Pindaro Demertzoglou Lally School of Management Data Resource Management.
DATA WAREHOUSING TECHNIQUES ROUNDTABLE Kathy Bronson Trevyn Bowden Clackamas Communtiy College 7/2016 Information Technology Forest Grove, Oregon NWEUG.
Supervisor : Prof . Abbdolahzadeh
Building a Polished Cube
ISQS 6339, Business Intelligence Database vs. Data Warehouse
Student Retention in Higher Education
Lecture-34 DWH Implementation: Goal Driven Approach (2)
Designing Business Intelligence Solutions with Microsoft SQL Server
Dimensional Model January 14, 2003
Designing Business Intelligence Solutions with Microsoft SQL Server
Data Warehouse Architecture
Warehouse Architecture
Data Warehouse Architecture
Introduction of Week 9 Return assignment 5-2
Chapter 11 Managing Databases with SQL Server 2000
Applying Data Warehouse Techniques
កម្មវិធីបង្រៀន SQL Programming ជាភាសាខ្មែរ Online SQL Training Course
CS4540 Special Topics in Web Development SQL and MS SQL
SQL AUTO INCREMENT Field
Implementing ETL solution for Incremental Data Load in Microsoft SQL Server Ganesh Lohani SR. Data Analyst Lockheed Martin
Presentation transcript:

MIS 4346/5346 Data warehousing Data Warehouse Implementation

Agenda Review Development Approach Review Dimensional Modeling Implementing the Data Warehouse with SQL Server Enterprise Edition Implementing Data Mart Physical Structures Creating the data mart database Creating dimension tables Creating fact tables Using scripts

DW Development Approach: Kimball Methodology DW Project Lifecycle Business requirements Business Requirements Documentation Bus Matrix Design, build and deliver in increments DW Architecture DW Design ETL system Cube, Reports, query tools, …

Review: Dimensional Modeling

Dimensional Model: Revisited

Data Warehouse Project Lifecycle Source: Mundy, Thornthwaite, and Kimball (2006). The Microsoft Data Warehouse Toolkit, Wiley Publishing Inc., Indianapolis, IN.

IT Architecture/Infrastructure Physical Design IT Architecture/Infrastructure Physical Design*: SQL Server Enterprise Edition SQL Server Database Engine * Specifically Product Selection & Installation

Data Warehouse Project Lifecycle Source: Mundy, Thornthwaite, and Kimball (2006). The Microsoft Data Warehouse Toolkit, Wiley Publishing Inc., Indianapolis, IN.

DW/DM Implementation: Building the Data Mart Database Typically one database per data mart Example: USE MASTER CREATE DATABASE ClassPerformanceDW; GO ALTER DATABASE ClassPerformanceDW SET RECOVERY SIMPLE

Creating Dimension Tables Naming is typically DimTableName Consider data compression Example: CREATE TABLE DimStudent( student_sk int identity(1,1) , student_id varchar(9) , firstname varchar(30) , lastname varchar(30) , major varchar(7) , classification varchar(25) , gpa numeric(2, 1) , clubname varchar(25) , undergradschool varchar(25) , gmat int , undergradORgrad varchar(10) , CONSTRAINT dimstudent_pk PRIMARY KEY (student_sk)); GO CREATE INDEX student_id_idx on DimStudent (student_id); ALTER TABLE DimStudent REBUILD WITH (DATA_COMPRESSION = PAGE); GRANT SELECT ON DimStudent TO PUBLIC; See http://blog.sqlauthority.com/2010/03/01/sql-server-data-and-page-compressions-data-storage- and-io-improvement/ OR http://sqlmag.com/database-performance-tuning/practical-data-compression-sql-server

Creating Fact Tables Naming typically FactTableName Example: CREATE TABLE fact_enrollment( student_sk int, class_sk int, date_sk int, professor_sk int, location_sk int, termyear_sk int, coursegrade numeric(2, 1), CONSTRAINT fact_enrollment_pk PRIMARY KEY (student_sk, class_sk, date_sk, professor_sk), CONSTRAINT fact_enrollment_student_fk FOREIGN KEY (student_sk) REFERENCES dimstudent(student_sk), CONSTRAINT fact_enrollment_class_fk FOREIGN KEY(class_sk) REFERENCES dimclass (class_sk), CONSTRAINT fact_enrollment_date_fk FOREIGN KEY(date_sk) REFERENCES dimtime (date_sk), CONSTRAINT fact_enrollment_professor_fk FOREIGN KEY(professor_sk) REFERENCES dimprofessor (professor_sk), CONSTRAINT fact_enrollment_location_fk FOREIGN KEY(location_sk) REFERENCES dimlocation (location_sk), CONSTRAINT fact_enrollment_termyear_fk FOREIGN KEY(termyear_sk) REFERENCES dimtermyear (termyear_sk), ); GO GRANT SELECT ON factenrollment TO PUBLIC;

Using Scripts Contains all statements to create data mart tables Advantages: Can easily create test environments Can easily create production tables Fewer files to manage Code reuse

Example Script “Design” CREATE Script Contains CREATEs for all tables TRANSFORM/LOAD Script (next topic) Calls individual transform/load scripts One for each table Cleanup Clear and shrink the log file Example: http://business.baylor.edu/gina_green/teaching/sqlserver/scripts/generate_c lass_performance_dw_tables.zip

Summary Physical Design: Infrastructure and DW Creating and Naming: Database Dimension tables Fact tables Considerations when creating above objects Using scripts