I NTRODUCTION OF W EEK 14  Assignment Discussion  Graded: (Lab4: Query Optimization)  Creating, reviewing, and interpretation are all important for this assignment.  Analysis of execution plans: scan (full table, index range, index unique), order of execution, join operations (hash, nested loop)  Comparing the plans, not statistics  Complexity of plan != efficiency of the query: I/O scans, sorting, joins are expensive  Understanding application query tuning  Due: 12-1 (Database Storage), 12-2 (Bulk Data Movement)  Working: 13-1 (Research paper: database metadata management)  Working: (Final Project Write-up)  Review of previous week and module  Metadata Management, Database Management Tools  Oracle 10g Data Dictionary and Dynamic Performance Views  Overview of this week – Module 5  Data Warehouse Administration  Course Summary  Final Exam Discussion 1 ITEC Fall

M ODULE 5 Metadata, Tools, and Data Warehousing Section 4 Data Warehouse Administration 2 ITEC Fall

D ATA W AREHOUSE AND C HARACTERISTICS A data warehouse is a subject-oriented, integrated, time-variant, non-volatile collection of data that is designed for query and analysis rather than for transaction processes. Subject-oriented – data pertains to a particular subject instead of the many subjects pertinent to the company’s ongoing operations. Integrated – consistent naming conventions, formats, encoding structures; from multiple data sources Time-variant – data is identified with a particular time period, can study trends and changes Non-updatable – data is stable in a data warehouse. Data loaded, and should not be removed Fall 3 ITEC 450

C OMPARISON OF D ATABASE C HARACTERISTICS 2011 Fall 4 ITEC 450

D ATA W AREHOUSE AND B USINESS I NTELLIGENCE A data warehouse usually contains historical data derived from transaction data and other sources. It enables an organization to consolidate data. It includes An extraction, transportation, transformation, and loading (ETL) solution An online analytical processing (OLAP) engine Client analysis tools Reporting 2011 Fall 5 ITEC 450

A NALYTICAL VS. T RANSACTION P ROCESSING Analytical processing – informational systems DSS – decision support system OLAP – online analytical processing Data mining – the process of mining or discovery of new information in terms of patterns or rules from vast amounts of data Transaction processing – operational system OLTP – online transaction processing 2011 Fall 6 ITEC 450

D ATA W AREHOUSE D ESIGN Star schema - data modeling technique used to map multidimensional decision support data into a relational database. It is ex cellent for ad-hoc queries, but bad for online transaction processing. It contains four components: Fact table Dimension tables Attributes Attribute hierarchies Snowflake schema – a star schema in which the dimension tables have additional relationships 2011 Fall 7 ITEC 450

S TAR S CHEMA C OMPONENTS 2011 Fall 8 ITEC 450

S TAR S CHEMA E XAMPLE 2011 Fall 9 ITEC 450

D ATA M OVEMENT – ETL P ROCESS ETL – Extract, Transform, and Load Capture – extract or obtaining a snapshot of a chosen subset of the source data for loading into the data warehouse Scrub or data cleansing – us es pattern recognition and AI techniques to upgrade data quality Transform – c onvert data from format of operational system to format of data warehouse Load – p lace transformed data into the warehouse and create indexes 2011 Fall 10 ITEC 450

D ATA W AREHOUSE P ERFORMANCE Perspectives of data warehouse performance Extract performance – how ETL process performs Data management – database design and data quality Query performance – OLAP tuning Server performance – hardware support Automated summary tables Provide a proper set of aggregate information Commonly implement with materialized views or batch operation tables DBMS features to support data warehousing Materialized views – automatically creation of summaries Bitmap indexes – widely used in data warehousing, in addition to B-tree Parallel execution – multiple processes work together simultaneously to run a single SQL statement 2011 Fall 11 ITEC 450

M ODULE 5 Metadata, Tools, and Data Warehousing Section 5 DBA Rules of Thumb 12 ITEC Fall

T HE R ULES OF T HUMB Personal DBA handbook Write down your own experience Categorize them in a searchable note or repository Backup everything and plan for worst all the time Before making any changes, ensure that you can recover from them Automation and share your knowledge Create a systematic way to troubleshoot problems Create, reuse and share scripts Knowledge sharing will open many revenues for you Next levels Understand the business, not just the technology Keep up-to-date on technology 2011 Fall 13 ITEC 450

C OURSE S UMMARY (Y OUR L EARNING ) DBA Roles and Responsibilities DBMS Architecture, Physical and Logical Structures DBMS Installation and Database Creation Database Connectivity and Network Components Database Security and Audit Capability Database Backup and Recovery Database Monitoring, DBMS System Tuning, Physical Configuration Optimization SQL Query Coding and Tuning, Data Loading Database Metadata, Data Dictionary Data Warehouse Characteristics and Overview 2011 Fall 14 ITEC 450

F INAL E XAM Midterm coverage (30%) Backup choices, recover mechanisms and high availability features Performance influential factors, Database performance tuning Optimizer overview and optimizer influential factors Oracle query optimizer processing, statistics collection, execution plan Oracle physical and logical database structures Space management, RAID technology The load utility, data pump export and import DBMS Metadata, metadata type Oracle data dictionary, dynamic performance views Data warehouse, characteristic differences vs. operational database, analytic (OLAP) vs. transactional processing (OLTP) Data warehouse database design (star schema), ETL 2011 Fall 15 ITEC 450

S CHEDULE R EMINDER ONE MORE TIME Final exam can be taken between Thursday, Dec. 8 and the week after Wednesday, Dec. 14. The final exam must be completed on or before Wednesday of Week 15, not Sunday! Check with your proctor or test center Fall 16 ITEC 450 All assignments are due by Sunday, December 11, and no late assignments will be accepted after the date. Please review your grade book, and let me know any missing grades right way.

T HANK Y OU AND G OOD L UCK 2011 Fall 17 ITEC 450