Soup-2-Nuts Alaska Department of Fish & Game Commercial Fisheries October, 2011

Soup-2-Nuts An Overview of the Commercial Fisheries Data Warehouse and Business Intelligence Structure Project Objectives Components Show Me Project Status & In Process

Project Links Project Site: Project Phases: oject%20Phases.aspx oject%20Phases.aspx

Project Objectives 1. Provide a Single Reporting Tool for All Commercial Fisheries Eliminate multiple technologies and make better use of our programmers and end users time Provide end users with a common interface where we can integrate data entry applications Provide users with both powerful ad-hoc capabilities and ‘canned’ reports

Project Objectives 1. Provide a Single Reporting Tool for All Commercial Fisheries (continued) Provide users with scheduled reports - supports low bandwidth areas Provide state-wide views and access to data across regions Provide ongoing workshops in this common tool & use staff to teach staff as well as share designs.

Project Objectives 2. Provide a Data Warehouse as a Reporting Source Integrate data Transform data when required Divorce Reporting from Data Entry ‘Reporting Friendly’ source of data 3. Data Rescue Support Per Subject Area, review data inventory and provide a home for endangered data

Components Source Database Oracle Data Integrator (ODI) Oracle Data Warehouse Oracle Business Intelligence Administrator Oracle Business Intelligence (OBI or BI) =

Source Database Original storage point of data entry Relational Database - Supports fast and valid data entry

Source Database Many ‘sources’ in CF Many types of databases Many input applications None ‘talk’ to each other Duplicated data

Source Systems What you don’t see: The plethora of individual reporting technologies The inability of staff to cross application boundaries Man-hours lost manually manipulating data Man-hours lost duplicating effort

Data Integrator Has one job – Map from point A to point B. Can change the shape of the data to optimize for reporting Can apply logic and combine source databases Is optional!

Data Warehouse Can consolidate from many sources Can be rebuilt from scratch Scheduled updates according to needs Is optional!

OBI Administrator A tool used to prepare the data for use from various sources for use in the OBI Interface Uses data direct from source databases or data warehouse (ODI and Data Warehouse optional) Can add value: Calculated fields, Hierarchies Organizes data by “Subject Areas”

OBI Interface The web based end-user tool Dashboards Custom and Ad-hoc reports Public web reports & charts

All Together Source Database Data Integrator Data Warehouse BI Administrator BI User Interface

What is ETL? ETL means “Extract, Transform and Load” Executed by Oracle Data Integrator (ODI) Extract – Grab the data from the source(s) Transform – Any restructuring to make reporting easier (i.e. Merge data from multiple tables) Load – Store the data in the data warehouse Only used if data is required to be integrated with other sources or transformed. (General rule).

Why Transform the Data?

eLandings Source Database

ETL Using Oracle Data Integrator (ODI) – Loading data warehouse

ODI Task – LOAD_ITEM_FACTS

Target Data Warehouse Table

Show ME – OBI Admin

BI Admin – Physical Layer Defines the physical data model of your data source

BI Admin – Business Model Allows you to set up Fact table with measures Dimension tables Hierarchies

BI Admin – Presentation Layer The structure that is presented to the user Accessed through logical sql No key columns

Show Me – OBI Interface

Show Me – OBI to the Public

Will All Data Use the Warehouse? Data that does not require transformation may not use the data warehouse Much simpler & quicker to build Can join later!

Current Status Region I waiting Oracle release (bug) Region I Public reporting in production eLandings is in Production Staff trained Dutch, Homer, Kodiak – SE this winter BlueSheets is in Production Mariner V2012 is in design Initial staff workshop September – more to follow Fish Tickets Detail – Iterative Testing of ETL MTAL in training

Future COAR/ITO Waiting eLandings planning ASL Repository Move/Expand Work in conjunction with ASL project Expansion of Mariner V2013 to Region III IT staff training Fish Ticket Summaries – Historical Incorporate views of regional summaries for identifying data cleanup issues