Enterprise Information Services (EIS) - Review Ray Chase Manager, EIS.

Slides:



Advertisements
Similar presentations
New Release Announcements and Product Roadmap Chris DiPierro, Director of Software Development April 9-11, 2014
Advertisements

Data Extracts Working Group (DEWG) Update Tuesday, September 25, 2006.
Data Warehouse Architecture Sakthi Angappamudali Data Architect, The Oregon State University, Corvallis 16 th May, 2005.
VP, CIO Information Technology
Data Warehousing - 3 ISYS 650. Snowflake Schema one or more dimension tables do not join directly to the fact table but must join through other dimension.
Introduction to Database Management
Page 1 Program Management Office “PMO” Update Steve Wallace February 3, 2003.
Data Warehouse Concepts & Architecture.
Solutions for Meter Data Management Layne Nelson Product Manager LODESTAR Corporation Copyright © 2005 LODESTAR Corporation - All Rights Reserved LODESTAR.
® IBM Software Group © IBM Corporation IBM Information Server Service Oriented Architecture WebSphere Information Services Director (WISD)
Database Management System Lecture 2 Introduction to Database management.
ENTERPRISE DATA INTEGRATION APPLICATION ARCHITECTURE COMMITTEE OCTOBER 8, Year Strategic Initiatives.
Business Intelligence
C A S E S T U D I E S—S T R A T E G I E S F O R S U C C E S S November 7 - 9, 2002.
1 Emerging Issues. 2 Enterprise Data Warehouse (EDW)
Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.
PROJECT NAME: DHS Watch List Integration (WLI) Information Sharing Environment (ISE) MANAGER: Michael Borden PHONE: (703) extension 105.
Objectives: Upgrade Siebel to a supported application Upgrade Oracle database to current version Deliver all existing user functionality with no degradation.
Lead FromThe Front Texas Nodal May 17 th 2006 – Nodal Organization V1.5 1 Getting physical – high-level packaging for delivery accountability Commercial.
ESCOP ™ System Center Operations Portal Expanding SCOM 2007 Data Warehouse and ACS Function.
Data Warehouse Overview September 28, 2012 presented by Terry Bilskie.
ETL Overview February 24, DS User Group - ETL - February ETL Overview “ETL is the heart and soul of business intelligence (BI).” -- TDWI ETL.
RO Project Priority List Update EDW Projects Update RMS Meeting Adam Martinez Mgr, Market Ops Divisional Projects Organization ERCOT April 12, 2006.
material assembled from the web pages at
SCR 740 Implementation of Web Services for ESIID level data Jackie Ashbaugh EAA Data Management.
Business Intelligence Zamaneh Jahed. What is Business Intelligence? Business Intelligence (BI) is a broad category of applications and technologies for.
IO – CART Project Status Protocol Revision Subcommittee Update 06/22/06.
Lead from the front Texas Nodal 1 External Web Services Update Nodal Implementation Team Presentation July 7, 2009.
PMO Update to RMS Troy Anderson Program Management Office April 12, 2005.
Page 1 Information Technology Update. Page 2 ERCOT Technology Architecture Today The current architecture is typical of an environment that started off.
ERCOT IT Update Ken Shoquist VP, CIO Information Technology Board Meeting November 2003.
COMMERCIAL OPERATIONS ORGANIZATION Betty Day Director, Commercial Operations (office) (cell) (fax)
June 10, 2009 RMS PR90006, Commercial Systems Information Lifecycle Management (ILM) Hope Parrish, ERCOT.
1 Categories of data Operational and very short-term decision making data Current, short-term decision making, related to financial transactions, detailed.
Overview of MID, MIR, & Extracts for DEWG 9/24/07 Steve Cox.
1/22/08 RTR Project Presentation to TPTF RTR Project Michael Daskalantonakis & Brian Cook.
1 C urrent Market Release TX SET V2.0 / Solution to Stacking.
MARKET/OPERATIONS SUPPORT ORGANIZATION Joel Mickey Director, Market Operations Support (office) (cell)
Chapter 5 DATA WAREHOUSING Study Sections 5.2, 5.3, 5.5, Pages: & Snowflake schema.
ERCOT PMO Update Robert Connell Director Program Management Board of Directors Update February 2005.
McGraw-Hill/Irwin ©2009 The McGraw-Hill Companies, All Rights Reserved CHAPTER 6 DATABASES AND DATA WAREHOUSES CHAPTER 6 DATABASES AND DATA WAREHOUSES.
PMO Update to PRS Troy Anderson ERCOT Program Management Office December 17, 2009.
Retail Transaction Processing Year End Review and Recent Issues RMS January 2007.
January 28, 2008 DEWG DEWG Discussion Questions ERCOT.
SQL Server 2008 R2 Manageability. Challenges facing database administrators today: Scaling management to multiple data centers Proactively monitoring.
ERCOT PMO Update Robert Connell Director Program Management May Board of Directors May 17, 2005 (Through 4/30/05)
Enterprise Information Services Ray Chase Manager, EIS.
1 New RO Projects Hope Parrish June NEW RO Projects for 2008 Requested by ERCOT - Overview Objective Objective of the following information is to.
1 Market Data Transparency SCR 740 Business & Technical Training SCR 727 Phase II: Web Services ERCOT presents: Market Data Transparency.
Changes to SSO Extract:  Based on DEWG discussions in 2005, ERCOT is recommending the following changes to the SSOE: In lieu of the current daily files.
ERCOT Market Services Organization Dale Goodman Director, Market Services (office) (cell) (fax)
September 21, Data Extract Projects Jackie Ashbaugh Commercial Operations Data Integrity & Administration September 21, 2006.
EIS Overview and Project Updates for DEWG Jackie Ashbaugh ERCOT April 13, 2006.
EIS Projects Update COPS Meeting Adam D. Martinez Mgr, Market Ops Divisional Projects Organization ERCOT April 28, 2006.
ERCOT Project Process Karen Farley COPS 3/22/05. Project Process  Overview of Project Requests  Process for Market Requests  Process for Internal ERCOT.
1 Copyright © Oracle Corporation, All rights reserved. Business Intelligence and Data Warehousing.
August 17, 2006 Data Extract Working Group EDW Project Updates 2006 / 2007 MO & RO PPL.
Chapter 8: Data Warehousing. Data Warehouse Defined A physical repository where relational data are specially organized to provide enterprise- wide, cleansed.
Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 5: Data Warehousing.
Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 8: Data Warehousing.
Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 8: Data Warehousing.
MODPO Project Update Overview of December Implementations & EDW Changes Commercial Operations Subcommittee December 11, 2006.
Business Intelligence dan Dashboard Ensys – minggu ke 6.
Role of Account Management at ERCOT EIS Project Updates Jackie Ashbaugh Mgr., Commercial Operations Data Integrity & Administration DEWG October 19, 2006.
Tapping the Power of Your Historical Data
Data Warehouse Overview September 28, 2012 presented by Terry Bilskie
Data warehouse.
Data Warehousing Concepts
David Gilmore & Richard Blevins Senior Consultants April 17th, 2012
Presentation transcript:

Enterprise Information Services (EIS) - Review Ray Chase Manager, EIS

EIS Vision Provide reliable, innovative, centralized, and comprehensive decision support and business intelligence systems to ERCOT, the Market, and the PUCT.

Why?  Protocol Requirements – Market Data Collection and Use (section 17)  Collect an Accurate Historical Record of all Source Systems Intra-day Capture – All Data Auditability  Single System Source Data Integration Opportunity to Offset Source System Load  Robust Reporting Environment Ease of Use Centralized Tools One Version of the Truth  Data Modeling Oversight Future ERCOT and Market Decision Support Opportunities

Overview  Data Collection  Data Archiving/Roll-off  Data Integration  Data Transparency and Delivery  SOA Framework  Data Modeling and Analysis  Business Intelligence and Reporting  Legacy System Transition  Current Project Relationships  Current Statistics

Data Collection  All Operational Source Systems feed into ODS  Replication is near real-time  All intra-day transactions  Nearly all tables (rules) Paper Free Lodestar lodstar Lodestar lodstar Lodestar lodstar Siebel TCH RSS Lodestar lodstar EMMS omsappdev EMMS ercot EMMS his_user Siebel Paper Free TCH ODS TCR

Data Archiving/Roll-off  Uses a table based, data partitioning strategy Performance Long-term Historical Data Archiving Source SystemODSRSS Record Insert, Update, Update, Update Insert Insert Insert Current History Table_A Long-term Tape

 Multi-source system data integration Removes fragmentation of solutions Single version of the truth Reusability  Various 2005 projects with integration – Collateral Calc – ESIID Lookup – TCR Reporting – BI Foundations Data Integration Business Intelligence ODS Lodestar lodstar EMMS omsappdev TCR Collateral Calculation

Portal Views Data Transparency and Delivery  Single Design - Multiple Solutions Large Scale Market Participant Extraction Public Data Extraction Market Reporting Tool (2006) Web Service Delivered Data (50024 – SCR740, ESIID)  Type-1 – Scheduled Data Delivery  Type-2 – Real-time Data Delivery Business Intelligence RSSODS CDW Market Data Extracts Web Services Report Tool

ERCOT Service Orientated Architecture Framework URLXML MP System URLXML MP System SOAP Request SOAP Reply ERCOT’s Service Orientated Architecture Framework will provide four methods to access ERCOT’s information and services Use an ERCOT supplied Web Interface URLXML XSLT URLHTML Build your own interface utilizing our XML REST Web Services System to System utilizing our XML REST Web Services System to System utilizing our SOAP Web Services SOA Framework

Data Modeling and Analysis  Provides a framework for decision support, business intelligence, and data modeling and analysis Business Intelligence RSSODS CDW PC.xls PC.xls PC.xls Legacy Current Lodestar EMMS PC MS Access.xls PC MS Access.xls PC MS Access. xls Business Logic …

Data Modeling and Analysis  Data Warehouse Marts Business Intelligence ODS TCH Siebel CDW Proposed Lodestar lodstar EMMS omsappdev EMMS ercot EMMS his_user ETS Data Mart PC.xls PC.xls PC.xls

Data Modeling and Analysis  Potential External Data Source Additions Business Intelligence RSSODS CDW PC.xls PC.xls PC.xls Census Geographic Detailed Weather

Business Intelligence & Reporting  Centralized view foundation Provides standardized querying of all data Secure - Prevents direct access to the ODS database objects Ensures all private data is protected by the ODS security layer Application Security Operational Data Store (ODS) API Views API Ref Cursor In-Bound Out- Bound Application Security Conformed Data Warehouse (CDW) API Views API Ref Cursor In-Bound Out- Bound

Business Intelligence & Reporting  Web-based tool for universal use B.I. tool allows “drilling-thru” to lower levels of granularity Extremely efficient for quickly & visually going from “10,000 foot” view to “extreme close-up” Prompts provide a way quickly choosing your selection criteria

Operational Data Stores Replicated Source Systems EIS Systems & Applications ETS Report Net Summary Reporting Business Intelligence Foundation Application Replication & Security Framework Security Foundations Standard Views Standard APIs Custom Views Custom APIs & BLAPIs Application Administration Data Delivery Module (DDM) Web Service Based Data Delivery Lodestar ODS EMMS ODS TCR ODS EIS System Transition Siebel ODS Paperfree ODS TCH ODS Lodestar RSS EMMS RSS Lodestar EMMS TCR Siebel Paperfree TCH Siebel RSS Paperfree RSS TCH RSS Detailed Reporting Extracts Data Mart Real-Time Reporting Conformed Data Warehouse (CDw) Data Transparency T D 6 D D D

Current Project Relationships P P P P P Initiated Project Completed Project Cancelled or Deferred Project Planning Project Executing Project Legend MO CO SO IO RO Project NOT Started P Affected by Parsing P

Proposed 2006 EIS Projects SourceCommitteeSrc DocProjectProj. No.Impact Area Add' l I n f o Priority Ran k PUCTIO EDW BI Foundation P2 - Repl Reserves Market Ad-hoc Reporting50001_03 Market Monitor ing EDW1.1a22 PUCTIO EDW BI Foundation P2 - Ad-hoc Access to QSE Rpt Data50001_04 Market Monitor ing EDW1.1a23 PUCTIO EIS BI Foundations IIINew Market Monitor ing EDW1.1a24 PUCTIO EIS Conformed Data Warehouse FoundationsNew Market Monitor ing EDW1.1a27 PUCTIO EIS CDW Data MartsNew Market Monitor ing EDW1.1a28 ERCOTRO EDW Siebel Transition50059 InfrastructureEDW1.1b48 ERCOTRO EDW TCH (Transaction Clearinghouse) Transition50060 InfrastructureEDW1.1b49 ERCOTRO EDW EAI Transition (inc. PaperFree & NAESB)50061 InfrastructureEDW1.1b50 ERCOTRO ETS (ESIID Tracking System) Transition to EDW50088 InfrastructureEDW1.1b51 ERCOTIO EDW Lodestar Decommission50030 InfrastructureEDW1.1b67 ERCOTIO EDW EMMS Decommission50031 InfrastructureEDW1.1b68 ERCOTIO EIS Reporting Tool IINew Market Monitor ing EDW1.1b69 ERCOTMO Lodestar Web ServicesNew Market ImpactEDW1.293

Current ODS Statistics  Migration - ~2 billion historical transactions from Lodestar and EMMS  Replication – no lost information to date Daily Average ~15 million transactions Total transactions posted  Developed/Installed Code Base Source SystemReplicated LODSTAR1,273,023,928 EMMS890,651,313 HIS_USER181,868,288 2,345,543,529 ObjectCountLines INDEX787 PACKAGE1,501150,935 PACKAGE BODY1,500610,481 PROCEDURE68245,400 TABLE794 TRIGGER76269,763 VIEW3,535103,208 9,561979,787

Conclusion  Comments?  Questions?  Thank You!