ERCOT Lodestar Extracts Timing Recommendation ERCOT May 24, 2005.

Slides:



Advertisements
Similar presentations
Information Technology Update Aaron Smallwood Manager, IT Business & Customer Services.
Advertisements

Data Extracts Working Group (DEWG) Update Tuesday, September 25, 2006.
9/13/2007 Nodal COMS Extracts & Reports Market Requirements Review SDAWG Jackie Ashbaugh ERCOT ***NOTE: All information contained in this presentation.
Draft for Discussion Day-ahead Settlement Timeline Options Commercial Operations Subcommittee May 23 rd, 2005.
04/21/2008 Nodal COMS RTM Extract Data Review SDAWG & DEWG Jackie Ashbaugh ERCOT ***NOTE: All information contained in this presentation is as of 04/17/2008.
Saturday May 02 PST 4 PM. Saturday May 02 PST 10:00 PM.
ERCOT Billing, Settlement Disputes & Data Extracts
Network Economy System Overview and Demonstration May 3, 2007 Raj Hundal.
Enterprise Information Services (EIS) - Review Ray Chase Manager, EIS.
1 Emerging Issues. 2 Enterprise Data Warehouse (EDW)
01/21/2008 Nodal COMS Extracts & Reports Market Requirements Review SDAWG Jackie Ashbaugh ERCOT ***NOTE: All information contained in this presentation.
1 AMS Data Workshop ERCOT Overview of AMS Data Processes June 27, 2014 ERCOT June 27, 2014.
PR Phase II SCR 727 SCR 740 Web Services Jackie Ashbaugh Commercial Operations Data Integrity & Administration August 18, 2006.
SCR 740 Implementation of Web Services for ESIID level data Jackie Ashbaugh EAA Data Management.
814_20 – Substation ID updates Background and Proposed Action Plan TX SET – 10/25/07.
PRR 568 – Settlement Timeline PRR 638 – Invoice Timeline PRR 599 – Mismatch Notification COPS January 2006 Updates.
Data Extracts & Reporting Recent Issues ERCOT Information Technology Data Extracts Working Group 11/27/07.
Lead from the front Texas Nodal 1 EDS 3 Release 5 Monday / Friday Market Updates November 30, 2007.
Market Impact Assessment TF Final Report to RMS June 11, 2008.
Retail Market Subcommittee Update to COPS Kathy Scott May 13,
June 22 and 23,  The information and/or flow processes contained in this Power Point presentation: ◦ Were created to allow interested parties to.
SCR786 - Alt Remove SIM dates from CERT Enhance CERT hardware and/or software Processing times at volume may not meet Performance Measures criteria Do.
Nodal Settlement Delta Calculations and Statements – 7/11/06 Workshop  Objective: Describe design concepts of nodal settlement delta calculations and.
June 10, 2009 RMS PR90006, Commercial Systems Information Lifecycle Management (ILM) Hope Parrish, ERCOT.
1 Nodal Stabilization Market Call December 06, :30 AM.
Mandy Bauld ERCOT October 9, 2012 RTM SETTLEMENT TIMELINE.
Overview of MID, MIR, & Extracts for DEWG 9/24/07 Steve Cox.
10/13/10 RMS RMS Market Reports – Recommendations Karen Farley Manager, Retail Customer Choice.
Changes to the Hours of Service Regulations For Truck Drivers Rob Abbott Vice-President of Safety Policy American Trucking Associations.
1 ESI ID SERVICE HISTORY AND USAGE DATA EXTRACT SYSTEM CHANGE REQUEST (SCR 727) February 24, 2003.
12/6/2015 ERCOT Production Issues Update ERCOT Matthew Tozer.
MARS Advanced Metering – ERCOT Market Facing Changes Jackie Ashbaugh Manager Data Integrity and Administration 3/9/2009.
Commercial Operations Subcommittee Update to TAC September 7, 2012 Harika Basaran, COPS Chair 2012 Jim Lee, COPS Vice Chair 2012.
Commercial Operations Subcommittee Update to TAC November 5, 2009.
1 Market Data Transparency SCR 740 Business & Technical Training SCR 727 Phase II: Web Services ERCOT presents: Market Data Transparency.
1 Next Steps to Reduce the RTM Settlement Timeline COPS Workshop August 29, 2013 Mandy Bauld ERCOT Director, Settlement & Retail Operations (512)
814_20 – Substation ID updates Background and Proposed Action Plan RMS – 11/07/07.
Distributed Renewable Generation Profile Implementation Plan.
1 RMS Update By Don Bender January 9, RMS Approved Resolution Upon TAC approval, suspend further True-up settlements for True-up resettlement.
Lead from the front Texas Nodal 1 EDS 3 Release 5 Monday / Friday Market Updates November 16, 2007.
Changes to SSO Extract:  Based on DEWG discussions in 2005, ERCOT is recommending the following changes to the SSOE: In lieu of the current daily files.
February 10, 2010 RMS ERCOT 1/24/10 Production Issue Overview and Lessons Learned Karen Farley Manager, Retail Customer Choice.
September 21, Data Extract Projects Jackie Ashbaugh Commercial Operations Data Integrity & Administration September 21, 2006.
01/09/2007 Extract Delays. 2 01/09/2007 EIS Extract Delays Problem –Extracts of Lodestar data from the ODS to the market have been delayed due to above.
Demand Response Task Force. 2 2 Outline  Overview of ERCOT’s role in the CCET Pilot  Overview of Stakeholder Process – What’s been done to date?  Questions.
February 19, 2009 ERCOT Follow up on questions from 2/11 discussion on proposed Expedited Switch rulemaking changes…
1DRAFT for DISCUSSION Transition From Non-IDR to IDR Load Profile and LSE 15-minute Data for AMS Market Advanced Readings and Settlements Taskforce 10/9/09.
Shortened RTM Settlement Timeline (NPRR509) Transition Plan - Phase 2 Matt Tozer / Ohlen Genove CSWG July 15, 2013.
1 Next Steps to Reduce the RTM Settlement Timeline COPS Workshop August 29, 2013.
Overview of also works.
February 25, 2009 ERCOT Follow up on questions from 2/18 meeting on proposed Expedited Switch rulemaking changes…
PR30026 Resource ID (RID) Extract and Recorder Extract Data Extracts Working Group May 18, 2006 Laura Zotter ERCOT Market Operations D P O.
COPS JULY 2013 UPDATE TO TAC 07/02/2013 Harika Basaran, Chair Jim Lee, Vice Chair.
PWG Demand Response Follow Up Jackie Ashbaugh October 23, 2007.
August 17, 2006 Data Extract Working Group EDW Project Updates 2006 / 2007 MO & RO PPL.
1.  What is the purpose of DEVs? Data Extract Variances (DEVs) are used to synchronize the data among all Market Participants (MP)  What is a DEV? It's.
3 rd Party Registration & Account Management SMT Update To AMWG May 24, 2016.
MODPO Project Update Overview of December Implementations & EDW Changes Commercial Operations Subcommittee December 11, 2006.
Role of Account Management at ERCOT EIS Project Updates Jackie Ashbaugh Mgr., Commercial Operations Data Integrity & Administration DEWG October 19, 2006.
-- Presentation from PWG -- Profile ID Assignment and Annual Review Process November 17, 2005.
02/01/2008 Nodal COMS Extracts & Reports Market Requirements Review SDAWG Jackie Ashbaugh ERCOT ***NOTE: All information contained in this presentation.
Nodal RTM Settlement Timeline Snapshot Timing for Load and Gen Data (Follow-Up) Mandy Bauld COPS November 10, 2009.
Settlement Timeline Workshop
MARS Taskforce RMS Update December 9, 2009.
DEWG Jackie Ashbaugh ERCOT
Revisit RTM True-Up Schedule and Invoicing
T+2 Live Date Last trade date for T+3 First trade date for T+2 Holiday
Sun Mon Tue Wed Thu Fri Sat
Sun Mon Tue Wed Thu Fri Sat
Sun Mon Tue Wed Thu Fri Sat
Presentation transcript:

ERCOT Lodestar Extracts Timing Recommendation ERCOT May 24, 2005

Overview Lodestar Extracts Current Extract Process for Lodestar Extracts from Data Archive ERCOT Recommended Process for Lodestar Extracts from ODS ERCOT Recommended Timing Process for Lodestar Extracts from ODS

Lodestar Extracts SCR 727 ESIID Extract Resource ID Extract (New) Load Extract Generation Extract Settlements & Billing Extract Settlement Input Data Extract (New) Shadow Pricing Extract Public Reference Table Extract (New)

SCR 727 ESIID Extracts Current Process from Data Archive Data Type: ESIID level data Data Flow Process: At 5:00AM, Lodestar Data Archive process runs to capture the existing records with addtimes/timestamps of previous day 4:00AM through current day 3:59:59AM Frequency: Generation and posting of extract occurs daily and is posted to market participants after being audited Extract Creation Trigger: Automatically by data inserts/updates/deletes

Load, Generation, and Settlements & Billing Extracts Current Process from Data Archive Data Type: – MOS and bill determinant trade date (Settlement Input Data) data as early as two days after operating day and may receive data changes up until the initial settlement – Data Aggregation and Settlement interval data outputs Data Flow Process: At 5:00AM, Lodestar Data Archive process runs to capture the existing records with addtimes/timestamps of previous day 4:00AM through current day 3:59:59AM Frequency: Extract process occurs Tuesday – Saturday only and is modified as necessary for holidays Extract Creation Trigger: By market participants scheduling the extract; Not dependent on Data Archive Process

Load, Generation, and Settlements & Billing Extracts Current Process from Data Archive – con’t Known Reasons for Improvement: – Potential for trade date outputs to be contained in multiple extracts – No dependency on settlement trade date approvals allowing trade date outputs to be received in multiple extracts with different addtimes/timestamps to account for backouts/reruns of unapproved settlements prior to posting an approved statement – Inconsistent synchronization of SCR 727 ESIID Extract data in conjunction with corresponding Load & Generation Extract postings

Shadow Pricing Extracts Current Process from Data Archive Data Type: Contains shadow pricing data Data Flow Process: At 5:00AM, Lodestar Data Archive process runs to capture the existing records with addtimes/timestamps of previous day 4:00AM through current day 3:59:59AM Frequency: Extract process occurs Tuesday – Saturday only and is modified as necessary for holidays Extract Creation Trigger: Automatically by data inserts

SCR 727 ESIID Extracts and Resource ID Extracts ERCOT Recommended Process from ODS Data Type: – SCR 727 ESIID Extract contains ESIID level data (granular data) – Resource ID Extract contains Resource ID level data (granular data) - New Data Flow Process: Continuous data flow from the Lodestar Source System -> Replicated Source System -> Lodestar ODS capturing intraday data updates and point in time selectivity Frequency: Generation and posting of extract occurs daily and is posted to market participants after being audited Extract Creation Trigger: Automatically by data inserts/updates/deletes

Recommended Changes: – Data load time window for the granular data Point in Time is captured by the initiation of granular data changes (Lodestar Batch usage data loading) at SYSDATE -3 to the next day’s initiation of granular data changes (Lodestar Batch usage data loading) – Produce on a 2 day lag from the data load time window stop time to accommodate for extract trade date synchronization and Lodestar ODS data synchronization including scheduled & unscheduled environment maintenance and troubleshooting data & system issues – Data included within daily extract is synchronized to one posting of a complete batch of trade dates for the Load, Generation and Settlements & Billing Extracts Timing proposal discussed with DEWG at February - May 2005 meetings SCR 727 ESIID Extracts and Resource ID Extracts ERCOT Recommended Process from ODS – con’t

SCR 727 ESIID Extract and Resource ID Extract Timing Change Day Extract is Generated and Published Current Data Addtime Time Window Recommendation Data Addtime Time Window MondaySunday 4:00AM - Monday 3:59:59AMFriday ~4:00:00 AM – Saturday ~3:59:59 AM TuesdayMonday 4:00AM - Tuesday 3:59:59AMSaturday ~4:00:00 AM – Sunday ~3:59:59 AM WednesdayTuesday 4:00AM - Wednesday 3:59:59AMSunday ~4:00:00 AM – Monday ~3:59:59 AM ThursdayWednesday 4:00AM - Thursday 3:59:59AMMonday ~4:00:00 AM – Tuesday ~3:59:59 AM FridayThursday 4:00AM - Friday 3:59:59AMTuesday ~4:00:00 AM – Wednesday ~3:59:59 AM SaturdayFriday 4:00AM - Saturday 3:59:59AMWednesday ~4:00:00 AM – Thursday ~3:59:59 AM SundaySaturday 4:00AM - Sunday 3:59:59AMThursday ~4:00:00 AM – Friday ~3:59:59 AM

Load, Generation, and Settlements & Billing Extracts ERCOT Recommended Process from ODS Data Type: – Load and Generation Extracts contain all outputs for a trade date in one extract – Settlement & Billing Daily Extract contains Settlements input data used and output data created for each approved trade date for all trade dates within a batch run becoming a complete trade date specific extract – Provide complete batch run with all Trade Dates approved accounting for subsequent Trade Date reruns prior to Settlements Approval Data Process Flow: Continuous data flow from the Lodestar Source System -> Replicated Source System -> Lodestar ODS capturing intraday data updates and point in time selectivity

Load, Generation, and Settlements & Billing Extracts ERCOT Recommended Process from ODS – con’t Frequency: Generation and posting of extract occurs daily and is posted to market participants after being audited Extract Creation Trigger: By market participants scheduling the extract Recommended Changes: – With extract creation and posting delay, No additional logic necessary to accommodate for receipt of unapproved Trade Date data and extract schedule changes – Up to an additional 3 day lag to the delivery timeline to accommodate for synchronization of extracts with approved settlement data (weekends & holidays), Lodestar ODS data synchronization including scheduled & unscheduled environment maintenance and troubleshooting data & system issues

Recommended Changes: – Consistent synchronization of SCR 727 ESIID Extract data and RID Extract data in conjunction with corresponding Load & Generation Extract postings – Granular data necessary to shadow settle is posted 2 days prior to the Aggregation and Settlement extracts Timing proposal discussed with DEWG at May 2005 meeting Load, Generation, and Settlements & Billing Extracts ERCOT Recommended Process from ODS – con’t

Load, Generation, and Settlements & Billing Extracts Timing Change Day Extract is Produced and Published Current Data Addtime Time Window Batch Trade Dates for due Settlement Statements Recommendation Data Time Window MondayNo extract produced and publishedThursday, Friday, SaturdayWednesday Batch Trade Dates TuesdayFriday 12:30 AM - Monday 12:29:59AM (Fri, Sat, Sun batch) SundayThursday Batch Trade Dates WednesdayMonday 12:30AM - Tuesday 12:29:59AM (Mon batch) MondayFriday Batch Trade Dates ThursdayTuesday 12:30AM - Wednesday 12:29:59AM (Tues batch) TuesdaySaturday Batch Trade Dates FridayWednesday 12:30AM - Thursday 12:29:59AM (Wed batch) WednesdaySunday Batch Trade Dates SaturdayThursday 12:30AM - Friday 12:29:59AM (Thurs batch) No approved statements posted Monday Batch Trade Dates SundayNo extract produced and publishedNo approved statements posted Tuesday Batch Trade Dates

Settlement Input Data Extracts (New) ERCOT Recommended Process from ODS Data Type: Contains all input data passed from MOS to the settlement system or entered by settlement personnel; data that is currently provided in the Settlement & Billing Extracts Data Flow Process: Continuous data flow from the Lodestar Source System -> Replicated Source System -> Lodestar ODS capturing intraday data updates and point in time selectivity Frequency: Generation and posting of extract occurs daily and is posted to market participants after being audited Extract Creation Trigger: By market participants scheduling the extract

Settlement Input Data Extracts (New) ERCOT Recommended Process from ODS – con’t Recommended Changes: – Redefines Settlement & Billing Extract to only provide complete approved trade date inputs and outputs synchronized with the Load and Generation Extracts – Data load time window is captured by the initiation of granular data changes (Lodestar Batch usage data loading) at SYSDATE -3 to the next day’s initiation of granular data changes (Lodestar Batch usage data loading)

Settlement Input Data Extracts Timing Change Day Extract is Produced and Published Current Settlements & Billing Extract Data Addtime Time Window Recommendation New Extract Data Addtime Time Window MondayNo extract produced and publishedFriday ~4:00:00 AM – Saturday ~3:59:59 AM TuesdayFriday 12:30 AM - Monday 12:29:59AM (Fri, Sat, Sun batch) Saturday ~4:00:00 AM – Sunday ~3:59:59 AM WednesdayMonday 12:30AM - Tuesday 12:29:59AM (Mon batch) Sunday ~4:00:00 AM – Monday ~3:59:59 AM ThursdayTuesday 12:30AM - Wednesday 12:29:59AM (Tues batch) Monday ~4:00:00 AM – Tuesday ~3:59:59 AM FridayWednesday 12:30AM - Thursday 12:29:59AM (Wed batch) Tuesday ~4:00:00 AM – Wednesday ~3:59:59 AM SaturdayThursday 12:30AM - Friday 12:29:59AM (Thurs batch) Wednesday ~4:00:00 AM – Thursday ~3:59:59 AM SundayNo extract produced and publishedThursday ~4:00:00 AM – Friday ~3:59:59 AM

Shadow Pricing Extracts ERCOT Recommended Process from ODS Data Type: Contains shadow pricing data Data Flow Process: Continuous data flow from the Lodestar Source System -> Replicated Source System -> Lodestar ODS capturing intraday data updates and point in time selectivity Frequency: Generation and posting of extract occurs daily and is posted to market participants after being audited Extract Creation Trigger: Automatically by data inserts Recommended Changes: – Data load time window is captured by the initiation of granular data changes (Lodestar Batch usage data loading) at SYSDATE -3 to the next day’s initiation of granular data changes (Lodestar Batch usage data loading)

Settlement Input Data Extracts Timing Day Extract is Produced and Published Current Data Addtime Time Window Recommendation Data Addtime Time Window MondayNo extract produced and publishedFriday ~4:00:00 AM – Saturday ~3:59:59 AM TuesdayFriday 12:30 AM - Monday 12:29:59AM (Fri, Sat, Sun batch) Saturday ~4:00:00 AM – Sunday ~3:59:59 AM WednesdayMonday 12:30AM - Tuesday 12:29:59AM (Mon batch) Sunday ~4:00:00 AM – Monday ~3:59:59 AM ThursdayTuesday 12:30AM - Wednesday 12:29:59AM (Tues batch) Monday ~4:00:00 AM – Tuesday ~3:59:59 AM FridayWednesday 12:30AM - Thursday 12:29:59AM (Wed batch) Tuesday ~4:00:00 AM – Wednesday ~3:59:59 AM SaturdayThursday 12:30AM - Friday 12:29:59AM (Thurs batch) Wednesday ~4:00:00 AM – Thursday ~3:59:59 AM SundayNo extract produced and publishedThursday ~4:00:00 AM – Friday ~3:59:59 AM

Public Reference Table Extracts (New) ERCOT Recommended Process from ODS Data Type: Contains full tables of non-interval market level data (ex. TDSP, STATION, CMZONE, etc.) from Lodestar extracts Data Flow Process: Continuous data flow from the Lodestar Source System -> Replicated Source System -> Lodestar ODS capturing intraday data updates and point in time selectivity Frequency: Generation and posting of extract occurs daily and is posted to market participants after being audited; only one daily extract available (no historical postings); delivery timeline same as SCR 727 ESIID and Resource ID Extracts Extract Creation Trigger: Automatically by data inserts/updates/deletes

What’s Next? Comments, Feedback, Questions ERCOT complete Analytical Designs of Execution Phase for the following projects: – PR – Resource ID Extract – PR – SCR 727 Phase II (SCR 727 ESIID Extract and Public Reference Table Extract) – PR 40042_06 – Lodestar Batch Extracts (Load, Generation, Settlements & Billing, Settlement Input Data, and Shadow Pricing Extracts)