1. CSWG/RCWG 05/11/2012 Price Validation and Correction Resmi Surendran Manager, Market Analysis

2. 2 Protocol - When Prices Are Final May 11, 2012ERCOT CSWG/RCWG Meeting 4.5.3(4) All DAM LMPs, MCPCs, and Settlement Point Prices are final at 1000 of the next Business Day after the Operating Day. After DAM LMPs, MCPCs, and Settlement Point Prices are final, if ERCOT determines that prices are in need of correction, it shall notify Market Participants and describe the need for such correction. DAM LMPs, MCPCs, and Settlement Point Prices cannot be changed unless the ERCOT Board finds that the DAM LMPs, MCPCs, or Settlement Point Prices are significantly affected by a software or data error. 6.3(4) All Real-Time LMPs, SASM MCPCs, and Real-Time Settlement Point Prices are final at 1600 of the next Business Day after the Operating Day. After Real-Time LMPs, SASM MCPCs, and Real-Time Settlement Point Prices are final, if ERCOT determines that prices are in need of correction, it shall notify Market Participants and describe the need for such correction. Real-Time LMPs, SASM MCPCs, and Real-Time Settlement Point Prices cannot be changed unless the ERCOT Board finds that the Real-Time LMPs, SASM MCPCs, or Real-Time Settlement Point Prices are significantly affected by a software or data error.

3. 3 Protocol – How Prices are Corrected 6.3 (3)(a) If it is determined that the Real-Time Settlement Point Prices are erroneous and correcting such prices will not affect the Base Points that were received by Qualified Scheduling Entities (QSEs), then ERCOT shall correct the prices before the prices are considered final in paragraph (4) below. 6.3 (3)(b) If it is determined that correcting the Real-Time Settlement Point Prices will affect the Base Points that were received by QSEs, then ERCOT shall correct the prices before the prices are considered final in paragraph (4) below and settle the Security-Constrained Economic Dispatch (SCED) executions as failed in accordance with Section 6.5.9.2, Failure of the SCED Process. 6.3 (3)(c) If the Base Points received by QSEs are inconsistent with the Real-Time Settlement Point Prices, then ERCOT shall consider those Base Points as due to manual override from the ERCOT Operator and settle the relevant Settlement Interval(s) in accordance with Section 6.6.9, Emergency Operations Settlement. 6.5.9.2 (2) For intervals that the SCED process fails to reach a solution, then the LMPs for the interval for which no solution was reached are equal to the LMPs in the most recently solved interval. For Settlement Intervals that the Real-Time Settlement Point Prices are identified as erroneous and ERCOT sets the SCED intervals as failed in accordance with paragraph (3)(b) of Section 6.3, Adjustment Period and Real- Time Operations Timeline, then the LMPs for the failed SCED intervals are equal to the LMPs in the most recently solved SCED interval that is not set as failed. ERCOT shall notify the market of the failure by posting on the MIS Public Area. ERCOT Board MeetingApr 19, 2011

4. 4 Price Correction Schemes and Timelines among ISOs May 11, 2012 Comparison of Pricing Schemes among ISOs NY ISOMid West ISO ERCOTISO New England PJMCAISO Ex-anteEx-postEx-anteEx-post Ex-ante Timeline for Price Corrections NY ISOMid West ISO ERCOTISO New England PJMCAISO One (1) day to reserve prices. Three ( 3 ) calendar days for investigation and correction. Five ( 5 ) business days. One (1) business day. Five ( 5 ) business days for RT; Three (3) business days for DA. One (1) business day for normal verification. Two (2) business days for notification. Ten ( 10 ) calendar days for corrections. Five ( 5 ) calendar days for validation and correction. Source: http://www.caiso.com/Documents/Agenda_Presentation-MarketPerformance_PlanningForumJan26_2012.pdf ERCOT CSWG/RCWG Meeting

5. 5 Reasons for Price Corrections NY ISO PJMISO NECAISOERCOT Hardware Outage √ √ √ Software Failure/ Instability/ Upgrade/ Migration √√√√√ Software Error/Limitation √ √√√ Operator Input Error √√√√ Data Error √√√√√ Telemetry Error √√√√ Posting Error √ √ Source: http://www.caiso.com/Documents/Agenda_Presentation-MarketPerformance_PlanningForumJan26_2012.pdf May 11, 2012ERCOT CSWG/RCWG Meeting

6. 6 Methods to Correct Prices NY ISOMid West ISO ERCOTISO New England PJMCAISO Replace with adjacent intervals; Replace with advisory intervals; Substitute with similar node price; Replace with other markets. Replace adjacent interval if it is less than 12 intervals; Replace the full hour if all intervals of the hour are affected. Recalculate prices; Replace with last successful interval; Substitute with similar node price; Recalculate prices to reasonably reflect the expected results without error. Recalculate prices; Replace with last successful interval if less than 6 intervals are affected; Replace the hour if more than 6 intervals are affected. Replace with adjacent interval; Rerun market; Selective recalculation; Replace with other market prices. Source: http://www.caiso.com/Documents/Agenda_Presentation-MarketPerformance_PlanningForumJan26_2012.pdf May 11, 2012ERCOT CSWG/RCWG Meeting

7. 7 ERCOT Historic Price Corrections Before prices became final Taken to Board for approval DAM 23 (Totally 8,020 SPPs corrected) 1 (Totally 4,579 SPPs corrected) RTM 39 (Totally 36,701 SPPs corrected) 2 (Totally 734 SPPs corrected) SASM 00 May 11, 2012ERCOT CSWG/RCWG Meeting

8. 8 ERCOT Price Validation Tool (PVT) Price Validation is done for:  DAM  Immediately after each DAM execution.  For each hour of the next Operating Day.  SCED  Automatically executed at real time.  For each SCED Interval and each Settlement Interval.  SASM  Automatically executed after each SASM run.  For each hour in the SASM. May 11, 2012ERCOT CSWG/RCWG Meeting

9. 9 Core Price Validation TestsTestsSub-testsDAMSCEDSASM PriceRecalculation Nodal LMP PTP Obligation Price CRR Options Price 15-minute SPP/RTRMPR Constraint SP Check Overload Constraint Penalty Check Binding Constraint Price Check Dispatch Consistency Tests Generation Ancillary Services PTP Obligation CRR Options Energy Only Offers/Bids JOU dispatch Commitment Tests Uplift Test Hypothetical Revenue Test May 11, 2012ERCOT CSWG/RCWG Meeting

10. 10 Core Price Validation TestsTestsSub-testsDAMSCEDSASM Marginal Units Check Energy Marginal Units Ancillary Service Marginal Units Offer Curve Check Offer Curve Pair Error Dead Bus Check Dead Bus LMP = λ with binding/violated constraints Duplicate Constraint Check Duplicate Constraints Shift Factor Check Resource SF Vs EB SF existence Resource SF = 0 EB SF = 0 No SF for active constraints SF on/off between adjacent SCED runs Multiple SFs for the same Constraint May 11, 2012ERCOT CSWG/RCWG Meeting

11. 11 Core Price Validation Tests (cont’d)TestsSub-testsDAMSCEDSASM LR RRS Check Load Resource RRS dispatch limit check Temporal Constraint Test Generator temporal constraint check System Consistency Check NCUC vs DSP price consistency check on Resource LMPs, SPPs, and PTP OBL/OPT Prices Generic Flow Limit Check Generic flow vs. Generic flow limit in DAM SPS vs Constraint Check Constraint solved by SPS is binding/violated Resource vs Resource Node Check LMP and SF consistency check on Resources and corresponding Resource Nodes May 11, 2012ERCOT CSWG/RCWG Meeting

12. 12 Core Price Validation Tests (cont’d)TestsSub-testsDAMSCEDSASM High Level Tests Price Spikes Check System Lambda spike and oscillation check LMPs 1-interval threshold limits 2-interval changes 3-interval oscillations Shadow prices 1-interval threshold limits 2-interval changes Shift factors 2-interval changes Base points 3-interval oscillations May 11, 2012ERCOT CSWG/RCWG Meeting

13. 13 Price Validation Statistics Market Type # of intervals# of price checksTypes # of validation checks DAM24 hours 544 Settlement Points 3,700+ CRR Options 42,000+ PTP OBL Error68 Warning8 Informative11 SCED 96 Settlement Intervals 544 Settlement Points 469 Meter Prices Error31 Warning18 Informative9 288+ SCED Intervals 10,189 Electrical Buses 449 Resources Error10 Warning5 Informative1 SASMcase by case Error28 Note: the above statistics are based on OD 5/8/12 snapshot. May 11, 2012ERCOT CSWG/RCWG Meeting

14. 14 Price Validation Team performs the following daily checks to identify potential pricing issues: –Error/Warning flags from automated daily Price Validation Reports –CRR/PTP Payout spike(s) –Price trends, congestion pattern and market convergence –Outages on heuristic table mapping If a potential pricing issue identified: –Upper management is notified –A price investigation market notice is sent out through –Price investigation is immediately conducted –A price correction may be proposed by the team If a price correction is approved internally by ERCOT managers: –A price correction market notice is sent out through –Price correction data is prepared and posted by deadline –Price corrections that past deadline are brought to Board Price Validation/Correction Process May 11, 2012ERCOT CSWG/RCWG Meeting

15. 15 Types of Errors for Previous Price Corrections ScenarioData Issues/Examples Planned Site failover Maintenance EMS Load Forecast application failed during planned Network Model Database Load. SCED Solutions of few subsequent intervals not sent to MPs (2/8/11) Planned Site Failover and Software Upgrade. 15 SCED Intervals failed during site failover execution. EBNRT data missing, stale EMS data, solutions of few subsequent intervals not sent to MPs (4/5/11) Planned Site Failover. 2 SCED intervals failed and solutions of few subsequent intervals not sent to MPs (6/7/11, 11/29/11) Planned Site Failover. 1 SCED interval failed and solutions of few subsequent intervals not sent to MPs (10/4/11) Planned Site Failover Maintenance for Software Migration. SCED was down for 55min (5/22/11) Planned Site Failover Maintenance for Software Migration. 11 SCED intervals were lost. EBP offset deployed (7/24/11) May 11, 2012ERCOT CSWG/RCWG Meeting

16. 16 Types of Errors for Previous Price Corrections ScenarioData Issues/Examples Hardware failure Node Eviction on EMS Database. Stale EMS data (5/10/11) Loss of Heartbeat on MMS Application Server. SCED intervals were lost and the solutions of few subsequent intervals not sent to MPs (7/21/11) Loss of Heartbeat on MMS Application Server. SCED intervals were lost, EBP offset deployed (7/23/11) Due to System issues (High RAC Interconnect Traffic & VIO Servers CPU high Utilization), SCED intervals got affected because failure of save case creation caused EMS Data Transfer to SCED to be delayed (8/13/11) Communication failure caused by following an incorrect process while decommissioning a DNS server. 8 SCED Intervals failed. (2/2/12) Unstable/ crashed database EBNRT missing – Unable to locate issue. (3/13/11) May 11, 2012ERCOT CSWG/RCWG Meeting

17. 17 Types of Errors for Previous Price Corrections ScenarioData Issues/Examples Application failure ETA application failure during planned Network Model load process. EB SF missing (12/15/10) QSE ICCP process stalled on both TCC and ACC sites. Suspicious telemetry data (4/1/11) RTCAM savecase issue. Constraint missing (5/25/11) EMS Applications failed to automatically restart following local failover during database load process (10/26/11) EMS STLF application did not calculate the short-term load forecast for 50 minutes. GTBD data error (11/6/11, 11/7/11) ETA application failure during planned Network Model load process (4/11/12) RLC Application had issue reading updated SCED base points. SCED solution was unavailable to EMS (4/26/12) May 11, 2012ERCOT CSWG/RCWG Meeting

18. 18 Types of Errors for Previous Price Corrections ScenarioData Issues/Examples Procedure Issue Incorrect monitor/ secure flag causing invalid constraint (11/7/11) Data transfer from MIINF staging tables to MFMASTER issue was affected by purging of the MIINF Audit Control Table entries within 7 days (1/7/12) Telemetry error SCED intervals failed due to huge telemetered MW value (8/12/11) Fat finger inputs Wrong offset (3/25/11) DAM Operator mistakenly ran phase 2 validation for 3/8/2012 on 3/8/2012 @ 7:30am (3/8/12) May 11, 2012ERCOT CSWG/RCWG Meeting

19. 19 Types of Errors for Previous Price Corrections ScenarioSoftware Issues/Examples Software error Incorrect RTRMPR calculation (4/1/11, 4/12/11) LCCRN Resources with NA status and non-zero BP were treated as offline (4/11/11, 5/10/11) Software Error in DAM NSM (5/11/11) Constraint missing in SCED due to an EMS issue which happens when operator activate/deactivate constraints in TCM before RTCA run and then if the RTCA run is aborted by running RT Sequence manually (5/11/11) SCED failed due to issues in capped and bounded curve logic (5/12/11) Incorrect EB Shift Factors caused by EMS defect (10/25/11, 10/26/11, 10/27/11, 11/8/11, 11/9/11, 11/11/11) Incorrect EB Shift Factors caused by EMS defect (11/17/11, 11/18/11) May 11, 2012ERCOT CSWG/RCWG Meeting

20. 20 Types of Errors for Previous Price Corrections ScenarioSoftware Issues/Examples Software error GTBD Replication not in place. Wrong GTBD. (7/15/11) Software Error in DAM DSP (10/17/11-01/21/11, 10/24/11- 20/27/12) Due to MMS defect, DAM LMP was wrongly assigned to the system lambda (12/6/11) Software Error in DAM NSM post-process (12/4/10, 12/3/10, 4/15/11, 4/8/11, 4/9/11, 4/10/11, 4/12/11, 8/25/11, 9/3/11, 9/5/11, 9/8/11, 2/10/12, 2/11/12, 3/7/12) May 11, 2012ERCOT CSWG/RCWG Meeting

21. 21 Why Is Price Correction Needed for Planned Events The calculation of Real Time Market Settlement Point Prices (SPPs) was Base Point weighted until the implementation of NPRR 326 & 355 on Feb 24, 2012. Even if everything went perfectly during a planned event, a price correction might still be needed. The calculation of Real-Time Price for the Energy Metered for each Resource (RTRMPRs ) remains Base Point weighted. Consequently price corrections may still be needed for future planned events even if they are accomplished perfectly. During planned events, SCED inputs may be stale or missing, or Base Points may not be telemetered to Market Participants due to data transfer issues. Consequently SCED LMPs and Base Points can be incorrect or inconsistent with what Market Participants have received. Price corrections are needed for such cases as well. ERCOT Board MeetingApr 19, 2011

22. 22 Types of Data Errors for Previous Price Corrections  Input data error o Incorrect/incomplete cost data, SF data, EBNRT, SPS, monitor flag o Stale/incorrect GTBD or telemetry o Incomplete list of de-energized buses o Dropping constraints  Output data error o Data transfer issues resulting in Base Points not being telemetered to Market Participants o Incorrect/incomplete data transfer o Price recalculation error  Software error o MMS defects  System failure infrastructure o Un-planned outages (Systems, Applications, Databases, ICCP telemetry) o Planned outages (Site failovers, Software releases) May 11, 2012ERCOT CSWG/RCWG Meeting

23. 23 FYI All market notices on price investigation and corrections can be found at: http://lists.ercot.com/scripts/wa-ercot.exe?A0=NOTICE_SETTLEMENTS NPRR456, Posting of Heuristic Table Database Load Schedule: http://www.ercot.com/gridinfo/transmission/opsys-change-schedule.html May 11, 2012ERCOT CSWG/RCWG Meeting