Page 1 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Summary of Findings and Recommendations May 21, 2002

Page 2 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential In the handout A more detailed report on findings and recommendations dated May 13th An appendix containing an interim findings discussion document dated May 2nd

Page 3 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Today’s Discussion Agenda Summary of Findings Summary of Recommendations Questions

Page 4 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Summary of Findings

Page 5 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Summary of Root Causes of Failures Probable Start up pains for all market participants –New transaction sets –New processes –New software –Many participants with many systems (new and legacy) –Tight time lines –New people in many jobs –Many “cooks in the kitchen” –Almost everybody is in some way learning how to do this for the first time Limitations in recognition and “graceful handling” of unexpected conditions in ERCOT systems exacerbated by: –Extraordinary transaction volumes –Transactions out of sequence or missing from sequence as defined by protocol –Limited ability to generate/communicate rejections to market ERCOT systems as defined and implemented are are not built to gracefully handle syntactically correct, but logically invalid TX SET transactions

Page 6 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Summary of Root Causes of Failures Occurrence of changing or evolving requirements definition during the later stages of system design and development. Limitations inherent in the approach to testing –Testing did not adequately cover Unexpected/aberrant data Out of sequence transactions Out of protocol delays Legacy systems dependent on transaction flows – Abbreviated integration testing across all affected market participants Inadequate end-to-end automated transaction flow management, monitoring and logging –“Leakage” –Not adequate as currently implemented –There should be zero “lost” transactions –A given transaction’s history and current status should be available at any point in time

Page 7 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Summary of Root Causes of Failures Side effects of “Safety Net” workarounds –Out of sequence transactions hitting ERCOT systems –ERCOT and market participant data out of sync resulting in future settlement problems Some parts of underlying IT application architecture appear to limit throughput resulting in delays in transaction processing

Page 8 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Other Key Issues Performance measurement in key transaction handling –Need to more completely define: What is necessary/expected of all parties? What are the related key measures and performance indicators? –Not just failed transactions, but also throughput How we will measure them? How we will ensure continuous improvement? –Focus should be on improvement, not punishment –Service Level Agreements, driven by the “real life” needs of the market must be solidified Role of ERCOT –Market needs Initiator/Coordinator/Driver for working key market issues Through key decision-making bodies (BOD, TAC, RMS, WMS, PRS) With all Market Participants (TDSPs, CR, QSEs etc) Acting as technical arbitrator (TDSP – CR) –Facilitator of thought leadership –Maintaining and actively championing a market-wide perspective –Taking an holistic approach –Roles evolving for ERCOT will likely require targeted increases in resources and in some cases different/stronger skill sets

Page 9 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Other Key Issues Volume of current recovery efforts is limiting resources available for efforts to achieve prevention ERCOT and especially ERCOT IT needs to continually look for ways to reduce the burden on the market participants in utilizing ERCOT services especially where doing so could improve the overall reliability of the transaction flow and the cost is acceptable With the market participants, ERCOT needs to define the acceptable risk profile in the areas of high availability/disaster recovery –Must agree to level of risk all parties will assume –Must agree on cost of agreed to profile Need to bring formal closure to the Accenture / System Acceptance / Warranty related questions

Page 10 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential What ERCOT does primarily…….. –Safely, efficiently, and effectively manage the flow of power across the electrical grid in covered areas within Texas –Manage the fair, expedient, efficient and reliable exchange of information needed to support all market transactions among the market participants, as well as provide central registration services This is a business in its own right.

Page 11 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Summary of Recommendations

Page 12 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Areas of Recommendations Communications across the Market Market Transaction Processing Pipeline –Monitoring –Performance –Scorecard –Workarounds –Functionality Other ERCOT IT and related issues Other ERCOT issues ERCOT’s role in the Market

Page 13 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Communications across the Market Improved market wide communication is needed to improve both problem prevention and problem recovery, as well as, consistent understanding of current status and performance of key transaction processing activities. This includes: –Unplanned events –Planned events –Technical issues –Business issues –Transaction status –Performance of key transaction processing services

Page 14 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Areas of Recommendations Market Transaction Processing Pipeline –Monitoring –Performance –Scorecard –Workarounds –Functionality

Page 15 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Retail Market Transaction Processing Pipeline Definition Market Participants

Page 16 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential ERCOT’s visibility into the Retail Market Transaction Processing Pipeline Visibility is limited by current capabilities of the underlying transaction processing software as implemented, especially at the interface point between major software modules. Market Participants

Page 17 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Monitoring should provide ERCOT’s visibility into the Retail Market Transaction Processing Pipeline Component Health (Up/Down) Performance & Throughput Exceptions & Events Transaction Status Market Participants

Page 18 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Component Health (Up/Down) Performance & Throughput Exceptions & Events Transaction Status ERCOT’s visibility into the Retail Market Transaction Processing Pipeline Monitoring Market Participants

Page 19 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Ensure the monitoring of key points and components within ERCOT’s transaction processing environment to be used in managing: –Availability –Performance/Throughput –Exceptions –Events Monitoring of Market Transaction Processing Pipeline

Page 20 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential “Beginning to End” monitoring of all key components and interface points within the transaction processing pipeline must be put in place –Increased market-wide visibility –ETS project –Retail market portal A “dashboard” must be defined and implemented that tells us the current status, health and performance of the transaction processing pipeline As dashboard alerts occur, a universally understood process must be followed for diagnosing and addressing the alert as well as reviewing for root cause determination. Monitoring of Market Transaction Processing Pipeline (cont.)

Page 21 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Performance of Market Transaction Processing Pipeline Need to improve the performance of ERCOT’s transaction processing and central registration services as defined by: –Availability –Performance/Throughput –Compliance with established protocols –Efficiency of operations –Number of exceptions and the effective handling of those exceptions

Page 22 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Solidify the FTP replacement project currently underway to improve visibility and reliability of that portion of the pipeline between ERCOT and the market participants Continue efforts to evaluate all existing hardware and software components to determine potential for upgrading to improve performance Review all key processing software components beginning with Siebel and PaperFree, to determine opportunities to establish multi-threading processing capability Performance of Market Transaction Processing Pipeline (cont.)

Page 23 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Market Transaction Processing Scorecard Develop and publish a Market Services Performance Reporting Scorecard –Define metrics and measures for successful Market Services operations. Establish appropriate Service Level Agreements (SLAs) Begins with - What is success in the market? Includes - What is success in the systems and processes supporting the market? –Define format, content, audience, and ongoing publishing requirements for scorecard –Collect relevant current volumes, success & error rates, and performance metrics –Measure against success criteria and publish scorecard via the WWW

Page 24 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Retail Market Transaction Processing Pipeline now has Workarounds outside of the original design Workarounds Market Participants

Page 25 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Market Transaction Processing Workarounds Fully understand the current workarounds and develop missing components to mitigate the adverse side-effects Ensure workarounds are either institutionalized or eventually eliminated by new transaction processing pipeline improvements Identify additional naturally occurring transactions that appear out of sequence to the protocol specifications, and develop workarounds as well as defining permanent solutions to be implemented in subsequent software releases

Page 26 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Market Transaction Processing Functionality Develop and implement plan to “harden” current transaction processing pipeline within ERCOT by: –Improving key error detection, notification and handling processes including EDI rejection detection, generation, and processing –Strengthening data validation in ERCOT systems specifically in Lodestar and Siebel –Ensuring the ability to reliably deliver, receive, trace and provide status of all transactions in and out of ERCOT –Reducing the constraints and limitations placed on market participants by ERCOT regarding the exchange of data

Page 27 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Market Transaction Processing Functionality (cont.) Future enhancements to the transaction processing pipeline must be properly sequenced and integrated TX SET 1.5 should be reviewed with the intent of first hardening the current transaction pipeline to establish a solid foundation upon which to add new functionality such as “stacked moves”

Page 28 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Areas of Recommendations Other ERCOT IT and related issues

Page 29 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Other ERCOT IT and related issues An increase in the capacity of ERCOT IT to resolve problems and build new functionality into the transaction processing pipeline is needed Remove obstacles to getting the most important work done and done well Additional IT resources within ERCOT are likely to be required One integrated list of all needed IT work/requests with a common agreed-to basis for setting priorities must be established and managed against A more complete and comprehensive approach to defining, estimating, costing and ensuring benefits are obtained in IT investments should be put in place ERCOT must develop, own and manage testing methodologies, “test cases” and the execution of testing of changes to the transaction processing pipeline. The scope of the ongoing testing must be materially expanded

Page 30 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Areas of Recommendations Other ERCOT issues

Page 31 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Other ERCOT related issues The capacity of ERCOT’s Retail Market Services support organization must be immediately expanded –Increased resources (ideally with contractors, probably Accenture) should be put in place within the existing “fix it” team to increase rate of problem resolution and prepare for summer volume Improve the utilization of ERCOT Retail Market Services support resources by: –Leveraging the recently revised Tiger Team processes for prioritizing, researching, and resolving market issues. –Continuing the Tiger Team problem research and problem resolution efforts at current staffing levels. –Begin moving Tiger Team responsibilities into Market Services as their capacity increases Overall improvement in problem management across market is needed

Page 32 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Areas of Recommendations ERCOT’s role in the Market

Page 33 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential ERCOT’s Role in the Market ERCOT should have an expanded role in the market defined by: –Assuming primary responsibility for ensuring the successful execution of standardized operations within the market –Providing the primary leadership in diagnosing, resolving and preventing transaction processing problems in the marketplace –Being the primary driver and facilitator of recognized market driven changes across the relevant market participants, committees, subcommittees, agencies, vendors and ERCOT Board of Directors –Owning, driving and being the authoritative “keeper” of core market processes and their ongoing evolution

Page 34 © 2002 the FeldGroup. All rights reserved. Proprietary and Confidential Next Steps for ERCOT and the Market Participants Confirm priorities as presented Launch near term efforts immediately Develop a formal plan including more detail estimates of what is required in terms of: –Time –Money –Resources Determine feasibility from a financial and personnel perspective Determine staffing implications and establish ownership/accountability Launch longer term investment efforts