1. 1 ERCOT LRS Precision Analysis PWG Presentation February 27, 2007

2. 2 Options for Determining Round Two Sample Size Increases Option 1: –Determine minimum sample size needed to obtain ±10% Accuracy at 90% Confidence for a selected percent of intervals for the year independently for each Profile Type / Weather Zone Combination For Options 2 – 5 ERCOT recomputed daily energy totals and dollars –Used SAS data aggregation tool developed for transition analysis –Applied load profiles from new models to spread monthly LSEG totals from Lodestar to intervals –Multiplied by weather zone weighted MCPE to associate a dollar value with each interval Option 2: –Determine minimum sample size needed to obtain ±10% Accuracy at 90% Confidence for enough intervals to account for a selected percent of the MWh for each Profile Type / Weather Zone Combination Option 3: –Determine minimum sample size needed to obtain ±10% Accuracy at 90% Confidence for enough intervals to account for a selected percent of the dollars (ΣMWh * MCPE) for each Profile Type / Weather Zone Combination Option 4: –Iteratively allocate increments of 20 sample points to the Profile Type / Weather Zone Combination which produces the most gain in terms of reducing MWh estimation error Option 5: –Iteratively allocate increments of 20 sample points to the Profile Type / Weather Zone Combination which produces the most gain in terms of reducing Dollar estimation error

3. 3 Option 1 To obtain ±10% Accuracy at 90% Confidence By Profile Type and Weather Zone - Independent of Interval For example: to obtain ±10% Accuracy at 90% Confidence for 50% of the intervals for BUSHILF_COAST would require a sample size of 11 points

4. 4 Option 1 To obtain ±10% Accuracy at 90% Confidence By Profile Type and Weather Zone - Independent of Interval

5. 5 Option 1 To obtain ±10% Accuracy at 90% Confidence By Profile Type and Weather Zone - Independent of Interval

6. 6 Option 1 To obtain ±10% Accuracy at 90% Confidence By Profile Type - Independent of Interval For example: to obtain ±10% Accuracy at 90% Confidence for 50% of the intervals for all Profile Type/Weather Zone combinations would require a sample size of 8,447 points

7. 7 Option 2 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the MWH within Each Profile Type and Weather Zone For example: to obtain ±10% Accuracy at 90% Confidence for intervals accounting for 50% of the BUSMEDLF MWh would require a sample size of 1677 points

8. 8 Option 2 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the MWH within Each Profile Type and Weather Zone

9. 9 Option 2 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the MWH within Each Profile Type and Weather Zone

10. 10 Option 2 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the MWH within Each Profile Type For example: to obtain ±10% Accuracy at 90% Confidence for intervals accounting for 50% of the MWh for each of the Profile Type / Weather Zone combinations would require a sample size of 7,438 points

11. 11 Option 3 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the dollars within each Profile Type & Weather Zone Note: Dollars = Σ (MWh * MCPE)

12. 12 Option 3 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the dollars within each Profile Type & Weather Zone

13. 13 Option 3 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the dollars within each Profile Type & Weather Zone

14. 14 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the dollars within each Profile Type Continues on next slide For example: to obtain ±10% Accuracy at 90% Confidence for intervals accounting for 50% of the Dollars for each of the Profile Type / Weather Zone combinations would require a sample size of 6,969 points Option 3

15. 15 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the dollars within each Profile Type Option 3

16. 16 Class Level MWH & Dollars Not all Profile Type / Weather Zone Combinations are created equal in either MWh or Total Dollars (ΣMWh * MCPE) !

17. 17 Class Level MWH & Dollars Not all Profile Type / Weather Zone Combinations are created equal in either MWh or Total Dollars (ΣMWh * MCPE) !

18. 18 Class Level MWH & Dollars Not all Profile Type / Weather Zone Combinations are created equal in either MWh or Total Dollars (ΣMWh * MCPE) !

19. 19 Class Level MWH & Dollars Totals from previous three slides. Is accuracy more important for RESLOWR (33.4% of Dollars) than for BUSNODEM (1.6% of Dollars)?

20. 20 Class Level MWH & Dollars - Descending Order by Dollars * Note: Dollars = Σ (MWh * MCPE) Continues on next slide Top 5 classes account for 53% of the MWh and 54% of the dollars

21. 21 Class Level MWH & Dollars - Descending Order by Dollars * Dollars = Annual MWh * MCPE Continues on next slide Bottom 28 classes account for only 10% of the MWh and dollars

22. 22 Class Level MWH & Dollars - Descending Order by Dollars * Dollars = Annual MWh * MCPE Continues on next slide

23. 23 Precision vs Sample Size Increasing sample size has a diminishing return on precision improvement Error Ratio (thus Precision improvement) varies across Profile Types / Weather Zones and across intervals Thus the impact of adding sample points varies by Profile Type and Weather Zone

24. 24 Options 4 & 5 Options 4 & 5 iteratively allocate increments of 20 sample points to the next Profile Type / Weather Zone Combination in order to produce the most gain in –Reducing MWh (Option 4) estimation error (Precision × MWh) summed across all intervals –Reducing Dollar (Option 5) estimation error (Precision × Dollars) summed across all intervals The allocations are based on –The MWh (or Dollars) associated with each of the Profile Type / Weather Zone combinations in each interval –The Error ratio in each interval for each Profile Type / Weather Zone combination –The cumulative number of sample points allocated by preceding iterations (including the original sample size) –The precision improvement that would be realized by adding 20 sample points, and the diminishing return on that improvement Minimum Sample Size –Profile to profile migration resulted in numerous instances of small sample sizes within strata –Small sample sizes resulted in both accuracy degradation and the need to drop strata from load research analysis –A minimum of 3 strata will be specified for each Profile Type / Weather Zone combination –A minimum of 40 sample points will be allocated to each stratum –Minimum sample size per Profile Type / Weather Zone combination will be 120 Maximum Sample Size –Maximum for Business Profile / Weather Zone combinations set to 400 –Maximum for Residential Profile / Weather Zone combinations set to 600

25. 25 Option 4 – MWh Error Reduction Optimization Cumulative sample sizes are shown in increments of 1,000; they were determined iteratively in increments of 20 sample points

26. 26 Option 5 – Dollar Error Reduction Optimization

27. 27 Dollar Error Reduction Based on Sample Size Increases Non - Optimized Allocation of points Optimized Allocation of Additional Points

28. 28 Impact of UFE Allocation The precision of estimates produced by a sample design will affect the accuracy of the resulting profile models The profile model outputs will be adjusted for UFE Sample design should take the UFE adjustment into consideration Intuitively the iterative sample design process should minimize the impact of UFE adjustment –Used iterative design based on dollar error minimization ERCOT ran Monte Carlo simulations to evaluate the impact of UFE –Simulated sample outcomes for 48 Profile Type / Weather Zone combinations for all intervals of the 19 month analysis period –Adjusted sample outcomes for UFE –Compared MAPE before and after UFE adjustment –Replicated this process 500 times for 3 sample designs (8000, 9000, 10000 sample points)

29. 29 Impact of UFE Allocation

30. 30 Impact of UFE Allocation UFE Impact with the 3 selected sample size designs and levels is negligible

31. 31 Dealing with Oil and Gas Migrations Round two sample design and selection will be complete prior to BUSOGFLT implementation ERCOT has obtained list of Oil and Gas ESIIDs from TXU-ED and AEP –ERCOT will check with other TDSPs for additional ESIIDs The ESIIDs will be eliminated from sample design, selection and the analysis population Sample estimates will be adjusted to reflect ESIIDs which have not migrated to BUSOGFLT –Usage for these ESIIDs will be aggregated and profiled as flat load –No sample points will be needed for Oil and Gas ESIIDs

32. 32 Dealing with Primary Voltage Migrations In Round one sample design for most Profile Type / Weather Zone combinations 3 – 4 separate strata were set up for Primary Voltage ESIIDs –These strata usually had small sample sizes, many were 100% sampled –Profile migrations were particularly problematic in performing analysis as a result of empty or sparse strata ERCOT has done preliminary statistical analysis of the Primary Voltage population (excluding Oil and Gas ESIIDs) Profile Type / Weather Zone combination –Appear to have significant differences from Secondary Voltage ESIIDs in the same weather zone and profile type –Separate Primary Voltage strata will be beneficial for Round Two sample accuracy A single Primary Voltage stratum will be established in each Profile Type / Weather Zone combination were applicable –Each stratum will be allocated a minimum sample size (40) –Profile migration issues should be less significant to future analysis ERCOT plans to evaluate the introduction of Primary Voltage as a potential new profile type –Adoption would be contingent on significant sample (and load profile model) accuracy improvements for both Secondary and Primary Voltage ESIIDs –If adopted, migrations to the new profile type would not create future analysis issues –Augmented samples would probably be necessary to build adequate models for Primary

33. 33 Conclusions and Follow-up Actions The iterative sample point allocation process has intuitive appeal –Seems to allocate sample points where they do the most good –Would be expected to maximize UFE reduction –UFE allocation has negligible impact on the final accuracy ERCOT will be selecting a total sample size of 9,000 points for secondary voltage ESIIDs and also will select a sample of primary voltage ESIIDs … 40 per profile type / weather zone combination were applicable ERCOT will run MBSS to determine stratum boundaries based on annualized kWH and to allocate sample points to the strata ERCOT will then randomly select primary and replacement sample points based on the design and forward the sample lists to TDSPs in March ERCOT will update the sample tracking database with the new samples, sample points, and will add the sample points to the samples