1. AEMO: DR Mechanism Baseline Methodology DRM Working Group July 2013

2. DRM Working Group July 2013 Content  Purpose of Study and Approach  Overview of DRM Performance Evaluation Methods  US ISO Demand Response Mechanisms  Interview Results 2

3. DRM Working Group July 2013 Purpose of Study & Approach  Purpose: - How has DRM and baseline consumption methodology been implemented in other electricity markets - Identify components of DRM and baseline consumption methodology that perform well and those that need improvement in other electricity markets - What lessons can be learnt to advise AEMO on the development of the baseline consumption methodology for the DRM implementation in the NEM  Approach: - Literature review - US ISO interviews 3

4. DRM Working Group July 2013 Bibliography  The following documents were reviewed: 4

5. DRM Working Group July 2013 Overview of DRM Performance Evaluation Methods 5

6. DRM Working Group July 2013 NAESB Service Types and Performance Evaluation 6  The North American Energy Standards Board (NAESB) has developed Business Practice Standards for DR M&V, which have been adopted by Federal Energy Regulatory Commission (FERC) Source: Measurement and Verification for Demand Response Prepared for the National Forum on the National Action Plan on Demand Response: Measurement and Verification Working Group, Goldberg and Agnew, p. 20.

7. DRM Working Group July 2013 Baseline Type I and Type II  Type I - Baseline performance evaluation methodology based on a DR resource’s historical interval meter data which may also include other variables such as weather and calendar data - Criteria for Type I usually includes: - Baseline window - Exclusion rules - Calculation type - Baseline adjustment - Adjustment window  Type II - Baseline performance evaluation methodology that uses statistical sampling to estimate the electricity usage of an Aggregated Demand Resource where interval metering is not available on the entire population 7 Source: Measurement and Verification for Demand Response Prepared for the National Forum on the National Action Plan on Demand Response: Measurement and Verification Working Group, Goldberg and Agnew,.

8. DRM Working Group July 2013 Other Baselines  Maximum Base Load - Based solely on a DR resource’s ability to maintain electric usage at or below a specified level during an event  Meter Before / Meter After - Where electricity demand over a prescribed period of time prior to deployment is compared to similar readings during the sustained response period  Metering Generator Output - The demand reduction value is based on the output of a generator located behind the DR resource’s meter 8

9. DRM Working Group July 2013 Baseline Adjustments  Day-of-event adjustments may be made to align the baseline with observed conditions of the event day - Additive – ADD a fixed amount to the provisional baseline load in each hour, such that the adjusted baseline will equal the observed load at a time shortly before the start of the event period - Scalar – MULTIPLY the provisional baseline load in each hour by a fixed amount, such that the adjusted baseline will equal the observed load on average during a window of time shortly before the start of the event period  NAESB guidance: adjustment window shall begin no more than four hours prior to deployment 9 Source: Measurement and Verification for Demand Response Prepared for the National Forum on the National Action Plan on Demand Response: Measurement and Verification Working Group, Goldberg and Agnew, p. 20.

10. DRM Working Group July 2013 US ISO Demand Response Mechanisms 10

11. DRM Working Group July 2013 Location of ISOs 11 Source: http://www.isorto.org/site/c.jhKQIZPBImE/b.2604471/k.B14E/Map.htm

12. DRM Working Group July 2013 US ISO Demand Response Potential 12 *Note: PJM delivery year 2011-2012 Source: FERC, Assessment of Demand Response and Advanced Metering, 2011, p. 10.

13. DRM Working Group July 2013 California ISO (CAISO)  Demand response participation in other ancillary services markets is currently limited in the Western Interconnection by WECC rules  CAISO intends to address as part of a multi-year ancillary services redesign initiative and through its demand response initiatives such as the proxy demand resource product 13 Source: ISO/RTO Council, North American Demand Response Program Comparison, 2011. Source: ISO/RTO Council, ISO/RTO Metrics Report, 2010, p.60.

14. DRM Working Group July 2013 Electric Reliability Council of Texas (ERCOT) 14 Source: ISO/RTO Council, North American Demand Response Program Comparison, 2011.

15. DRM Working Group July 2013 ISO New England (ISO-NE) 15 Source: ISO/RTO Council, North American Demand Response Program Comparison, 2011.

16. DRM Working Group July 2013 Midwest ISO (MISO) 16 Source: ISO/RTO Council, North American Demand Response Program Comparison, 2011.

17. DRM Working Group July 2013 New York ISO (NYISO) 17 Source: ISO/RTO Council, North American Demand Response Program Comparison, 2011.

18. DRM Working Group July 2013 PJM Interconnection (PJM) 18 Source: ISO/RTO Council, North American Demand Response Program Comparison, 2011.

19. DRM Working Group July 2013 Customer Baseline Methodologies by ISO and Service Type 19

20. DRM Working Group July 2013 Overview of Baseline Methodologies 20  The most common type of baseline, the “X of Y” baselines: Note: These are generalizations of the weekday baseline calculation. Weekend baselines are calculated in a similar nature, but generally require fewer days (e.g., 4 most recent weekend days).

21. DRM Working Group July 2013 Overview of Baseline Methodologies (cont.)  ISO-NE 90/10 - Consists of a weighted average of the preceding day’s baseline (90%) and the current day’s actual metered load (10%) - The baseline is updated on every non-event weekday - It is not calculated on weekends or holidays - On (weekday) event days, the baseline is defined as the previous day’s baseline  ERCOT - Regression model to predict consumption - The explanatory variables in the model include: - calendar variables (e.g., day of the week, holiday indicators, season), - weather variables (dry-bulb temperature and various functions thereof), and - daylight variables (e.g., daylight saving time, times of sunrise and sunset) - Matching Day Pair - Compares loads on the “business-as-usual” hours of the event day itself up to one hour before the start of event plus the entire 24 hours of the preceding day to like day-pairs in the preceding year 21

22. DRM Working Group July 2013 Additional Baseline Methods 22

23. DRM Working Group July 2013 Interview Results 23

24. DRM Working Group July 2013 US ISO Interviews 24

25. DRM Working Group July 2013 Content of Interviews  Current baselines - Baseline methods and adjustments - Calculated prospectively or retro-actively? - Reasons for selecting method - Challenges when selecting method, now implemented - Evaluation  Future - Plans for making modifications  Participants - Process of selecting baseline for a participant - Process to resolve disputes - What if no historical data - ISO interaction with participants 25

26. DRM Working Group July 2013 Content of Interviews (cont.)  Data & Administration - Submission of data - What entities receive load data - Objectionable data - Exclusion of data - Software  Baseline Performance - Triggers for baseline review - Audits for performance - Changes to baseline, why and how often - Public reporting requirements  Gaming - Concerns, observations - Timing of notifications 26

27. DRM Working Group July 2013 Baseline Methodology Selection and Challenges  Reasons for selecting baselines: - Empirical performance - Administrative burden / simplicity of implementation - Minimize gaming/free-ridership - Looked to other ISOs with experience / precedence - Need to follow NAESB  Challenges, when selecting: - Getting everyone to agree on one baseline (weighing simplicity vs. accuracy) - Industry had little or no experience with baselines  Challenges, now implemented: - Large volumes of data to store - Methods are somewhat sophisticated and hard for participants to understand - Computationally intensive to evaluate all choices for each customer - Determining an accurate baseline for customers with highly variable loads 27

28. DRM Working Group July 2013 Current Baselines – Issues and Evaluation Activity  ERCOT currently has a pilot for weather sensitive customers  PJM is working on determining a baseline for customers with highly variable loads  ISO-NE is analyzing the effects of inaccurate meter data on its rolling average baseline 28

29. DRM Working Group July 2013 Participant Baseline Selection Process  ERCOT tests all the baseline methods offered for each participant at the time of registration and authorizes the most accurate baselines - The participant makes the final selection  PJM requires the participant to use the CBL with a relative root mean square error (RMSE) no greater than 20% - An alternative may be suggested by participant and used if proven to be more accurate  ISO-NE and CAISO only offer one baseline methodology  Disputes about baseline selection: - Handled by formal dispute resolution process at ISO - Not an issue at ISOs with only one method - MISO and PJM offer the option of the customer developing the baseline - ERCOT has not had any disputes given the options and frequent re-evaluation 29

30. DRM Working Group July 2013 Data Processing and Administration  ERCOT receives the data from the T&D service provider  CAISO, ISO-NE and PJM receive the data from demand response provider. - PJM, MISO and CAISO receive the data only after a DR event - ISO-NE continuously receives data every 5 minutes, in real time  ISOs receive already-validated or revenue grade meter data  ISOs also perform validation checks on the data  ISOs calculate the baseline and the load reduction for an event  Off-the-shelf software: ALSTOM (formerly UISOL) 30

31. DRM Working Group July 2013 Baseline Performance  CAISO and ISO-NE have one baseline  ERCOT evaluates all baselines for each participant when four-month contract is new/renewed  Upon registration, PJM certifies the CBL, requiring the RMSE <= 20%. Registration cycle is one year  Changing baselines: - PJM will review CBL when customer load significantly changes - Rarely do participants change baselines - ERCOT will review CBL if there was not enough data previously - Participants change CBLs “fairly frequently,” depends on contract period 31

32. DRM Working Group July 2013 Gaming  ISOs do have concerns about paying participants for something they would already be doing (free-ridership)  Some ISOs have not observed “strategic behavior”  One ISO observed gaming issues: - As season changed and loads were decreasing due to weather, participants locked in high baselines from high usage season, and then bid load when consumption was naturally low - Turned off behind-the-meter generator during baseline period, turned it back on during DR event  Participants are sophisticated, the rules need to be VERY clear  Concerned about gaming the adjustment period - Harder to catch than gaming the baseline period  Solutions have been to change the market rules and/or the baseline methodology 32

33. DRM Working Group July 2013 www.dnvkema.com www.dnv.com