Download presentation
Presentation is loading. Please wait.
Published byViolet Dennis Modified over 9 years ago
1
Baseline Analysis CBP, AMP, and DBP Steve Braithwait, Dan Hansen, and Dave Armstrong Christensen Associates Energy Consulting DRMEC Spring Workshop May 7, 2014 May 2014 1
2
2 Presentation Outline Objectives Methodology Data Performance measures Aggregator program (CBP and AMP) results Demand Bidding Program (DBP) results
3
May 2014 3 Objective: Assess Performance of Alternative Baseline Types For each Utility and Notice type: All customers, with BL adjustment as chosen All customers, simulated with universal selection of the BL adjustment Sum of individual BL vs. portfolio BL (constructed from aggregated customer loads), for AMP and CBP only Examine unadjusted and day-of adjustments with 20%, 30%, 40%, 50% caps, and uncapped
4
May 2014 4 Analysis Details For actual program event days The “true” baseline is the estimated reference load from the ex post evaluation For event-like non-event days The “true” baseline is the observed load
5
May 2014 5 Performance Measures (1) Percentage Baseline Error Percentage BL error for each customer/portfolio- event day is: Percentage error = (L P d – L A d ) / L A d L A d = actual, or “ true ” baseline load on day d L P d = “predicted” baseline to be evaluated Positive value = over-estimated baseline (implies over-stated program load impact) Negative value = under-estimated baseline (implies under-stated program load impact)
6
May 2014 6 Performance Measures (2) Accuracy Accuracy is measured as the median absolute percentage error (MAPE) Calculate the absolute value of the percentage error for each customer/event-day Calculate the median of values across customer/event- days (mean can be misleading due to extreme values) Higher values correspond to larger baseline errors
7
May 2014 7 Performance Measures (3) Bias Bias is measured by the median percentage error, without taking the absolute value Positive values indicate upward bias (i.e., the program baseline tends to over-state the “true” baseline) Negative values indicate downward bias (i.e., the program baseline tends to under-state the “true” baseline)
8
Nominated Customers by Choice of BL Adjustment – CBP and AMP May 2014 8
9
9 Accuracy (Median Abs. % Error) PG&E CBP-DO
10
May 2014 10 Bias (Median % Error) PG&E CBP-DO
11
May 2014 11 Percentiles of % Errors – PG&E CBP-DO Actual Events, by Adjustment Cap
12
May 2014 12 Percentiles of % Errors – PG&E CBP-DO Simulated Events, by Adjustment Cap
13
Summary: Accuracy & Bias (Aggregated Indiv.; Universal Adj.; 40% cap) May 2014 13
14
Summary: Percentiles of % Errors (Aggregated Indiv.; Universal Adj.; 40% cap) May 2014 14
15
May 2014 15 Summary of Findings Accuracy and bias measures vary by utility, program and notice type Suggests that factors other than baseline type and adjustment caps may be most important, such as types of customers (e.g., highly variable load) and event-day characteristics (e.g., event on isolated hot day) Day-of adjustment often improves accuracy and reduces bias, but level of cap is less important Largest errors typically occur for Unadjusted BL and Unlimited cap BL with small median error (e.g., 1%) can have >10% errors in 20 percent of cases
16
May 2014 16 DBP Results: PG&E Distribution of % Errors
17
May 2014 17 DBP Results: SCE Distribution of % Errors
18
Summary Day-of adjustments tend to improve baseline accuracy and reduce bias The analysis provides support for making the day-of adjustment the default option The effectiveness of the day-of adjustment is not very sensitive to the level of the cap May 2014 18
19
May 2014 19 Questions? Contact – Steve Braithwait or Dan Hansen, Christensen Associates Energy Consulting Madison, Wisconsin Steve@CAEnergy.com Steve@CAEnergy.com Danh@CAEnergy.com Danh@CAEnergy.com 608-231-2266
20
Appendix SCE – CBP DO SDG&E – CBP DO PG&E – AMP DO SCE – AMP DO May 2014 20
21
May 2014 21 Accuracy (Median Abs. % Error) SCE CBP-DO
22
May 2014 22 Bias (Median % Error) SCE CBP-DO
23
May 2014 23 Percentiles of % Errors – SCE CBP-DO Actual Events, by Adjustment Cap
24
May 2014 24 Percentiles of % Errors – SCE CBP-DO Simulated Events, by Adjustment Cap
25
May 2014 25 Accuracy (Median Abs. % Error) SDG&E CBP-DO
26
Accuracy – Med. Abs. Err. (MW) SDG&E CBP DO May 2014 26
27
May 2014 27 Bias (Median % Error) SDG&E CBP-DO
28
May 2014 28 Accuracy (Median Abs. % Error) PG&E AMP-DO
29
May 2014 29 Bias (Median % Error) PG&E AMP-DO
30
May 2014 30 Accuracy (Median Abs. % Error) SCE AMP-DO
31
May 2014 31 Bias (Median % Error) SCE AMP-DO
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.