1. CPUC Meeting on 2010 – 2012 Evaluation WO033 Custom Impact Evaluation Results May 2, 2014

2. TOPICS WO033 Custom Impact Overview WO033 Custom Impact Elements Evaluation Results Gross Impact Findings Net Impact Findings Lower Rigor Assessment (LRA) Findings Selected WO033 Findings and Recommendations Q&A/Discussion Emphasis on new sampling and evaluation approaches, tailored scope allowing for measure- and program-level results, and new recommendations 2 WO033 Slides - Stakeholders Mtg 5/2/2014

3. WO033 CUSTOM IMPACT OVERVIEW Evaluation of Custom (non-deemed measures) across multiple programs and the four major California IOUs  Core Calculated Programs  Third Party Programs  Local Government Partnerships and Universities / Colleges  And More!!  Over 100 programs within the WO033 scope Why is Custom Impact Evaluation Important?  Custom measures account for 19% of electric [1,754 GWh] and 65% of statewide natural gas savings claims [134 Gigatherms]  History of low realization rates for savings claims Not Just Another Custom Impact Evaluation 3 WO033 Slides - Stakeholders Mtg 5/2/2014

4. WO033 CUSTOM IMPACT ELEMENTS What’s New?  Two Periods, BD and AD, based on D.11-07-030  Large Gross Impact Sample  Expanded Net Impact Sample  Lower Rigor Assessment for Program-Level Impact Feedback  Interim Report for Early Feedback  Stand-Alone Net-to-Gross Report (forthcoming)  New Findings and Recommendations These evaluation elements support program-level results for “programs of interest” and some additional measure- specific results Not Just Another Custom Impact Evaluation 4 WO033 Slides - Stakeholders Mtg 5/2/2014

5. WO033 CUSTOM IMPACT SAMPLE SIZES Accomplishments (1)429 gross impact points across five IOU-fuel analysis domains, (2)telephone survey supporting net to gross (NTG) estimation and industry standard practice (ISP) determinations for a total of 1,388 projects, and (3)a total of 536 engineering reviews supporting LRA assessment 5 IOU/Fuel Sampling Domain Number of Completes Gross Impact (M&V)* NTGLRA PG&E Electric 112558155 PG&E Gas 7523097 SCE Electric 100367139 SDG&E Electric 7312573 SDG&E/SCG Gas 6910872 Total 4291,388536 * The 429 M&V sample points account for 495 electric and gas fuel-differentiated results. This is because some sample points included claims for both electric and gas savings. WO033 Slides - Stakeholders Mtg 5/2/2014

6. GROSS IMPACT FINDINGS Project Lifecycle Gross Realization Rates by Fuel Domain and Energy Metric (kWh, kW, and Therms) 6 WO033 Slides - Stakeholders Mtg 5/2/2014 * Lower value excludes extreme points.

7. PROJECT LIFECYCLE GROSS REALIZATION RATES BY SAMPLE DOMAIN AND ENERGY METRIC 7 Energy Metric Sample Size (n) Mean Gross Realization Rate Population (N) Error Ratio 90% Confidence Interval PG&E Electric kWh*1390.596,9941.270.49 - 0.70 kW1180.466,2481.670.35 - 0.58 PG&E Gas Therms*910.641,2700.580.58 - 0.70 SCE Electric kWh*1000.613,0521.030.51 - 0.71 kW940.572,7481.080.47 - 0.67 SDGE Electric kWh*770.551,4691.080.44 - 0.66 kW590.787902.200.43 - 1.14 SCG and SDG&E Gas Therms, all points881.451,0774.820.28 - 2.62 Therms, less extreme points 840.631,0770.840.54 - 0.73 WO033 Slides – Stakeholders Mtg 5/2/2014 Note: First year savings are also reported in Chapter 5.

8. LIFECYCLE EX-ANTE AND EX-POST ELECTRIC SAVINGS (KWH) FOR SAMPLED PROJECTS 8 WO033 Slides – Stakeholders Mtg 5/2/2014

9. LIFECYCLE EX-ANTE AND EX-POST GAS SAVINGS (THERMS) FOR SAMPLED PROJECTS 9 WO033 Slides – Stakeholders Mtg 5/2/2014

10. GROSS IMPACT FINDINGS (CONT’D) Program-specific and Measure–specific Findings 10 GRR results for program and measure groups with substantial sample sizes examined in detail Core, third party and new construction programs generally have above average GRR results While quite sensitive to outliers, lower GRR results for compressed air, HVAC and HVAC controls measure groups Higher GRR results for HVAC controls (PG&E Gas), HVAC (SCE), steam traps (SCG/SDGE) WO033 Slides - Stakeholders Mtg 5/2/2014

11. GROSS IMPACT FINDINGS (CONT’D) Reasons for Discrepancies in Realization Rates 11  Three principal reasons  observed operating conditions  baseline specification  the IOUs calculation methods Explain differences for 44%, 17% and 19% of the records studied Impact on savings  observed operating conditions -17% (electric), -14% (gas)  baseline specification-16% (electric), -14% (gas)  the IOUs calculation methods -2% (electric), -1% (gas) Other reasons for differences  incorrect equipment specifications  ineligible measures  incorrect measure counts  tracking database discrepancy WO033 Slides - Stakeholders Mtg 5/2/2014

12. FREQUENCY OF DISCREPANCY FACTORS 12 WO033 Slides – Stakeholders Mtg 5/2/2014

13. EFFECT OF DOMINANT DISCREPANCY FACTORS ON TOTAL EX-ANTE KWH SAVINGS CLAIMS 13 WO033 Slides – Stakeholders Mtg 5/2/2014

14. GROSS IMPACT FINDINGS (CONT’D) Non-Residential New Construction 14 43 sites in the NRNC measure group (25 EnergyPro) GRR: 81% (kWh), 72% (kW) and 51% (therms) Primary discrepancies  Operating conditions  Calculation methods  Inappropriate baselines EnergyPro-related issues  All whole building sites used T-24 schedules instead of the as- designed scheduled to estimate the ex-ante savings  Ex ante energy models were not created or updated to match the physical as-built conditions of the building  Operating under sub-optimal conditions, e.g., low occupancy Non EnergyPro-related issues  Use of inappropriate baseline on data center projects  Estimated I.T. loads on data centers were higher than observed WO033 Slides - Stakeholders Mtg 5/2/2014

15. GROSS IMPACT FINDINGS (CONT’D) Monitoring-Based Commissioning 15 10 sites in the MBCx measure group  GRR: 58% (kWh), 108% (kW) and 34% (therms) Primary discrepancies  Validity of ex ante regression model  Retrofit measures implemented during and after MBCx  Benchmarking of project sites  Reliability of energy meters and flawed metered data  Changes incompatible with the building equipment capability  Negative energy savings that may not be due to MBCx measures  Adjustments to energy usage baseline WO033 Slides - Stakeholder's Mtg 5/2/2014

16. NET IMPACT FINDINGS Net to Gross Completes Mix of CATI and In Depth/Professional Interviews Weighted NTGRs by program or program groupings 16 WO033 Slides - Stakeholders Mtg 5/2/2014

17. NET IMPACT FINDINGS Net to Gross Results 17 Results Electric NTGRsGas NTGRs StatewidePGESCESDG&EStatewidePGESDG&E/SCG Weighted NTGR0.470.460.490.450.530.560.50 90 Percent Confidence Interval0.46 - 0.48 0.45 - 0.480.47 - 0.500.43 - 0.480.50 - 0.570.53 - 0.58 0.42 - 0.57 Relative Precision0.030.04 0.060.070.050.15 n NTGR Completes1,050558367125338230108 N Sampling Units11,5156,9943,0521,4692,3471,2701,077 Error ratio (ER)0.520.590.440.430.790.460.99 WO033 Slides - Stakeholders Mtg 5/2/2014

18. NET IMPACT FINDINGS Net to Gross Findings Significant levels of free ridership have persisted  Statewide NTGRs of 0.47 (electric) and 0.53 (gas)  Similar in magnitude to NTGRs from prior cycles  (Significant improvement in PG&E gas NTGR from PY2006-2008) Little evidence of actions taken to reduce free ridership via changes in program design or implementation procedures Factors causing non-zero levels of free ridership:  Size and sophistication of customers.  Custom program design Despite these challenges, many strategies available to reduce free ridership  Many recommended in previous cycles 18 WO033 Slides - Stakeholders Mtg 5/2/2014

19. OVERALL NET REALIZATION RATES PG&E  kWh – 0.27, kW – 0.22, Therms – 0.36  Evaluated as % of claimed – kWh – 44%, kW – 34%, Therms – 59% SCE  kWh – 0.30, kW – 0.28  Evaluated as % of claimed – kWh – 49%, kW – 46% SDG&E/SCG  kWh – 0.25, kW – 0.36, Therms – 0.72  Evaluated as % of claimed – kWh – 50%, kW – 53%, Therms – 124%* * 54% excluding extreme points. 19 WO033 Slides - Stakeholders Mtg 5/2/2014

20. LRA Overview LOWER RIGOR ASSESSMENT FINDINGS Uses a desk review process similar to EAR Provides early feedback based on qualitative information Findings by program/program group, fuel domain, and IOU Expanded LRA form in AD period Results supported by a minimum of 8 assessments/category Low-frequency issues (n<100) use “GNP” denominator, see next slide Tables highlight  strengths of project documentation  related patterns, trends  areas for improvement LRA Score in Final Report – program-level LRA performance 20 WO033 Slides - Stakeholders Mtg 5/2/2014

21. LOWER RIGOR ASSESSMENT FINDINGS (CONT’D) 21 WO033 Slides - Stakeholders Mtg 5/2/2014 Final Report Table 7-1: LRA Complete Portfolio Sample Issue areas with fewer than 100 responses included only LRAs with valid responses in the denominator as shown in “Assessment Results” column, otherwise the count of all LRAs (536).

22. LOWER RIGOR ASSESSMENT FINDINGS (CONT’D) LRA Results Wrap-up Issues with high rate of deficiencies (percent “poor” shown in parentheses): Project documentation (26%) Pre- and post-retrofit M&V (22% and 30%) Calculation methods and inputs, specifically HVAC interactive effects (33%) Issues with high rates of poor assessments within lower-frequency issue areas Accounting for non-IOU fuel and ancillary impacts (73%) Accounting for multiple IOU fuel impacts (52%) Fuel switching supported with 3-prong test (69%) Lower rigor assessment qualitative outcomes are not necessarily indicative of gross impact evaluation results (LRA Score vs GRR: R 2 =0.04 for kWh) 22 WO033 Slides - Stakeholders Mtg 5/2/2014

23. LOWER RIGOR ASSESSMENT FINDINGS (CONT’D) 23 Final Report Table 7-3: IOU Fuel Domain Program Performance WO033 Slides - Stakeholders Mtg 5/2/2014

24. WO033 REPORT RECOMMENDATIONS Project Savings Estimation  Improve implementation and QC of project operating conditions, ex-ante baselines, calculation methods  Improve accuracy of savings in IOU tracking databases based on post-installation inspections and M&V  Improve engineering and simulation modeling for novel measures and projects with limited access to data Sufficient Quality Control for Documentation and Tracking Data  Increase consistency between project files and tracking data  Minimize miscommunication of project claims.  Provide additional scrutiny and QC o Accurate, comprehensive project documentation o Internally reviewed o Consistent with tracking system entries  Provide electronic access to complete documentation 24 WO033 Slides - Stakeholders Mtg 5/2/2014

25. WO033 REPORT RECOMMENDATIONS (CONT’D) IOU Project Verification  Perform pre and post-retrofit verification and M&V more frequently, document results completely  Goal to accurately identify o pre-existing equipment o ex-ante operating conditions o baseline considerations, and other issues LRA Evaluation  Reduce documentation inconsistencies  Improve impact calculation methods  Comply with program rules or justify exceptions  Conduct LRA-type reviews to enhance project reviews 25 WO033 Slides - Stakeholders Mtg 5/2/2014

26. WO033 REPORT RECOMMENDATIONS (CONT’D) Nonresidential New Construction  Use as-built schedule to update estimates of ex-ante savings and use “non-compliance” mode calculations for savings  Submit a Title-24 acceptance test report, perform site visits to verify key measures and revise energy model to physical “as-built” conditions  Provide technical outreach assistance to all program participants  Thoroughly review and make appropriate use of the data center baseline document  Conduct an ISP study focused on changes in load over time for various types and sizes of data centers 26 WO033 Slides - Stakeholders Mtg 5/2/2014

27. WO033 REPORT RECOMMENDATIONS (CONT’D) MBC X  Develop guidelines for regression modeling, review early stage projects  ensure conformance, applicability, exceptions  Keep centralized records of all building operation changes, especially for higher education campuses  For project benchmarking, use historical data for buildings with similar characteristics, test modeling with independent variables  Use meters that conform to the adopted metering guidelines  Verify that MBCx recommendations address the control system. Recommended changes must be compatible with existing system  Retain records of each measure implemented, along with post- implementation functional performance tests  List specific activities done, verify work was done correctly  Perform more extensive and thorough data collection, analysis, model development and validation of the engineering models 27 WO033 Slides - Stakeholders Mtg 5/2/2014

28. WO033 REPORT RECOMMENDATIONS (CONT’D) Program Influence/NTG  Adopt procedures to screen for and increase efficiency levels for high likelihood free ridership projects. Advise customers to o move to a higher level of efficiency o undertake a bundled retrofit to ensure deeper savings o move to another EE project under consideration  Carefully review the list of qualifying measures; eliminate eligibility for standard practice measures  Make changes to the incentive design. Tier incentives by technology class. Consider adopting a payback floor. 28 WO033 Slides - Stakeholders Mtg 5/2/2014

29. THANK YOU www.itron.com CONSULTING & ANALYSIS GROUP 1111 Broadway Oakland, CA