1. DNV GL © SAFER, SMARTER, GREENER DNV GL © ENERGY Industrial, Agricultural and LARGE Commercial- 4(IALC4) 1 PY2013 NRNC WHOLE BUILDING IMPACT EVALUATION RESULTS April 20, 2015 CPUC and DNV GL NRNC WB Impact Team

2. DNV GL © Topics Overview of IALC4 NRNC Whole Building Impact Evaluation IALC4 NRNC Whole Building Impact Element Evaluation Results  Gross Impact Findings  Net Impact Findings  Project Practice Assessment Findings(PPAs) NRNC WB Findings and Recommendations Questions? 2

3. DNV GL © Overview of NRNC Whole Building Evaluation  NRNC Whole Building projects that received incentives under the state-wide Savings By Design (SBD) program  The scope of work includes an independent estimation of gross and net savings and development of findings and recommendations that can be used to improve program, project, and measure effectiveness  Three main evaluation activities supported the findings and recommendations:  M&V activities for estimating gross impacts  Professional telephone survey data collection supporting net to gross (NTG) estimation  Engineering reviews of the gross sample points to support the qualitative project practices assessment (PPA) 3

4. DNV GL © NRNC Whole Building Impact Evaluation Portfolio 4

5. DNV GL © Claimed Energy Savings by IOU for 2013 Projects in the NRNC Work Order Energy Savings Claims by IOU IOU Electric Energy (GWh) Electric Demand (MW) Gas Energy (Million Therms) NRNC WB Savings Claim PG&E35120.65 SCE2270.14 SDG&E1030.03 Total68220.83 Portfolio Savings Claim PG&E1,49024235 SCE7861421 SCG8626 SDG&E322472 Total2,60643664 NRNC WB Percentage of Portfolio Claim PG&E2%5%2% SCE3%5%14% SDG&E3%6%2% Total2.6%5.0%1.3% 5

6. DNV GL © NRNC Whole Building Impact Element  IALC4 NRNC Whole Building Evaluation addressed each of the following:  Verified installed measures and confirmed that they were operating as intended  Evaluated project savings- Engineering file review, on-site survey, end-use metering, energy modelling, updated the savings using observed site condition  Estimated ex post gross energy and demand savings  Estimated ex post NTG ratios and net savings  Assessed expected useful life(EUL) and calculated life cycle savings 6

7. DNV GL © NRNC WB Impact Schedule  Gross Impact Evaluation  Focused on estimation of gross realization rates for statistically representative samples of NRNC Whole Building projects  Utilized project specific measurement verification(M&V) to estimate gross realization rates  Analysis involved onsite survey, short-term monitoring, simulation modelling with EnergyPro and eQuest and model adjustments/ calibration with end-use/billing data to calculated ex post impact of the sampled points  Net Impact Evaluation  NTG evaluation consisted of an interview-based evaluation for the same sample points included in the gross impact activities to yield NTG ratio  25 gross impact sample points and 25 net points for program year 2013 7

8. DNV GL © NRNC Sample by IOU 8  Gross and net impact sampling by IOU  Electric and gas savings were collapsed  Sampling and analysis used source energy equivalents* IOUs Sample Points Sampled Source MMBtu NRNC Source MMBtu Sample % of Population MMBtu Sample Points Stratum 1 Sample Points Stratum 2 Sample Points Stratum 3 PG&E1583,081424,31620%456 SCE554,680244,82322%320 SDG&E519,450109,00118%113 Total25157,211778,14020%889 * Conversion rates obtained from “2001 Energy Efficiency Standards for Residential and Non-residential Buildings, California Energy Commission,” June 2001: 1 kWh = 10,239 Btu source energy; 1 Therm = 100,000 Btu source energy. 1 MMBtu =1,000,000 Btu

9. DNV GL © NRNC Whole Building Report Contents 1.Executive Summary 2.Introduction, Purpose and Methods 3.Sample Design 4.Methods 5.Gross Impact Results 6.NTG Results 7.Project Practice Assessment Results 8.Detailed Evaluation Findings and Recommendations 9

10. DNV GL © Gross Impact Findings Electric SavingsGas Savings Source Energy Savings Impact ElementkWh/yearAverage Peak kWTherms/yearMMBTU/year Tracking a. Claimed Gross Savings67,909,04921,886828,183778,139 b. Claimed GRR*0.9 c. Claimed Adjusted Gross Savings61,183,61519,710745,328700,992 Evaluation g. Evaluation GRR0.940.790.570.85 h. Evaluated Gross Results (h = a x g)63,903,10917,397474,104663,454 10 Statewide First Year Weighted Gross Realization Rate Estimates and Comparisons Energy Metric Sample Size (n) Gross Realization Rate Population (N) Error Ratio* 90% Confidence Interval RR > 1.5 RR = 0 RR < 0 PA Statewide kWh250.942390.810.89 - 0.99401 kW250.792390.900.75 - 0.84302 Therms250.572391.380.52 - 0.62355

11. DNV GL © Gross Impact Findings Electric SavingsGas Savings Source Energy Savings Impact ElementkWh Average Peak kW ThermsMMBTU/year Tracking a. Claimed Gross Savings1,072,032,790346,43113,026,36012,279,180 b. Claimed GRR * 0.9 c. Claimed Adjusted Gross Savings 965,863,066312,00011,723,15211,061,783 Evaluation g. Evaluation GRR 0.920.790.57 0.85 h. Evaluated Gross Results (h = a x g) 987,494,279274,3277,428,26210,266,156 11 Statewide Life Cycle Weighted Gross Realization Rate Estimates and Comparisons Energy Metric Sample Size (n) LC Gross Realization Rate Population (N) Error Ratio* 90% Confidence Interval RR > 1.5 RR = 0 RR < 0 LC GRR Mean PA Statewide kWh25 0.922390.790.88 - 0.973010.94 kW25 0.792390.910.75 - 0.843020.79 Therms250.572391.440.52 - 0.623550.57

12. DNV GL © First Year Weighted Ex-Ante and Ex-Post Source MMBTU Savings for Sampled Projects 12

13. DNV GL © First Year Weighted Ex-Ante and Ex-Post Electric Savings (kWh) for Sampled Projects 13

14. DNV GL © First Year Weighted Ex-Ante and Ex-Post Electric Demand Savings (kW) for Sampled Projects 14

15. DNV GL © First Year Weighted Ex-Ante and Ex-Post Gas Savings (Therms) for Sampled Projects 15

16. DNV GL © Gross Impact Findings(Cont’d)  The kWh GRRs for the sampled projects ranged from -0.46 to 2.28, the resulting overall kWh GRR of 0.92  A number of projects had kW GRRs less than 1, resulting overall program level kW GRR was 0.79  Gas GRRs were erratic with about half of the sample, resulting overall program therm GRR was 0.57  Sample points with refrigeration measures had relatively tight GRR ranges with kWh GRR ranging from 0.83 to 1.11 and kW GRR ranging from 0.97 to 1.07 16

17. DNV GL © Gross Impact Findings(Cont’d)  The four principal reasons that ex ante gross impacts differ from ex post results:  Differences in operating conditions  Differing baselines  Differing calculation methods  Model calibration, where models were adjusted to calibrate model energy using to billing-meter or end-use data 17 Other factors such as Tracking data discrepancy, Measure count, ineligible measure and inoperable measure didn’t have substantial impact on the WB projects

18. DNV GL © Factor Contribution to Total Discrepancy by Energy Metric 18

19. DNV GL © Net Impact Results Results Electric NTGRGas NTGR MMBTU NTGR Statewide Weighted NTGR0.530.510.53 90 Percent Confidence Interval0.51 - 0.560.49 - 0.540.51 - 0.55 Net Savings Relative Precision24%25%20% n Survey Completes25 N Sampling Units239 Final NTGR0.530.510.53 19 Net-to-Gross Ratios by Fuel Type

20. DNV GL © Net Impact Findings (Cont’d)  On a statewide basis, the net-to-gross ratio was estimated at 0.53 for kWh and 0.51 for therms. These NTGRs indicate a moderate level of free ridership  Evaluated NTGR falls below the NTGR for 2006-08 SBD program electric energy domain of 0.62 (However this included systems projects as well as Whole Building analyses)  The combined NTGR of 0.53 is likely not viewed as favorable, and it is below the PA claim of 0.65  Across all program groupings analyzed, financials were the most significant driver  The other important influencing factors were recommendations from consultants, compliance with organizations normal maintenance or equipment policies, corporate policy or guidelines, followed by non-energy benefits  Less important factors included prior experience with the measure(s) and program and the least important factor was recommendation from a vendor or manufacturer 20

21. DNV GL © IOU Statewide First Year Gross and Net Realization Rate Estimates and Comparisons Electric SavingsGas Savings Source Energy Savings Impact ElementkWh/year Average Peak kW Therms/yearMMBTU/year Tracking a. Claimed Gross Savings67,909,04921,886828,183778,139 b. Claimed GRR*0.9 c. Claimed Adjusted Gross Savings61,183,61519,710745,328700,992 d. Claimed NTGR0.65 0.640.65 e. Claimed Net Savings (e = c x d)39,564,19612,746476,312453,116 f. Claimed Net Realization Rate (f = b x d)0.58 Evaluation g. Evaluation GRR0.940.790.570.85 h. Evaluated Gross Results (h = a x g)63,903,10917,397474,104663,454 i. Evaluation NTG Ratio0.530.500.510.53 k. Evaluated Net Results (k = h x i )33,986,7988,765242,122353,595 l. Evaluation Net Realization Rate (l = g x i ) 0.500.400.290.45 m. Evaluated Net Savings as a Fraction of Claimed Net Savings (m = k / e) 0.860.690.510.78 21

22. DNV GL © IOU Statewide Lifecycle Gross and Net Realization Rate Estimates and Comparisons Electric SavingsGas Savings Source Energy Savings Impact ElementkWh Average Peak kW ThermsMMBTU/year Tracking a. Claimed Gross Savings1,072,032,790346,43113,026,36012,279,180 b. Claimed GRR*0.9 c. Claimed Adjusted Gross Savings965,863,066312,00011,723,15211,061,783 d. Claimed NTGR0.65 0.640.65 e. Claimed Net Savings (e = c x d)624,572,367201,7547,491,8357,150,258 f. Claimed Net Realization Rate (f = b x d)0.58 Evaluation g. Evaluation GRR0.920.790.570.84 h. Evaluated Gross Results (h = a x g)987,494,279274,3277,428,26210,266,156 i. Evaluation NTG Ratio0.530.500.510.53 k. Evaluated Net Results (k = h x i )525,197,744138,2163,793,5765,471,463 l. Evaluation Net Realization Rate (l = g x i )0.490.400.290.45 m. Evaluated Net Savings as a Fraction of Claimed Net Savings (m = k/e) 0.840.690.510.77 22

23. DNV GL © Project Practice Assessments(PPAs)  PPAs provide qualitative, cost-effective, program-specific, impact-oriented findings and feedback to the Program  PPAs are largely a “desk review” of project application paper work received from the PAs as part of the evaluation data request  PPA is broken down into several main categories such as:  Project eligibility  Baseline selection and documentation  EUL/RUL  Cost and incentive documentation and calculations  Calculation methods; models/methods, inputs, and assumptions 23

24. DNV GL © Project Practice Assessment Findings  Most of the projects met the project eligibility consideration except one project  From the 25 projects, only one project didn’t meet the baseline criteria  A single EUL value was provided for each project although the WB project comprised of multiple measures  Incremental cost documentation did not always meet quality or appropriate expectations. In some cases there were no documentation available to support the IMC value claimed in the tracking data  There were some large discrepancies between the tracking data incentive amount and the project document’s incentive amount for a number of sites 24

25. DNV GL © Detailed Evaluation Findings and Recommendations  Issue-1: Ex ante models are not properly trued-up to match physical as-built conditions of the buildings Recommendation-1: PAs should require their inspectors and engineers to perform on-site visits to visually verify that the proposed ECMs have been installed and are operating as intended and as simulated in the building model  Issues-2: PA submitted Energy models used compliance mode to estimate ex ante savings for the NRNC WB projects Recommendation-2: It is recommended to use compliance mode to assess the project eligibility using the NR T-24 Performance module, but ex ante savings should be determined using the NR Performance module; as-built design schedules should be used in both the baseline and post-retrofit models  Issues-3: Ex ante Energy model used inappropriate baseline systems and controls strategies to estimate the savings Recommendation-3: It is recommended that the PA modelers verify that the baseline model, specifically the mechanical and HVAC systems, is in accordance with the Title-24 ACM manual 25

26. DNV GL © Detailed Evaluation Findings and Recommendations (Cont’d)  Issues-4: Ex ante Savings do not account for Cogeneration or Site-generated Power Recommendations: Clearly assess impact of Cogeneration or other site-generation on ex ante savings Claims Issue-5: The NTGRs were estimated at 0.50 to 0.53 across fuel domain indicates NRNC WB has a moderate level of free ridership Recommendations: To reduce free ridership is to shift program delivery to attract earlier project involvement. One suggestion is to reduce incentive payments or disqualify projects that have completed and committed designs before program interactions began Issue-6: Inappropriate baseline designation was used in the ex ante savings calculation Recommendation-6: The evaluator recommends reviewing baseline building use type selection and confirming that it is the most appropriate option while generating the energy model. 26

27. DNV GL © Detailed Evaluation Findings and Recommendations (Cont’d)  Issues-7: A single EUL value was provided for each project although the WB project comprised of multiple measures Recommendation-7: EUL is crucial for lifecycle savings calculations, and the evaluator recommends collecting EULs for each measure of the Whole Building project from DEER and calculating the project EUL by weighting them by measure savings  Issue-8: Incremental cost documentation did not always meet quality or appropriate expectations. In some cases there were no documentation available to support the IMC value claimed in the tracking data Recommendation-8: We recommend standardizing incremental cost documentation where both the baseline and the installed measure cost data should be provided along with their reference sources to validate the incremental cost estimates  Issue-9:There were some large discrepancies between the tracking data incentive amount and the project document’s incentive amount for a number of sites Recommendation-9: The evaluator recommends having more transparency and thoroughness when documenting incentives. Incentive amounts in the project documents should match what was listed in the tracking data. 27

28. DNV GL © SAFER, SMARTER, GREENER www.dnvgl.com Thank you. Questions? 28 Name: Amit Kanungo Email: amit.kanungo@dnvgl.com Phone:510-891-0446