1. New Construction Studies Performed by: MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 EM&V of the Statewide Savings By Design Program Presenter: Matt Brost – matt.brost@rlw.com EM&V of the Statewide California Energy Star New Homes Program Presenter: Bob Kasman – robert.kasman@rlw.com

2. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 2 Agenda Non-residential New Construction  History  Project Goals  Methodology  2003 EM&V Results  2004-05 EM&V Plan  2004-05 EM&V Timing Residential New Construction  Background  02-03 EM&V Results  Evaluation Methodology  More Results  Conclusions  04-05 EM&V Plan

3. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 3 History RLW and Architectural Energy Corporation NRNC EM&V for SCE and PG&E since 1994  PY 94  DOE-2 Simulations, metering and Econometrics  PY 96, 98 and 99 Carry over  DOE-2 Simulations, limited metering and Difference of Differences 1999-2005 Savings By Design (A.K.A Building Efficiency Assessment)  EM&V becomes statewide study  Net effects by self-report  NP Spillover measured  Industrial projects start showing up in PY 2003 EM&V  2004-05 introduction of gas measures to EM&V

4. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 4 BEA Project Goals Gross and net whole building savings  Incented measures  All measures (participant spillover)  Energy and demand (coincident) Baseline assessment Program Process  Participant and non-participant DM interviews New constructions trends  Efficiency, design and construction practices, etc.

5. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 5 BEA Methodology Overview Stratified participant sample by kWh savings  Based on paid year, not program year Non-participants selected from FW Dodge  Matched sample 2001-2002  Representative sample 2003  Very few in 2004-05 On-site surveys DOE-2 Models Decision Maker Surveys Measure level net analysis  Informed by self-report  Modeled in DOE-2

6. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 6 2003 Statewide Gross Savings Results Two gross estimates  All measures  Includes non-incented measures better than Title 24 (Part. spillover)  Measures only  Applies to systems projects

7. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 7 2003 Statewide Net Savings Results Two net savings estimates  Participant net savings  Gross minus free-ridership  Comprehensive net savings  Gross minus free-ridership plus non-participant spillover

8. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 8 BEA Results Comparison Commercial  75-82% Comprehensive net-to-gross  59-76% Participant net-to-gross Industrial  35-59% Net to Gross  No 1999-2001 projects

9. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 9 Commercial Efficiency – 1998 and 2001 Title 24 (2003) Participant and Non-participant Energy Savings as a Percentage of Baseline Consumption – 2001 Title 24 versus 1998 Title 24, Commercial Sites Only (Unweighted)

10. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 10 2003 Commercial Efficiency Combined Participant and Non-participant Energy Savings as a Percentage of Baseline Consumption – Commercial Sites Only

11. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 11 Participant Feedback Importance of design assistance and analysis  Helps “sell” the measure  Corroborates internal decisions  Introduces new measures and technologies Incentives very important  Helps measures meet investment criteria  Makes measures easier sell  Insurance against savings uncertainty when considering new measures and technologies  Not likely to get participation without it

12. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 12 2004-05 EM&V Much of the same Also, many new aspects  Introduction of gas measures and savings analysis  Much larger more complex projects  Very few non-participants, nearly all participants (n=180)  Large proportion of projects “industrial”  End-use metering and DOE-2 calibrations PG&E add-on study  More projects and more metering  Load profile tool All due April 2007

13. EM&V of the Statewide California 2002-03 Energy Star New Homes Program Robert Kasman robert.kasman@rlw.com (707) 939-8823 x32

14. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 14 Agenda Non-residential New Construction  History  Project Goals  Methodology  2003 EM&V Results  2004-05 EM&V Plan  2004-05 EM&V Timing Residential New Construction  Background  02-03 EM&V Results  Evaluation Methodology  More Results  Conclusions  04-05 EM&V Plan

15. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 15 CA Energy Star Homes Program Statewide program (PG&E, SCE, SDG&E and SCG) Pays cash incentives to single family and multifamily developers/builders Started 2002 Funded by Public Goods Charge (PGC) Requires minimum 15% compliance margin increase over Title 24

16. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 16 CA Energy Star Homes Program Incentives Compliance margins are relative to Title 24 building code Package D (set of prescriptive measures)

17. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 17 Requirements for Effective Evaluation A well-conceived program theory and logic model Complete and accurate program tracking data Consideration of appropriate baseline data

18. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 18 Results - Dwelling Units by Utility & Type ’02-’03

19. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 19

20. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 20

21. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 21 Single Family Electricity (kWh) Net to Gross Ratios Statewide NTG > 1! (Yikes) How can this be?

22. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 22 Single Family Gas (Therms) Net to Gross Ratios Note statewide NTG is << 1 Why so different?

23. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 23 Evaluation Methodology – Engineering Analysis Based on Title 24 compliant energy modeling software On-site inspections: 110 SF homes, 123 MF structures Re-simulation of Title 24 energy models when differences found SF average “as-built” findings used to adjust tracking savings using ratio estimation at end-use level SF Difference of differences SF Billing analysis MF Builder/decision maker surveys for free ridership

24. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 24 Evaluation Methodology Quantity to estimateData sources usedAnalysis Methods SF simple gross savingsCHEERSData queries SF adjusted gross (ex post) savings On-site inspection data, CHEERSTitle 24 energy modeling, ratio estimation SF net ex post savingsSF adjusted gross savings, SF RNC baseline study, utility billing data Difference of differences, billing analysis SF free ridershipGross ex post savings, net ex post savings Gross - Net SF spilloverSF RNC baseline study, CHEERSHypothesis testing MF simple gross savingsCHEERSData queries MF adjusted gross savingsOn-site inspection data, CHEERSTitle 24 energy modeling, ratio estimation MF net ex post savingsMF builder surveysSERA MF free ridershipMF builder surveysSERA Ex ante savingsIOU PIPs, CHEERSNA

25. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 25 Methodology Definitions (reference)

26. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 26 Single Family Compliance Margins (110 ENERGY STAR® Homes)

27. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 27

28. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 28 Single Family Billing Analysis No conclusive results. Why?  Baseline study by RMST climate zones not high enough resolution  Behavioral and demographic variation further blur results  Billing data acquisition/quality issues

29. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 29 California 16 CEC Climate Zones

30. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 30 Multifamily Compliance Margins on MF Inspected Plans (n=123)

31. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 31 Multifamily Energy Savings

32. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 32 Multifamily Results Number of multifamily structures = 758 Number of multifamily dwelling units = 7,281 Annual Net kWh Savings = 93,599 kWh Annual Net Therms Savings = 233,258 therms Average Net to Gross =.63 (from surveys) Average free ridership = 37% Non-participant spillover not estimated

33. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 33 Conclusions ESH program resulted in significant energy savings Both implementers and evaluators based impacts on modeling software – results less conclusive New Title 24 code expected to have significant impact (beyond free-ridership) on multifamily projects Evaluation challenges:  Tracking database QC issues (primarily due to many input sources)  Baseline data issues (climate zones, sample sizes)

34. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 34 Conclusions Most single family homes not built to plan, but average energy savings are realized anyway Most multifamily projects built as planned, energy savings dominated by water heating SF free ridership rates dependent upon energy -- negative for electricity, positive for gas MF not like SF -- wide variation in type, design and energy modeling of multifamily projects… significantly more complex to conduct EM&V analysis MF high free ridership rates due to loopholes in Title 24. (Loopholes intended to be closed with the October ‘05 code revisions.)

35. MAESTRO/CALMAC Evaluation Showcase ● Pacific Energy Center ● July 26-27, 2006 35 04-05 EM&V Plan RLW Analytics currently conducting ’04-’05 ESH program evaluation Includes metering study of both SF and MF new construction yielding actual consumption, not modeled data Process and non-energy benefit analysis More comprehensive billing analysis in select climate zones Data acquisition issues