CALMAC July 18, 2007 Meeting Attribution and Net to Gross Examples for Discussion Clark Bernier, RLW Analytics, Inc.

2 Energy Star ® Homes Evaluation “Difference of Differences” Usage/savings results from all Participant and a sample of Non- Participant Title 24 models ESH 02/03 and 04/45 evaluations verified a sample of Participant models Baseline study verified Non-Participant models Matched Non-Participants to Participants by CEC CZ and/or RMST Calculated average compliance margins for participants and non- participants in each CZ (relative to “standard” Package D) Difference between the average compliance margins was the net savings margin (energy saved / baseline energy usage) Multiplied by adjusted tracking baseline usage to get net savings

3 Energy Star ® Homes Evaluation “Difference of Differences” - Issues Need a well-matched Participant and Non-Participant group Separate baseline study made this difficult Different sampling frames RMST w/clustering v. Population Distribution over CEC CZs not comparable RMST zones good for characteristics overview, bad for predicting energy usage/savings Didn’t meter non-participants: assumed metering adjustments affected Parts and Non-Parts equally

4 Savings by Design Evaluation: Self-Reported Model Adjustments Surveyed the builders of all sites in the sample For each site, determined influence of the program measure-by-measure  Dichotomous for yes/no measures  Continuous for continuous measures Adjusted each measure accordingly in each site’s DOE- 2 model Re-ran simulation with net-adjusted measures—savings compared to baseline were net savings

5 Savings by Design Evaluation: Self-Reported Issues Significant “gaming” of system  Learned to ask “how” program affected savings Some buildings are built from common plans  Learned to ask about the program’s impacts on measures in the plans/prototypes instead of in the buildings Potentially more accurate, but more difficult to compute net-to-gross ratio  Without running many different combinations of measures by source, it is difficult to attribute savings beyond a simple net/gross split

6 Utility-Specific Savings from Statewide Programs/Evaluations Statewide Program/ Evaluation Statewide Sample Utility Sample Utility Samples Utility Ratios Statewide Ratio Utility Ratios Utility Tracking Utility Sample Utility Savings