1. EvergreenEcon.com ESA 2011 Impact Evaluation Research Plan Public Workshop #1 February 20, 2013 Presented By: Steve Grover, President

2. 2 Agenda for Today Introductions / Opening Remarks (15 min) Presentation of Draft Research Plan (30 min) Q&A / Discussion on Research Plan (1 hour) Break for Lunch Discussion on Research Plan (Continue) Wrap Up / Action Items (15 min)

3. Evaluation Team The ESA Impact Evaluation team consists of the following: Evergreen Economics (Prime Contractor) Michaels Energy (Engineering Support) CIC Research (Phone Surveys) Betsy Wilkins (Technical Editor) John Stevenson (Survey Design) 3

4. ESA Impact Evaluation Tasks Task 1: Project Management Task 2: Kickoff Meeting Task 3: Research Plan Task 4: Public Workshop #1 Task 5: Data Collection and Analysis Task 6: Draft Report Task 7: Public Workshop #2 Task 8: Final Report Task 9: Data Documentation 4

5. Task 3: Research Plan The ESA Draft Research Plan available for public comment on the CPUC website www.energydataweb.com. The Research Plan includes the following: Complete discussion of analysis methods and project tasks Discussion of project issues and pros/cons of approach Sampling plan for phone surveys General report outline Timeline for key deliverables Description of project management procedures and communication protocols Comment period ends February 22, 2013. 5

6. Task 5: Data Collection and Analysis The primary analysis method will be a billing regression model: Utilizes a fixed effects model specification (same as prior evaluations) Will use model from the last impact evaluation as a starting point (will allow for comparisons with last 2 evaluations) More resources devoted to exploring other billing regression options, data screening methods, etc. Model will produce impacts by utility, house type, and measure type/group (same as last evaluation) Will explore using hourly (rather than daily) temperature data to calculate CDD and HDD variables. 6

7. Fixed Effects Model (General Form) 7

8. Fixed Effects Model Advantages Primary model advantages : Fixed effects specification contains the desirable statistical properties for producing unbiased estimates Allows for controlling some of the non-program influences that affect savings Provides opportunity to compare current impact estimates with prior ESA evaluations Changes in savings estimates from the last evaluation due to weather and/or household energy use can be tracked. 8

9. Two-Stage Regression Model A second model will be estimated that utilizes a two-stage format: Stage 1: Estimate whole-house savings using a fixed effects regression model. Stage 2: Estimate share of household savings from Stage 1 attributable to individual measures using an ordinary least squares (OLS) regression model. The two-stage model may provide a better opportunity to develop measure- level impacts. 9

10. Two-Stage Regression Model (2 nd Stage) 10

11. Data Screening for Regression Models Some data screening will be needed for any regression model. Screens that we will explore include: Monthly usage less than or equal to zero Very high monthly usage (threshold levels TBD) Customers with less than 12 months of pre-installation billing data Customers with less than 12 months of post-installation billing data (may relax to 10 months) Customers with very large changes in energy use between pre and post periods (thresholds TBD) All screenings and their effect on savings will be clearly documented. 11

12. Task 5: Phone Surveys Phone Surveys Participant phone surveys will be conducted (150 per utility, 600 total) Due to limited sample, surveys will focus on sample of customers with increased energy use after ESA participation. Questions will be designed to collect additional information that might help explain increased consumption: Changes in how equipment is used after ESA participation How equipment is used in extreme weather Condition of existing equipment Demographic information / changes to household 12

13. Task 5: Engineering Review Engineering review Review savings estimates if needed (both ex ante and ex post) Help attribute savings to individual measures if only measure group savings can be estimated from the billing regression Provide insights and analysis from on-sites from previous ESA evaluation (if needed) 13

14. Task 6: Draft Report Draft Report will document all analysis and results Similar to last evaluation report; summary information contained up front with detailed information included as appendices Report outline included in the Research Plan Our technical editor Betsy Wilkins will edit the draft report prior to delivery 14

15. Task 7: Public Workshop #2 Public Workshop covering the Draft Report will include: Overview of analysis methods and results Draft report will be posted on CPUC website prior to Public Workshop Meeting materials (e.g., agenda, presentation) will also be made publicly available prior to the Workshop. Majority of meeting will be spent fielding questions and comments on the Draft Report We will take notes on the Workshop and summarize key comments and action items. 15

16. Task 8: Final Report Final Report Will address questions and comments received during the Public Workshop #2 Also will address comments and questions from the IOUs and Energy Division Will be delivered no later than August 31, 2013 16

17. Project Management / Data Documentation Project Management (Task 1) Weekly conference calls with SDG&E Project Manager Monthly written progress reports Weekly data collection dispositions while surveys are in the field Other ad hoc calls and meetings as needed throughout the project. Data Documentation (Task 9) All data files and programs will be provided to the Energy Division at the end of the project Datasets will include a data dictionary that defines each variable in the dataset A separate document will list all the programs used to compile the raw data, run the billing regression models, and produce the final impact estimates. Documentation will be comprehensive so that an independent third party can replicate the results. 17

18. Project Timeline (Highlights) Key Dates: Draft Research Plan Posted: February 14 Public Workshop #1: February 20 Research Plan Comment Period Ends: February 22 Final Research Plan: March 1 (estimated) Participation and billing data compiled from IOUs: March 1 Data Collection and Analysis Complete: May 31, 2013 Draft Report: July 7, 2013 Public Workshop #2 on Draft Report: TBD Final Report: August 31, 2013 18