1. 2013-2015 Regional Energy Networks Impact Evaluation Research Plan July 20, 2016 1

2. REN Program Impact Evaluation (Multifamily) –Program Overviews –Evaluation Overview, Activities, Calculations REN Program Impact Evaluation (Single Family) –Program Overviews –Evaluation Overview, Methods and Samples Schedule and Next Steps Agenda 2

3. How Did We Get Here? 3 7/15/2016: Draft plan released 7/20/2016: Research Plan Webinar 7/29/2016: Comment Period Closes 8/22/2016: Final Research Plan Released

4. Program Overview: REN Multifamily Whole Building Program 4

5. Multifamily Whole Building Program (MF-WB) –Administered by BayREN and SoCalREN –Customized project scopes Savings based on building simulations –Technical and financial assistance Project incentives range: $3,750 - $594,000 Program Overview – MF-WB 5

6. 6 Program ElementSoCalRENBayREN Service Territory Joint SCE/SCG territory, minus municipal service territories (e.g. LADWP) 9-County Bay Area Eligibility Requirements 3+ units; SCE & SCG service; 3+ measures; work with Participating Rater 5+ units; 9-county Bay Area; PG&E gas and/or electric; 2+ measures Rater Delivery Model Open RaterTech Assist. direct delivery Audit Requirement ASHRAE Level 2Clipboard audit CAS Testing MF HERCC ProtocolsMF HERCC protocols Energy Modeling Software EnergyProEnergyPro Lite Assessment Incentive # UnitsIncentive Free site visit and technical assistance up to $5000 value 1-49$5,000 50-100$10,000 100+ $20/Unit Increment Improvement Incentive Improvement$/UnitImprovement$/Unit 10%$55010%$750 15%$625 20%$800 25%$1,000 _> 30%$1,200 Verification Participating RaterTech. Assist. (AEA)

7. Incented ~248 properties in 2013-2015 –>15,000 dwelling units BayREN claimed most projects (94%) and electricity savings (80%) Program Overview – MF-WB 7

8. Evaluation Overview: REN Multifamily Whole Building Program 8

9. Evaluation Overview – MF-WB 9 Evaluation Research Questions 1)What are the gross energy and demand savings (therms, kWh, kW) achieved by the BayREN and SoCalREN programs? 2)What are the net energy and demand savings achieved by the programs? 3)How can the RENs improve their ex ante savings claims so that they align with ex post values? 4)How do the REN savings and costs compare to those claimed from the IOU MF-WB programs?

10. Evaluation Overview – MF-WB 10 Evaluation Activities Telephone Surveys & Onsite Recruitment Onsite Visits Consumption Analysis Calibrated Simulation Models Savings Calculations

11. Evaluation Overview – Data Sources 11 REN provided project details CPUC claim savings database Consumption (billing) data REN EnergyPro Models Participant Survey (Property Managers/Owners) –2013-2014 impact assessment completes –2015 completes

12. Evaluation Overview – MF-WB 12

13. Evaluation Activities: Telephone Survey and Onsite Recruitment 13

14. Evaluation Activities– Telephone Survey 14 Goals: Recruit for onsite visits Supplement 2013-2014 survey results –NFR Estimation –Baseline Assessment Sample: All 2013-2015 participants eligible for onsite recruitment –Survey conducted only with those not contacted in 2013-2014 effort

15. Evaluation Activities– Telephone Survey 15 Survey Topics: Confirmation/verification of installed measures Anticipated actions in absence of program intervention Importance of program education and incentives on the decision to install high efficiency equipment Working status and estimated age of replaced units Timing for building maintenance/upgrades Survey Instrument: Rely on 2013-2014 instrument –Comparability, consistency, holistic 2013-2015 results, vetted batteries and analysis

16. Evaluation Activities: Onsite Visits 16

17. Evaluation Activities– Onsite Visits 17 Goals: 1)Verify key measure & building characteristics –Measures and characteristics that contribute most to project savings –Use in simulation models 2)Collect meter numbers –Link to IOU consumption information for project Sample: All 2013-2015 participants eligible Target 22 completed site visits

18. Evaluation Activities: Consumption Analysis 18

19. Evaluation Activities– Consumption 19 Goals: 1)Comprehensive pre-program energy use at participant property. –Use in simulation models 2)Determine if meter numbers fully capture / link to IOU consumption information Sample: Completed onsite visits where meter numbers were obtained

20. Evaluation Activities– Consumption 20 Process: 1)Link meter numbers to IOU consumption information 2)Assess completeness and reasonableness of data

21. Evaluation Activities: Calibrated Simulation Models 21

22. Evaluation Activities– Models 22 Goal: Ex post gross savings values –Based on administrator project models –Using EnergyPro –Calibrated to pre-program consumption –Incorporate key measure and building characteristics found onsite –Adjusted for early replacement and replace on burnout scenarios Sample: Completed site visit participants –Meter numbers were obtained –Reasonable consumption information

23. Evaluation Activities: Savings Calculations 23

24. Evaluation Review 24

25. Evaluation Activities– Calculations 25

26. Program Overview: REN Single Family Home Upgrade Program 26

27. Overview: Single Family –2 Regional Energy Networks (REN) and 4 Investor Owned Utilities (IOU) –Home Upgrade – deemed savings, deemed net to gross ratio (RENs) –Advanced Home Upgrade – modelled savings, estimated net to gross ratio (IOUs) 27

28. Overview: Prior Study Recap Prior impact evaluation –Focused Impact Evaluation (PY 2013-2014) estimated gross savings only for Home Upgrade –Whole House Retrofit (PY 2010-2012) estimated savings gross and net savings for Advanced Home Upgrade Outcome –Ex post saving estimates were less than ex ante but had wide ranges. –Issues with data quality and quantity limited the application of findings 28 This research will estimate gross and net savings, NTG ratio of free riders, realization rate for Home Upgrade and Advanced Home Upgrade using projects from 2013, 2014 and 2015

29. Methods: Energy Savings Billing analysis –Billing analysis using meter data and pooled model Comparison group consists of future program participants Variable duration for blackout period (min 30 days) Consideration of interim activities and behavioral changes Hourly data for kW Daily data for kWh Monthly data for therms –Expect population to be around 18,000 homes statewide 29 Approach is similar to last impact evaluation to facilitate comparisons. Refinements will be made from lessons learned and to be as consistent as possible with CalTRACK effort

30. Methods: Savings Drivers NTG and Exploratory Research –To date, the less than expected savings outcomes from HUP have not been fully explained Combine data and findings from past and current studies Conduct participant surveys –Topic examples: Contractor effect / program messaging? Geographic or demographic patterns? Proportions and types of measures installed? Primary motivations or significant barriers to adoption of one path or the other? Other actions taken by participants? 30 Primary data collection and mine existing data to uncover new insights

31. Single Family Schedule Data Collection –Start now (July 2016) Project data collection Prior studies Survey informed by billing data and prior studies Interval billing data collection (Sep-Oct 2016) Weather data Analysis –Over the summer (Q3 and Q4 2016 ) Reporting Q1 2017 31 Interim results can be released as available

32. Schedule and Next Steps 32 Milestone Schedule Draft Research Plan – Comments Due7/29/2016 Final Research Plan made available PDA/Basecamp 8/22/2016 Draft Data Collection Instruments9/15/2016 Telephone and Onsite Data Collection10/1 - 12/31/2016 Consumption Analysis1/1 - 2/28/2017 Simulation Modeling2/1 - 3/31/17 Gross Impact Analysis4/1 - 5/1/17 Net Impact Analysis4/1 - 5/1/17 Draft Report made available to public5/12/2017 Webinar - Draft Report5/17/2017 Draft Report - Comment Period Closes6/2/2017 Final Report made available7/1/2017

33. Contact Information 33 DNV GL Contact : Jon Vencil, jon.vencil@dnvgl.com jon.vencil@dnvgl.com (619) 929-3232 Apex Contact: Katie Parkinson, katiep@apexanalyticsllc.com katiep@apexanalyticsllc.com (303) 590-9888 CPUC Contact: Jeremy Battis, jeremy.battis@cpuc.ca.gov (415) 703-3041