Tool Lending Library Program evaluation Impact Evaluation December 12th, 2017

Agenda Background information Methodology Project and Tool Borrower Characterization Savings Potential Claiming Savings in the Future Findings Recommendations Tool Lending Library Impact Evaluation

Background Four PA Supported Tool Lending Libraries (TLLs) Terminology PG&E’s Pacific Energy Center (PEC) PG&E’s Energy Training Center in Stockton SCE’s Agricultural Technology Application Center SDG&E’s San Diego Energy Innovation Center Terminology Application Notes Borrower Loan Project Project Type Site Tool Tool use Tool Lending Library Impact Evaluation

Research Objectives Objective 1: Characterize projects initiated through a TLL loan across all four PA’s TLLs in 2016 Objective 2: Quantify the savings potential for two projects supported by the PEC’s TLL and assess the tool’s contribution to those savings Objective 3: Explore if and how the PEC TLL can claim savings in the future Tool Lending Library Impact Evaluation

Methods Interviews with PEC Staff (n=3) Secondary Data Review Review of Program Tracking Data CPUC EM&V Documents Past evaluation reports Participant Web Survey (n=107) Census attempt of 2016 borrowers Case Studies (n=2) Engineering analysis of two projects supported though PEC tool loans Tool Lending Library Impact Evaluation

Participant Survey Primary Goals Characterize projects and borrowers Understand how borrowers used tools Education and training Energy Center Completed Surveys Response Rate PG&E Pacific Energy Center (PEC) 67 30% SDG&E San Diego Energy Innovation Center 23 20% SCE Agricultural Technology Application Center 15 25% PG&E Energy Training Center 2 18% Total 107 27% Tool Lending Library Impact Evaluation

Case Studies—Selection Criteria Used participant survey to recruit projects for deeper analysis Employed the following criteria for selection: Data availability—Ensured borrowers had pre- and post- implementation data Most recent projects—Borrowers may have used tools to support multiple projects Projects with direct savings—Asked borrowers through the survey to specific if project’s had measureable savings (or renewable generation) Tool Lending Library Impact Evaluation

Project Characterization Broad project categories Baseline Assessment Pre-Installation Measurement Health, Safety, and Maintenance Post-Installation Measurement Training, Education, and Outreach TLL projects with savings that received a utility incentive (45%) Tool Lending Library Impact Evaluation

Borrower Characterization Half of all borrowers said their 2016 loan was not their first TLL experience How borrowers learned about TLL Energy center course (21%) Word of mouth (11%) Visiting their local energy center (7%) Energy center website (5%) Utility incentive program (4%) Other (4%) Tool Lending Library Impact Evaluation

Borrower Characterization Majority of borrowers are employed in the energy industry Nearly one quarter (23%) of borrowers supported projects in borrower’s homes Tool Use Percent of Borrowers For my work 63% For my home 23% For a course I taught 2% For a course I took 5% Tool testing or evaluation 3% Other Tool Lending Library Impact Evaluation

Borrower Characterization—Business Type Nearly half (49%) identified as either ESCO or Contractors Tool Lending Library Impact Evaluation

Borrower Knowledge Tool Lending Library Impact Evaluation

Borrower Education and Training Application Notes Received application notes (83%) Used application notes (82%) Information in application notes was new (53%) Application notes were “Very Useful” (85%) Energy Center Course or Ad-hoc Training Received additional training (35%) Information in training was new (81%) Training was “Very Useful” (89%) Tool Lending Library Impact Evaluation

Borrower Behavior Change Identifying energy-saving opportunities for the buildings you serve 71% took this action since their 2016 tool loan 78% said the TLL was “Very Influential” Other energy saving actions Energy-Saving Action Took Action Very Influential Turned off lights more frequently at home 55% 61% Purchased energy-saving equipment for your home 50% 62% Adjusted your HAVC system to save energy at home 49% Turned off lights more frequently at work 41% 73% Tool Lending Library Impact Evaluation

How Borrowers Use Tools Tools used largely for benchmarking and measurement * Percentages will add up to more than 100% as loans may have been used for more than one purpose listed Tool Lending Library Impact Evaluation

Savings Potential—Case Studies #1 Central Plant EE #2 Residential Solar PV Multi-measure C&I controls project Borrower works for an energy services company Tools borrowed: Hobo data logger Ultrasonic flow meter Incentive  Yes Homeowner trying to determine best location to install PV system Borrower is a retired PG&E employee Tools borrowed: Solmetric SunEye Incentive  No Tool Lending Library Impact Evaluation

Savings Potential—Case Studies TLL had very high level of influence on both projects Neutral to higher likelihood projects would have been implemented and had the same savings without the TLL Survey Question Case #1 Case #2 TLL Mean Rating Influence on decision to implement the project 9 8 8.4 Likelihood that the project would have been implemented without the TLL 5 4 3.5 Likelihood that the project would have generated the same capacity without the TLL 7 3.8 Tool Lending Library Impact Evaluation

Savings Potential Majority of loans supported projects that did not produce measureable savings (56%) Of projects that did lead to savings (36%), majority leveraged tool loan prior to implementation 76% of projects with savings, 27% overall Revised Project Categorization Count Percent Projects with Savings 38 36% Tool Used Before Implementation 29 27% Tool Used After Implementation 9 8% Projects with No Savings 55 56% Unsure if Project lead to Savings Total 107 Tool Lending Library Impact Evaluation

Savings Potential—TLL Influence The majority (88%) of borrowers say the TLL was “Very Influential” on their decision to move their project forward Fifty-five percent said their project was not likely to move forward without the TLL Note: Savings are based on survey responses and are self-reported Tool Lending Library Impact Evaluation

Claiming Savings in the Future Evaluation Protocols M&V Protocols Impact Evaluation Protocols Net Impact Evaluation Methodologies Indirect Impact Evaluation Protocol Barriers Transaction costs Potential for high levels of Free-ridership Splitting savings between programs How to handle projects without savings Tool Lending Library Impact Evaluation

Indirect Impact Evaluation Protocol Source: California Energy Efficiency Evaluation Protocols, State of California Public Utilities Commission, April 2006 Tool Lending Library Impact Evaluation

Claiming Savings in the Future Based on CA Evaluator’s Protocols Project-level tracking Pre- and post-installation energy usage data; Pre- and post-specifications (i.e. product cut sheets); Detailed scope of work; Demographic characteristics; and Other administrative documentation as needed. Tool Lending Library Impact Evaluation

Key Findings TLLs frequently serve repeat borrowers Tool loans support a range of project types Energy measurement activities represent largest share of tool loans The TLLs play some role in moving projects forward Loans likely contribute to savings claimed through other programs Education and training that TLLs provide, specifically those with dedicated staff, is of high value to borrowers Indirect impact evaluation protocol provides some guidance on how the TLLs may claim savings in the future Tool Lending Library Impact Evaluation

Recommendations The PEC should consider tracking additional categorical details at the project-level If claiming savings, decision makers should consider: How to track data projects with indirect savings potential (44%) Added transaction costs Process for multiple PA programs to claim savings TLL decision makers and staff should build on education and support offered to tool borrowers Tool Lending Library Impact Evaluation

Thank You! Ellen Steiner, Ph.D. Vice President eteiner@opiniondynamics.com