1. Measured Energy Savings Program Results ACC 2003 - Kansas City David Carroll, APPRISE Incorporated

2. Program Data Tracking Systems How much information is needed to furnish measure-level savings?

3. Required Information Measure savings estimates require data on measures for each home ….More detailed data give better measure savings estimates.…Tracking systems take time, cost money, and overwhelm some contractors ….Tracking systems can be a valuable tool for program management

4. Savings Estimation Model Model: Savings is a function of expected savings from measures or measure groups Observations: Treated units Dependent Variable: Consumption savings Independent Variables: Expected savings for each measure or measure group Y = a + bx 1 + cx 2 + … (where x 1 is insulation, x 2 is air sealing, etc…)

5. Savings Disaggregation Model For each unit… Projected Savings = a + bx 1 + cx 2 + … Ratio = Actual Savings / Projected Savings If x 1 is the insulation term, bx 1 is the projected savings from insulation, and Ratio* bx 1 is the normalized savings attributed to insulation

6. 6 Option #1- Installed Measures Data Inputs Data on: Insulation – Y/N Air Sealing – Y/N Thermostat – Y/N Tank Wrap – Y/N Model Outputs Savings for: Insulation Air Sealing Thermostat Tank Wrap

7. Requirements/Assumptions/ Caveats Requirements: Units vary on the types of measured installed Assumptions: Standard regression assumptions / collinearity and outliers cause problems / issue with intercept term Caveats: Lots of variance in expected and actual savings for measures across homes leads to high variance in model statistics Expectations: If all goes right, you may get a decent rough estimate of savings by measure

8. 8 Option #2- $ By Measure Group Data Inputs Data on: $ for WX $ for Low Cost $ for Appliances $ for Education Model Outputs Savings per: $ of WX $ for LC Measures $ for Appliances $ for Education

9. Option #3 - $ By Measure Data Inputs Data on: $ for Insulation $ for Air Sealing $ for Thermostats $ for Water Measures Etc… Outputs Savings per: $ for Insulation $ for Air Sealing $ for Thermostat $ for Water Measures Etc…

10. Requirements/Assumptions/ Caveats Requirements: Spending differs across treated homes by measure or measure group Assumptions: Standard regression assumptions / collinearity and outliers cause problems Caveats: Measures are “lumpy” and impacts may not be linear. Be careful with savings per $ parameters Expectations: Lower variance than simplest model.

11. Option #4 – Measure Quantities Data Inputs Data on: Amount/location of insulation Hours/location of air sealing Thermostat and setback protocol # and type of water measures Etc… Outputs Actual savings per projected savings for: Insulation Air sealing Thermostats Water Measures Etc…

12. Improvements in Model #3 Variance: Measure based system may improve model specification and reduce collinearity problems Model Validity: Unsupportable results may highlight measurement or statistical issues Usefulness: Savings attributed to individual measures may better focus quality control

13. Further Tracking Enhancements Measures and Site Conditions: Amount and location of insulation / pre and post coverage and R-value Measures / Site Conditions / Inspections: Include outcomes of onsite inspections regarding installation quality

14. Bottom Line Statistical models measure association Differential treatment and/or investment among units can be used to statistically estimate differential impacts of measures Using measure-based engineering estimates can reduce variance between actual and predicted savings and improve reliability of findings Unobservable factors will always result in some uncertainty in estimates Models can lead to program improvements

15. Energy Education Untapped Resource or Unrealistic Expectations?

16. Savings from Education ProgramYearGas SavingsElectric Savings NMPC Power Partnerships 199210%3%* PSE&G E-Team Partners 1997No savings statistically attributed to customer behaviors NMPC LICAP1998--7%* Ohio EPP2002NAAction plans limited NJ Comfort Partners 2002Low level of action plans but high level of actions

17. Education Protocols ProgramProcedures NMPC Power Partnerships Three-visit protocol with both energy savings and budget counseling goals PSE&G E-Team Partners Energy education was part of audit. Procedures include measures visit and insulation visit. NMPC LICAPOptions: Energy education workshop / Energy education video / Education with energy services Ohio EPPElectric Baseload - Bill reconciliation process identifies biggest users and biggest savings actions NJ Comfort Partners Energy education staff use education tools to review bills, inform customers of costs, and identify actions

18. Characterizing Education ProgramProcedures NMPC Power Partnerships Education supported the weatherization process, leading to better and more persistent savings PSE&G E-Team Partners Education supported the weatherization process by informing customer of measures in home NMPC LICAPWorkshop is focused on independent customer behavioral change Ohio EPPReconciliation tool is designed to identify best energy saving actions and communicate to customer NJ Comfort Partners Energy education is expected to be integral to audit, measures installation, and follow-up

19. Purposes of Education Awareness of Measures – Understand what was done and accept the outcome Support of Measures – Understand how to keep the measure working Supplemental Behavioral Changes – Make some other change in behavior that reduces energy consumption

20. Examples of Effective Education Awareness of Measures – Retain thermostat setback and hot water turndown, Replace broken CFLs with CFLs Support of Measures – Replace furnace filters, fix faucet leaks, clean lint filter Supplemental Behavioral Changes – Wash clothes in cold water, turn off lights, use energy saver cycle on dishwasher

21. Understanding the Results ProgramProcedures NMPC Power Partnerships In-depth discussion lead to better weatherization, acceptance of measures, and support of measures PSE&G E-Team Partners Auditor focus on client communication lead to positive understanding of measures NMPC LICAPWorkshops furnish motivated clients with tools, may not be able to consistently target highest savers Ohio EPPBill reconciliation process is challenging, auditors don’t believe PIPP clients have motivation NJ Comfort Partners Ongoing commitment to and payment for education leads to constant improvement in attitudes and effectiveness

22. Bottom Line Potential: Site conditions under client’s control represent major saving opportunities Motivation: Clients have the motivation to make changes Challenge: Find the best energy actions and to communicate the action to the client Effectiveness: Education is challenging to integrate into large scale programs

23. Affordability Who gets the benefits?

24. Affordability Logic Arrearage customers fail to pay for 10% to 25% of energy usage. Weatherization and baseload programs reduce energy usage by 10% to 25% Therefore: Usage reduction programs can make energy affordable, can resolve payment problems, and save collections costs.

25. Affordability Barrier 1996 Study of NMPC LIHEAP Customers Half of LIHEAP customers had 60-day arrears Many 60-day arrears customers had high energy bills BUT – 75% of arrears customers also had problems with other bills

26. A Design That Worked Negotiate payments that equaled or exceeded payments made last year. Offer arrearage credits for making negotiated payments Furnish targeted energy services Follow-up for missed payments OUTCOME – On average, customers increased payment coverage from 75% to 90%.

27. A Design That Didn’t Work Furnish energy services Assign budget bill that anticipates 10% usage reduction and LIHEAP grant Offer arrearage credits for bill payment Don’t follow-up missed payments OUTCOME – Customers reduced usage and payments, coverage remained at 90%

28. Bottom Line Logic tells us that usage reduction makes energy more affordable Research shows us that arrearage customers have multiple financial problems Program design affects the extent to which the program results in better payments Good program designs can yield collections-related benefits

29. Health and Safety You can’t take credit for it if you don’t measure it.

30. Health and Safety Problems Existing – Household in immediate danger from Gas Leak, Ambient CO, or other active problem Potential – System is potentially dangerous because of high flue CO or back drafting of flue gases; or other situations where the potential for fire or health risk exists Behavioral – Household regularly engages in energy behaviors that present the potential for fire of health risks

31. E-Team Measurements - 1999 ProblemRateComments High Ambient CO1 in 35018 ppm in basement High Flue CO3%-5%>200 ppm in flue Draft Problems5%-7%1%-2% with spillage Gas Leaks6%-10%Found at both first and second visit Hot Water Temps33%>140 degrees

32. Power Partnerships - 1992 ProblemRateComments Use stove or oven for heat 40%Reduced to 15% by program Health problems because home too cold 36%Reduced to 15% by program Health problems because of air quality 30%Reduced to 10% by program Home is too drafty80%Reduced to 16% by program Home is too cold66%Reduced to 25% by program

33. Measurement E-Team: H&S measurements recorded in database. Detailed analysis of data collection forms for problem homes. Supplemental sample of forms to check for data not entered into database. Power Partnerships: Random assignment to test and control groups. Post treatment survey with clients.

34. Bottom Line Health and safety opportunities exist The incidence can and should be measured Low rate of serious problems Moderate rate of potential problems High rate of behavioral problems Challenging to quantify value and get funding support