Washington State Weatherization Program Evaluation Julie Palakovich Weatherization Program Manager DOE National Weatherization Conference December 9, 2007

Strategy for Washington’s 2007 Weatherization Program Evaluation: Figure out how to get blood from a turnip in less than six months.

Motivation for Evaluation  Respond to demands from the legislature and public for government accountability.  Last State weatherization evaluation done by ORNL in  State recognizes need for an on-going evaluation program as a performance management tool.  State and local agencies value taking a proactive approach to maintaining and improving high quality program services.

No easy task……  26 Local Weatherization Agencies  39 Counties  63 Utilities  4 Weatherization Programs DOE WX LIHEAP WX Bonneville Power Administration WX Energy Matchmakers Program  Measly data collected at State level

Evaluation Process First Step: SEEK PROFESSIONAL HELP WSU Extension Energy Program Rick Kunkle, Research Engineer Website:

Next Steps……..  Convene Evaluation Working Group  Develop a WX Program Evaluation Plan  Create a mock WX Evaluation Report  Collect and analyze data  Incorporate findings into the Evaluation Report

Washington State Weatherization Program Evaluation Plan Section 1: Evaluation Overview o About the Program o Program Management and Funding o Past Evaluation Results o Purpose o Summary of Evaluation Methodology and Implementation

Evaluation Methodology and Implementation −Program Description and Delivery −Program Benefits (NOT just energy!) −Program Costs and Cost Effectiveness −Program Recommendations −Evaluation Implementation Process

Section 2: Evaluation Details oEvaluation Measures oData collection oAnalysis oReporting oDevelopment and implementation of the evaluation process.

Evaluation Measures 1. Program Description and Delivery oNumber of homes weatherized oMeasures implemented oTypes of agencies delivering WX services oMethod of service delivery oAgency accomplishments oService delivery challenges or problem areas oDemographics oHousehold survey information

Evaluation Measures 2. Program Benefits oHousehold oRatepayer oSocietal oEnergy

Evaluation Measures 3.Program Costs oAdministration oProgram Operation oHealth and Safety oWeatherization-Related Repair oTraining and Technical Assistance

Evaluation Measures 4.Program Cost Effectiveness -Costs and benefits for electrically heated and gas heated weatherized homes -Weatherization benefit cost ratios -Levelized cost

Evaluation Measures 5.Program Recommendations -Recommendations from local agencies for improving the weatherization program -Training priorities identified by local agencies -Overall recommendations for improving the weatherization program

Sources for Data Collection  Local Agencies  Agency Work Plan submitted to CTED  Agency Monitoring Visit Reports  Weatherization Training Survey  Household Surveys  Oak Ridge National Laboratory (Non-Energy Benefits of the WX Program)  Utility Bills and Billing Analysis  Cost-Effectiveness Analysis

Agency Data Collection  About 4000 units weatherized per year  10 of our 26 agencies complete 82% of the units weatherized  Developed Local Agency Data Request spreadsheet and instructions  Sent draft out to the local agencies  Held two conference calls to explain the goal of the request  Collecting data and beginning analysis

Analysis  Utility Bill Analysis  Other Benefits Analysis  Cost-Effectiveness Analysis

Reporting  Executive Summary  Introduction  Program Description and Delivery  Program Benefits  Program Costs  Program Cost-Effectiveness  Summary and Recommendations  Appendix A. Detailed Agency Data

Mock Evaluation Report  Purpose: Use the end result to help better define the evaluation approach.  Questions: oAre the correct evaluation measures being used? oAre the evaluation measures presented in a meaningful way? oHow can the data for each evaluation measure be obtained?

NOTE! Data created for illustrative purposes only!!

Results  Energy Savings in Electrically Heated Homes about 5-12%  Energy Savings in Gas Heated Homes about 10-25%  Cost-Benefit Ratio usually >1  Non-Energy often greater than Energy Benefits

Number of Low-Income Homes Weatherized

Percentage of Homes Receiving Measures 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Attic Insulation Wall Insulation Floor Insulation Windows Envelope Sealing Heating System Ducts Other Measures Repair Safety Percentage of Homes Receiving Measure

Program and Measure Costs

Household Member Demographics

Types of Agencies Delivering Weatherization Services

Size of Agencies by Number of Households Weatherized in 2006

Energy Savings  Electrically Heated Houses  Natural Gas Heated Houses  By Size, Type and Level of Energy Use  By Weatherization Measures

Electricity Savings for Electrically Heated Houses Participant Normalized Savings Control Group Normalized Savings Participant Adjusted Savings Sample Size (N) Electricity Savings (kWh/yr) Electricity Savings (kWh/sqft) Electricity Savings (%) 13%2%11% Electricity Cost Savings ($/yr)

Distribution of Energy Savings for Electrically Heated Houses

Natural Gas Savings for Gas Heated Houses Participant Normalized Savings Control Group Normalized Savings Participant Adjusted Savings Sample Size (N) Natural Gas Savings (kWh/yr) Natural Gas Savings (kWh/sqft) Natural Gas Savings (%) 27%5%21% Natural Gas Cost Savings ($/yr)

Distribution of Energy Savings for Natural Gas Heated Houses

Energy Savings by Level of Energy Use Electrically Heated House Savings ($) Natural Gas Heated House Savings ($) Low Energy Use Medium Energy Use High Energy Use Energy Savings by Size of House Electrically Heated House Savings ($) Natural Gas Heated House Savings ($) Small House13090 Medium House Large House Energy Savings by Type of House Electrically Heated House Savings ($) Natural Gas Heated House Savings ($) Single Family Mobile Home180 Multi-Family150NA

Estimated Energy Cost Savings for Weatherization Measures

Participant Feedback  How Participants Heard About Program  Reason for Participating  Satisfaction with the Program  What Participants Learned  Actions Taken by Participants

How Households heard about the Weatherization Program

Primary Reason Households Participated in the Weatherization Program

Satisfaction of Households with the Weatherization Program

What Households Learned as a Result of Participating in the Weatherization Program

Actions Taken by Households as a Result of Participating in the Weatherization Program

Development and Implementation of Evaluation Process  Outline of potential steps for developing and implementing an on-going Weatherization Program Evaluation  Pilot evaluation proposed for first year using sample of agencies to test the waters  Based on experience, revise evaluation plan before implementing for entire Weatherization Program

For more information: Julie Palakovich Weatherization Program Manager CTED Housing Division PO Box Olympia, WA Phone: