Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results Prepared by: Rick Kunkle July 2013

Purpose of Evaluation Identify and document Weatherization Program outcomes, benefits and costs to: Assure prudent use of funds (accountability) Improve the quality and effectiveness of program services Assess progress toward Weatherization Program outcomes as measured by key performance measures

Weatherization Program Overview Serves low income families by installing energy efficiency measures and making health and safety improvements and necessary repairs Services provided in Washington since 1977 Managed by the Housing Improvement and Preservation Unit in the Department of Commerce Commerce contracts with 28 local agencies to deliver services Funded by a mix of state and local sources The American Recovery and Reinvestment Act (ARRA) significantly increased funding, primarily in 2010 and 2011

Previous Evaluation Findings Oak Ridge National Laboratory completed an impact evaluation for Washington in 2001 showing energy savings compared favorably with other states WSU Energy Program completed Washington evaluations for 2006 and FY2010 Benefits exceeded costs Production increased significantly in FY2010 from ARRA funding Recommendations for improving data collection, estimates of benefits, and cost tracking and allocation Commerce has made investments and progress in addressing evaluation recommendations

Evaluation Approach The evaluation covers calendar year 2011 Project Data: from the Weatherization information Data System (WIDS) for projects with final inspections in 2011 Program Data: from Commerce including expenditures, agency work plans, and historical data

Bottom Line Results for 2011 Expenditures were $48 million. This is more than twice annual expenditures. Over half was from ARRA Production was 7,451 units. This is a little less than the peak in 2010, but still more than twice pre-ARRA production. Two-thirds of production was multi-family units Over 12,000 people lived in housing that was weatherized On average, 9 improvement measures were installed in each weatherized unit Estimated energy savings is $1.17 million/year. On average this is $157/unit. Other benefits are estimated to be $198/unit The Program benefit-cost ratio is estimated to be 1.12, meaning that benefits slightly exceed costs

What is in this Presentation Program Delivery Program Benefits Program Costs Program Cost-Effectiveness Summary and Recommendations

Key Findings – Program Delivery

Historical Production Trends

Monthly Production Trends

Production by Agency in 2011

Agency Service Delivery Type of Agency Delivery Method # of agencies% of FY10 production

Funding Sources

Number of Funding Sources

Project Delivery Time

Average Days to Completion by Agency

Heating Fuel

Weatherization Measures by Unit

Weatherization Measure Categories

Top Weatherization Measures

Top Weatherization Measures by Building Type

Tier 2 Weatherization Measures

Household Demographics

Household Poverty Level

Accomplishments, Challenges, Recommendations Each year, local agencies produce a work plan that summarizes their processes, procedures, accomplishments, and challenges Accomplishments: Energy savings, comfort, health and safety and housing repair benefits of their weatherization work Partnerships, cooperation and referrals Challenges Decreasing funding along with increasing costs from wage and other requirements (reporting, monitoring, training, etc.) Recommendations Clearer communication and more transparent decision- making about funding availability and allocations Streamlining program requirements

Quality Assurance Inspections S pecifications Correction Factor

Key Findings – Program Benefits

Energy Savings Analysis in WIDS Deemed savings estimates: Developed savings coefficients by measure, building type, fuel source, and heating zone Used savings coefficients with measure data in WIDs in savings calculations for each measure Accounted for interactions between measures that save heating energy Energy savings are calculated for each weatherization project with caps and checks Energy cost savings based on average Washington energy costs by fuel source Energy benefits calculations use weighted average measure life – 34 years

Total Energy Savings Fuel SourceBuilding Type

Total Energy Savings by Measure CategoryMeasure

Energy Savings per Unit by Agency

Energy Cost Savings Fuel SourceBuilding Type

Energy Cost Savings per Unit

Other Benefits Calculations This is the same approach used for the FY2010 Evaluation Utility benefits > estimated from best practices review Reduce delinquent bills and bad debt write-off Service shut offs and reconnects Participant benefits > estimated from best practices review Water/wastewater savings Increased property value Fewer moves Health, safety and comfort benefits Societal benefits Reduce greenhouse gas and pollutant emissions > directly estimated Improvements to the economy > estimated from evaluation of Weatherization Program for Pacific Power

Program Benefits $/Household/YrMidPercent of Total Utility 164% Participant 10329% Societal - Economic 6017% Societal - Emissions 195% Other Total 19856% Energy 15744% Total %

Key Findings – Program Costs

Program Expenditures by Fund Source

Program Expenditures by Category

Direct Project Cost by Fund Source

Direct Project Cost by Measure Category

Direct Average Unit Cost by Agency

Distribution of Direct Unit Cost

Cumulative Direct Unit Cost Curve

High Cost Projects

Average Direct Unit Cost

Total Direct Cost Expenditures

Total Unit Cost Estimate

Why are per unit costs higher in 2011 compared to FY2010? Many of the reasons could be associated with ARRA More comprehensive weatherization Wage requirements Reporting requirements Quality assurance requirements Others?

Key Findings – Program Cost Effectiveness

Per Unit Cost Divided by Energy Savings

Simple Energy Payback

Cumulative Direct Unit Cost and Energy Savings

Cost-Effectiveness Analysis Approach Cost-Effectiveness = total benefits per unit/total costs per unit Program benefits occur over time and are converted to a present value 30 year average measure life 2.3% discount factor (OMB) 1% fuel escalation factor A high, mid and low scenario are considered to account for uncertainty in the benefits estimates Alternate approaches with different costs and benefits are also considered

Cost-Effectiveness Analysis Results ($/unit costs) Present ValueMidLowHigh Emissions Benefit$421$369- Economic Benefit$1,313$685$1,967 Utility Benefit$338$76$674 Participant Benefit$2,233$920$4,588 Total Non-Energy$4,305$2,050$7,229 Energy Benefit$3,934$2.992$4,571 Total Benefit$8,239$5,042$11,800 Total Cost$7,362 Benefit-Cost Ratio

Cost-Effectiveness Alternate Approaches ($/unit costs) Present ValueEnergy OnlyPlus No Admin or T&TA $ Direct Cost Only Plus No H&S and WRR Emissions Benefit0000 Economic Benefit0000 Utility Benefit0000 Participant Benefit0000 Total Non-Energy0000 Energy Benefit$3,934 Total Benefit$3,934 Total Cost$7,362$6,414$4,780$3,724 Benefit-Cost Ratio

Summary and Recommendations

Some Interesting Results for 2011 Higher production during ARRA is due to large multi- family units Top 9 producers account for 80% of production City and county government agencies account for over half the production, but less than half the agencies Compared to FY2010 there is evidence of more comprehensive weatherization Ceiling, wall, and floor insulation and air sealing account for 55% of the estimated energy savings Large multi-family units account for 60% of the units, but only a quarter of the energy savings and a quarter of total direct costs The 10% of highest cost units account for 32% of total direct costs and 27% of total energy savings

Conclusions to Think About The total program unit cost estimate increased in 2011 compared to FY2010 Large multi-family units contribute to higher production, more people served, and lower costs, but they also have fewer measures installed and produce less energy savings High cost units contribute significantly to total Program costs, but they also have more measures installed and greater energy benefits Energy cost benefits are less than other benefits While high production agencies had lower unit costs, they did not have lower unit costs/energy savings

Recommendations We need to develop better ways to track Program expenditures for Commerce The Program unit cost estimate includes an estimate of local Program operations costs that did not go directly to unit weatherization. We need to check the accuracy of this estimate and identify what these cost are for and how to track them Identify ways to reduce the trend towards higher costs, which reduces Program cost-effectiveness if benefits do not increase How do we measure success? Identify key Program metrics and use WIDS data or other sources to document Program success Consider looking at the benefits from high cost units

Next Steps Review and discuss the evaluation results Make needed revisions and corrections to the evaluation Complete an executive summary for the evaluation (text document) Share the evaluation with the weatherization network and stakeholders Apply the experience from the 2011 evaluation to plan for and conduct the 2012 evaluation