1. Non-Proven Residential Duct Sealing Measures: Research Strategy Josh Rushton and Adam Hadley Research and Evaluation Subcommittee October 7, 2015

2. Overview Today’s focus is Research Strategy for three Planning measures: Prescriptive duct sealing, SF Performance-based duct sealing, SF Performance-based duct sealing, MH Odd man out: Prescriptive duct sealing, MH, already proven based on evaluations from Avista and PSE 2

3. Goals for Today R&E feedback on Research Strategy – Are the research objectives right? – Is outlined research approach reasonable? Realistic? – Alternative approaches – more to say, or less to say? – Does estimated cost range match outlined research? – Is level of detail right for an RTF Research Strategy? Existing research we should consider? – PSE 2011 evaluation, others? Important themes: – Duct sealing is tricky to research (when program designs vary, savings can be a moving target) – What will it take to get research that generalizes to an RTF UES? 3

4. Outline Background and Scope Duct Sealing Research Strategy Existing Research 4

5. Background and Scope 5 - Background

6. RTF Decision: Approval to “Planning” Category, “Active” Status RTF Decision: Approval to “Provisional” Category, “Active” Status. RTF Decision: Approval to “Proven” Category, “Active” Status. 6 October 2015 RTF Meeting Later, or maybe never Later still (or never)

7. What is a Research Strategy? Clarifies knowledge gaps in non-Proven measures Focuses on high-priority research objectives – What does the RTF need for the measure to be proven? – Anything researchers should pay special attention to? Outlines a straw-man approach to data collection and analysis – Demonstrates one feasible research path – Research Sponsors develop final Research Plan Sponsors can work with RTF staff to ensure plan addresses RTF needs RTF reviews final Research Plan later (at Provisional  Planning step) Calls out approaches that probably wouldn’t suffice (optional) Provides a rough cost estimate (based on straw-man approach) Research Strategies are BRIEF: Critical items shouldn’t get lost in a sea of helpful suggestions 7 - Background

8. Duct Sealing Research Strategy 8 – Research Strategy

9. Measure Description Two basic types of specs: Eligibility requirements – Presence of accessible ducts in unconditioned spaces – Electric heat (FAF or HP) – Ducts not previously sealed by program – Pre-existing leakage requirement based on test (performance-based only) Job requirements – Technical specs describing how ducts should be repaired, sealed – Technician certification requirements – Post-case leakage test must demonstrate certain level of improvement (performance-based only) 9 – Research Strategy

10. Research Strategy: Objective Objective: Calculate UES values that reliably estimate savings for duct sealing measures going forward. Savings estimates must reflect the measure specifications that define the RTF duct sealing measures. Prescriptive Duct Sealing (Single-Family). Prescriptive Duct Sealing (Single-Family) – Specs emphasize features that can be verified after duct sealing – UES values shouldn’t reflect program screens based on pre-existing conditions Performance-based Duct Sealing – Prescriptive specifications apply, plus – Requirements related to measured duct leakage Pre-existing ducts must meet certain requirements, Post-case leakage must not exceed certain limits. Measures allow exceptions to leakage standards. Researchers should consider testing whether the exceptions are associated with different savings levels. Researchers don’t have to use current RTF measure specs, but RTF can only get proven UES values for measures with specs/eligibility similar to study sample. 10 - Research Strategy

11. Research Strategy: Outlined Approach Outlined approach is billing analysis – No recommendation on modeling approach (two-stage VBDD analysis versus fixed-effects or other) – Analysis must “account for additional efficiency measures installed during the study period” – “Eligibility requirements and measure specifications reflected in the study must be clearly documented” – “Study should perform delivery verification for a sample of participants to estimate the percent of jobs that comply with specifications” Research product is percent savings estimating – RTF Staff will use this to develop UES values (kWh) adjusted for heating zone, system type, and measure interactions Separate study needed for each of the three measures – RTF recommends at samples of least 750 participants (plus 750 comparison sites, if included) for each study 11 - Research Strategy

12. Research Strategy: Discussion Are the duct sealing measures UES-able? Suppose a smart researcher takes care to address the items mentioned. – Is the likely outcome a Research Plan that will get the measure to (or closer to) a Proven? – Are there other things we should mention now, in the Research Strategy? 12 - Research Strategy

13. Research Strategy: Recommendation Is the outlined research approach reasonable? Does the estimated cost range ($25K - $100K, per study) match the outlined research? Bottom line: Does the subcommittee recommend accepting the Research Strategy for these measures? 13 - Research Strategy

14. 2011 PSE Study 14 - PSE Study

15. Sample size and measures 15 - PSE Study Large sample size Lots of other weatherization measures

16. Study/Program Details Fixed effects billing analysis Included indicator variables for different measures (and interaction terms) Report provides incremental savings due to duct sealing – But we think we can work out savings values that treat measure interactions how we want them Main concerns have to do with recruitment and eligibility (a lot of basement homes) 16 - PSE Study