1. Research Strategy: New-Construction Manufactured Home Measures Research and Evaluation Subcommittee Josh Rushton November 19, 2015

2. Overview Today’s focus is the proposed Research Strategy for two new-construction MH measures: ENERGY STAR (NEEM 1.1) EcoRated Homes Seeking subcommittee feedback on outlined research: Is the approach reasonable? – Likely to succeed as advertised? – Does estimated cost range match outlined research? – Recommended modifications? Is there a better path? 2

3. Measure Background HUD is current minimum federal standard. Some homes still being built to this spec – HUD update currently in the works, likely to take effect within a few years – New requirements expected to come close to current NEEM specs (NEEM 1.1) HUD+ is estimated average spec for non-NEEM/EcoRated home since about 2010 NEEM 1.0 was Super Good Cents after that program ended; NEEM 1.0 lasted until 2004 NEEM 1.1 is NEEM since 2004 (current at time of this presentation) – NEEM 1.1 specs cover shell, duct sealing and ventilation, plus a tiny DHW thing (0.93) – Big change from NEEM 1.0 to NEEM 1.1 was locking down window efficiency at U-0.35 (before, efficient windows traded off against other shell components) – NEEM 1.1 meets current ENERGY STAR requirements, but the reverse does not always hold. EcoRated is NEEM 1.1, plus window U-0.32, 80% efficient lighting, ENERGY STAR fridge and dishwasher, and low-flow showerheads and faucets HPMH is a very efficient spec that has been studied in small demonstration projects but currently has very little traction in the region NEEM 2.0 doesn’t yet exist. It is currently a placeholder name for the program that will replace NEEM 1.1 after the coming HUD update takes effect 3 Efficient Baseline

4. Why Planning? Construction specs pretty well-understood, but engineering/simulation estimates not enough for Proven savings values Need data that provides direct insight into difference between energy consumption in efficient new MHs and standard new MHs 4

5. Research Strategy 5 – Research Strategy

6. Research Objectives Savings estimates should be closely tied to observed differences in energy consumption between samples of baseline and efficient-case homes – Engineering models okay for making adjustments but final savings estimates should be “largely driven” by observed kWh differences For a sample of efficient-case homes and a comparable sample of baseline homes, need to collect – energy consumption data – home/site characteristics needed to adjust or filter the samples so the observed differences in energy consumption will be clearly meaningful Improving understanding of as-built shell components in baseline or efficient homes is not an explicit objective of this research 6 – Research Strategy Red print for items added after draft went out for review

7. Straw-man Approach: Sample Size To estimate savings ±750 kWh/year (with 90% confidence), analysis sample target is 154 efficient homes and 154 baseline homes Previous draft indicated 85 of each to achieve ±1000 kWh/year Change motivated by reassessing a priori savings estimate Precision level depends on magnitude of savings (more on this later) Limiting sample to homes without natural gas or heat-pumps may simplify the analysis Sample attrition may be caused by data collection problems and/or data filters used in the analysis 7 – Research Strategy

8. Straw-man Approach: Sample selection Qualitatively, researchers should make reasonable efforts to avoid major discrepancies between baseline and efficient-case samples Problem if efficient sample mostly sited on private lots and baseline sample mostly sited in parks With broad sample frame, random or quasi- random sampling can help reduce discrepancies Samples do not need to be rigorously representative with respect to geography – for instance, sites may be chosen from a limited number of parks or within a single climate zone 8 – Research Strategy

9. Straw-man Approach: Data Collection Required data: Billing data (at least 12 consecutive months), Location (county or zip code and whether site is in a park or a private lot), Construction standard (NEEM, EcoRated, or Other), Building characteristics (as-built component U- factors and square footage), Heating system type and off-grid heat (presence or absence at minimum), – Probably need customer contact (phone interview) or visual site inspections for these 9 – Research Strategy

10. Straw-man Approach: Analysis (1) Billing analysis – Perform site-level billing analysis to estimate annual heating energy for each home in the sample – Calculate billing data’s average difference between efficient and baseline homes: bill.heat.delta = bill.heat.base – bill.heat.efficient Engineering analysis: – Run SEEM for each house using generic shell inputs based on building standard and site-specific inputs for climate, component U- factors, and square footage (may need to adjust internal gains for EcoRated homes) – Calculate SEEM’s average difference between efficient and baseline homes: seem.heat.delta = seem.heat.base – seem.heat.efficient Realization rate: – Ratio of the differences is the realization rate used in final savings estimates: RR = bill.heat.delta / seem.heat.delta 10 – Research Strategy

11. Straw-man Approach: Analysis (2) Final heating energy savings by zone calculated in two steps: – Run SEEM models for baseline and efficient cases; difference is ex ante heating energy savings – Final heating energy savings is ex ante times RR Other savings components (as applicable) estimated separately – Lighting and appliance savings (EcoRated) based on RTF workbooks for lighting and affected appliances (with appropriate HVAC interaction factors 11 – Research Strategy

12. Straw-man Approach: Sample size Sample size calculated in terms of absolute error instead of relative precision because we don’t know savings If savings is 2300 kWh, then ±750 kWh is 33% precision Sample size proportional to inverse of squared error – For ±750 kWh, n = 154 – For ±1000 kWh, n = 154*(750/1000)^2 = 86 – For ±500 kWh, n = 154*(750/500)^2 = 346 RTF will need to determine what precision is acceptable (90/10 probably not feasible) 12 – Research Strategy

13. Straw-man Approach: Cost Need subcommittee feedback Currently indicated as $100K - $250K for data collection only 13 – Research Strategy

14. Discussion and Feedback Is the approach reasonable? – Likely to succeed as advertised? – Does estimated cost range match outlined research? – Recommended modifications? Is there a better research path? 14

15. Additional Slides 15 – Additional Slides

16. RTF Decision: Approval to “Planning” Category, “Active” Status RTF Decision: Approval to “Provisional” Category, “Active” Status. RTF Decision: Approval to “Proven” Category, “Active” Status. 16 December 2015 RTF Meeting Later, or maybe never Later still (or never)

17. What is a Research Strategy? Clarifies knowledge gaps in non-Proven measures Focuses on high-priority research objectives – What does the RTF need for the measure to be proven? – Anything researchers should pay special attention to? Outlines a straw-man approach to data collection and analysis – Demonstrates one feasible research path – Research Sponsors develop final Research Plan Sponsors can work with RTF staff to ensure plan addresses RTF needs RTF reviews final Research Plan later (at Provisional  Planning step) Calls out approaches that probably wouldn’t suffice (optional) Provides a rough cost estimate (based on straw-man approach) Research Strategies try to be BRIEF: Critical items shouldn’t get lost in a sea of helpful suggestions 17 – Additional Slides