1. New-Construction Manufactured Homes: Update on the Path to Proven Savings Regional Technical Forum November 10 th, 2015 Josh Rushton & Mohit Singh-Chhabra

2. Today’s Objective Review recent discussions related MH calibration and new construction measures Get RTF advice and consent on general direction for MH new construction measures – Staff/CAT recommends moving NEEM (ENERGYSTAR) and EcoRated Homes measures to Planning category – Research needed to directly compare average energy consumption in efficient-case homes and in standard (HUD+) new-construction homes – No change to measures for existing MHs 2

3. Measure Background Two measures for new-construction Manufactured Homes (MH) – MH ENERGY STAR (NEEM 1.1): Only shell specs – MH Eco-Rated: Shell specs + lighting + appliances – Confused about specs? See Additional Slides! Both measures currently – Category “Proven” – Status “Under Review” – Sunset November 30, 2015 3

4. Calibration History (Existing MH) August 2014: RTF approved MH Calibration for existing homes (Phases I & II) – Calibration captures adjustment factors that align modeled heating energy consumption (SEEM.69) with observed heating energy consumption (VBDD) – Adjustments correct an unknown mix of misalignments related to heating load, system performance, behavioral, and other factors December 2014: CAT presented Wx savings for existing MH based on calibration – Savings values much lower than expected – RTF directed CAT to review calibration, compare with pre/post program data June 2015: CAT showed how calibration results compare to pre/post data from Idaho Power program – Reasonable agreement for Wx savings in Electric Resistance (ER) homes; RTF approved calibration curve and Wx savings for existing MHs with ER heat – Poor agreement for Heat Pump (HP) conversions and Wx in HP homes; HP-related MH measures moved to planning; Research Strategy approved at August meeting 4

5. Calibration History (New MH) September RTF Meeting – CAT proposed extending calibration curve for new construction homes by adding 89 NEEM homes to the RBSA dataset – Baylon observed that VBDD naturally over-estimates space heating Algorithm picks up HDD-correlated energy consumption from other end-uses (e.g., winter-time increases in lighting and water heating) Worrisome because calibration objective was to align SEEM’s space heating estimates with VBDD May be especially important for NC homes with low heat loads Since September – NEEA contracted Ecotope and Energy RM to look into the VBDD question and assess whether calibration can reliably estimate savings for new construction measures – CAT worked with NEEA, Ecotope, and Energy RM to resolve easy issues and identify difficult issues 5

6. Recent Discussions 6

7. NEEA Working Group Concern 1 CAT Response in RED Back-of-envelope calculations suggest VBDD picks up about 1000 kWh of HDD-correlated consumption from end-uses other than space heating Makes sense. Easy to modify calculations – Subtract 1000 kWh from VBDD heating energy variable at the outset – Notation: Adjusted VBDD = VBDD.ht.kWh - 1000 – Leads to ≈1000 kWh reduction in baseline and efficient-case heating energy estimates No significant effect on savings estimates 7

8. NEEA Working Group Concern 2 CAT Response in RED The relationship between SEEM and VBDD is distorted when the calibration curve is defined in terms of percent differences (SEEM – VBDD)/SEEM – A linear relationship or simple off-set between SEEM and VBDD will appear curved when expressed in percent terms – This leads to more extreme adjustments for efficient homes than standard homes, and this will bias savings estimates CAT disagrees. Single relationship can be expressed in more than one way – Trend may look different under different transformations, but this doesn’t change the basic relationship (more on this next) – Can change how different points influence regression fit This can affect final fitted trend, especially when data is sliced thin Effect is not systematic – not a bias threat – No systematic effect on savings (see additional slides) 8

9. How do predictions based on percent-scale regression look when we convert them to predicted kWh and add them to the absolute-scale plot (left)? Concern 2: Percent differences vs absolute kWh Data filters: No heat pumps, R2 > 0.45, SEEM & VBDD < 20,000 kWh

10. Predictions from percent- difference regression roughly agree with trend in absolute kWh plot

11. Concern 2: Percent differences vs absolute kWh …a closer look at that lower plot Linear fit Predictions from percent- differences plot Loess smoother fit

12. NEEA Working Group Concern 3 CAT Response in RED Calibration may be capturing behavioral or other issues. Doesn’t seem consistent with pure-physics account of heat loss. Right – main concern is how much slower the meter spins 12

13. Concern 4.1: Comparing RBSA to NEEM CAT Response in RED 13 RBSA not a sufficient comparison group Only includes 6 non-NEEM homes built since 2004 Other RBSA U-values may not be consistent with NEEM U-values ∆kWh / ∆SEEM.69 (slope) differs between RBSA and NEEM samples CAT agrees data sets may not be similar enough for “fair” comparison Today’s recommendation based on this concern Probably shouldn’t read much into apparent difference in slopes NEEM homes all have the same U- values in SEEM input Data sets look more similar when x- axis scaled as kWh/sq.ft. (see additional slides)

14. Concern 4.2: Comparing NEEM to NEEM? CAT Response in RED 14 ∆kWh / ∆SEEM.69 (the slope) near 1 for NEEM homes Makes physical sense – maybe we should use this trend for calibration adjustments CAT disagrees. In NEEM sample, SEEM.69 only varies because of climate and sq. ft. Slope ≈ 1 only says SEEM.69 treats sq. ft. & climate about right in NEEM homes Not enough to generalize ∆kWh / ∆SEEM.69 to address changes in Uo

15. Proposed Path Forward 15

16. Motivation Need data that provides direct insight into difference between energy consumption in efficient new MHs and standard new MHs Ideally, data will include billing kWh and building specs for samples of efficient and standard new homes 16

17. Staff Recommendation New-Construction MH (ENERGYSTAR and EcoRated Homes): Move both measures to Planning. Research strategy to emphasize that savings estimate(s) should be closely tied to observed differences in kWh between efficient and baseline groups. – Final UES values likely to include adjustments based on SEEM models (plus observed ∆kWh / ∆SEEM.69) Examples: Savings by heating zone; cross-walk between NEEM and EcoRated savings estimates – Area of special emphasis: For basic savings estimates to be accepted, need fairly “clean” comparison across groups. Confounding variables (heat pumps, wood heat, etc.) tracked where possible to avoid major imbalances between the groups “Fair” comparison probably more important than wide regional representation or highly detailed audit data Existing MH measures: No change. We’ve already compared savings against IDP pre/post program data – More data always welcome; no need to move back to planning in the mean time 17

18. Discussion and RTF Direction Is this the right path? Can feasible research really get us to more solid ground? Should Staff/CAT develop a Research Strategy in order to move the ENERGYSTAR and EcoRated manufactured homes measures to the Planning category? 18

19. Decision needed “I, _______, move that the RTF extends the sunset date for both manufactured homes measures, ENERGYSTAR and EcoRated homes, to March 31, 2016” 19

20. Additional Slides 20

21. Alphabet Soup HUD. Minimum federal standard, some homes still being built to this spec – HUD update working its way through the system, likely to take effect within a few years – New HUD requirements expected to come close to the current NEEM specs (NEEM 1.1). HUD+. Estimated average spec for non-NEEM/EcoRated home since about 2010. NEEM 1.0 was SuperGoodSense after that program ended. NEEM 1.0 lasted until 2004. NEEM 1.1 refers to the NEEM program as updated in 2004 (current spec at time of this presentation) – Relative to NEEM 1.0, big change is that NEEM 1.1 locks down window efficiency at U-0.35 (prior to this, efficient windows could be traded off against other shell components). – NEEM 1.1 specs cover shell components, duct sealing and ventilation (exhaust fan, max 32 Watts, running continuously, no heat exchanger), plus a tiny DHW thing (0.93). – NEEM 1.1 meets current ENERGY STAR requirements, but the reverse does not always hold. It might not make sense for RTF to use the ENERGY STAR label in its measure workbooks because NEEM 1.1 is really the requirement, and the national ENERGY STAR spec would not always meet NEEM. (Marketers would still be able to use ENERGY STAR label for all NEEM homes) EcoRated is NEEM 1.1, plus window U-0.32, 80% efficient lighting, ENERGYSTAR fridge and dishwasher, and low-flow showerheads and faucets. HPMH is a very efficient spec that has been studied in small demonstration projects but currently has very little traction in the region. NEEM 2.0 doesn’t yet exist. It is currently a placeholder for the program that will replace NEEM 1.1 after the coming HUD update takes effect 21

22. Savings estimates according to three different calibration regressions (1) Estimated savings for existing construction measures. For demonstration only (figures have not undergone thorough QC) Represented calibrations: Percent-difference regression with original VBDD estimates – Essentially the calibration from September Percent difference regression with VBDD -1000 – Upper-left graph in four-plot slide above Absolute-scale regression – Upper-right graph in four-plot slide 22

23. Savings estimates according to three different calibration regressions (2) Estimated savings for existing construction measures. For demonstration only (figures have not undergone thorough QC) Represented calibrations: Percent-difference regression with original VBDD estimates – Essentially the calibration from September Percent difference regression with VBDD -1000 – Upper-left graph in four-plot slide above Absolute-scale regression – Upper-right graph in four-plot slide 23

24. 24 Both graphs use y = percent difference Only difference is x-variable: SEEM kWh/ft 2 (left) vs SEEM kWh (right) Left: As SEEM kWh/ft 2 varies, trend in percent differences similar between NEEM and RBSA Right: As SEEM kWh varies, trend in percent differences looks different between NEEM and RBSA This highlights the risk of drawing conclusions from a comparison of different data sets Note: x-variable choice (kWh versus kWh/ft 2 ) has little net effect on estimated savings in RBSA-only calibration Note on X-axis scale Data filters: No heat pumps, R2 > 0.45, SEEM & VBDD < 20,000 kWh