1. Manufactured Homes Calibration: Existing and New Homes Josh Rushton & Mohit Singh-Chhabra RTF Update June 16, 2015

2. Presentation Objectives Approve existing manufactured homes calibration based on comparative analysis with program data Review applicability of existing manufactured home comparison to New Construction based on comparison with reported billing analysis of NEEM homes 2

3. EXISTING MANUFACTURED HOMES CALIBRATION 3

4. Calibration Timeline, Existing MHs August 2014: RTF approved MH Calibration (Phases I & II) December 2014: CAT presented Wx savings based on calibration – Savings values much lower than expected – RTF directed CAT to review calibration, compare with program data March 2015: CAT summarized available data and proposed subcommittee approved methodology for evaluating applicability of existing home calibration – RTF gave head-nod on proposed methodology June 2015 (Today): – Review results of calibration-program data comparison 4 – Existing MH Calibration

5. Two comparisons Phase I curve comparison: Compare existing RBSA-based calibration curve to pre-/post- program data Savings comparison: Compare bottom-line calibrated savings estimates to actual program-billing data Preview: Things look pretty good for weatherization in homes with electric- resistance heat. Heat pumps are a problem. 5 – Existing MH Calibration

6. Phase I Curve: Review Phase I adjustments based trend in differences between SEEM.69 and VBDD that were observed in RBSA – SEEM.69: Heating kWh estimated by SEEM (given standardized inputs, such as 69/64 ⁰F day/night t-stat) – VBDD: Heating kWh estimated from billing data (via Variable-Base Degree-Day algorithm) – Data set: Portion of MH RBSA with good VBDD fits and no wood fireplaces, hot tubs, etc. (n=140) Weak link: RBSA not a pre-/post- data set! 6 – Existing MH Calibration

7. Phase I Curve: Review 7 – Existing MH Calibration Average SEEM.69 too low for more efficient homes Average SEEM.69 too high for less efficient homes MH RBSA Homes

8. Phase I Curve: Review 8 – Existing MH Calibration Will the trend carry through pre/post? Answer affects how saving estimates should use calibration: Adjustment factors that change pre- /post- Adjustment factor(s) that don’t change pre-/post- Something else? MH RBSA Homes

9. Phase I Comparison: Data Idea: Compare calibration curve to actual pre-/post- program data. Data: From 2011-12 Idaho Power (IDP) low-income weatherization assistance programs Includes both Wx and heat pump conversion measures Initial sample: n = 106 homes with... – Complete pre-/post- audit data (with blower door test results) – At least one full heating season pre- and post- Analysis sample: n = 85 homes… – Filtered to have “okay” VBDD fits pre and post – Not able to identify/filter out fireplaces, hot tubs, etc First step: Generate SEEM.69 and VBDD values for each IDP site, pre- and post-weatherization 9 – Existing MH Calibration

10. Apples to apples: SEEM inputs (1) Many SEEM inputs come directly from IDP audit data… Location (for climate) Square footage (floor area as well as windows/doors/walls) U-factors, pre and post, for each shell component – Based on auditor estimates of nominal R-value and U-factor table from DOE Low-Income WAP – Mapping from building characteristics to U-factors differs from that used in RBSA (and the SEEM calibration) Some noticeable differences: For most vintage cohorts, pre-case IDP window- and door- U somewhat higher, and ceiling-U somewhat lower, than in RBSA. Little net difference: Uo values very similar between IDP pre-cases and RBSA (true for all vintage cohorts up to mid-1990s; see additional slides) Infiltration, pre- and post, from blower-door tests – RBSA infiltration rates also based on blower door tests (values for homes w/o tests were imputed based on values for homes with tests) Equipment type (electric resistance furnace or heat pump) – For SEEM HP parameters, used same efficiency levels (HSPF 7.9 / SEER 13.0), and same iterative sizing technique, as the calibration 10 – Existing MH Calibration

11. Apples to apples: SEEM inputs (2) Duct tightness inputs not taken directly from audit data IDP data indicates some duct sealing activity for many sites – Data not available for all sites – 69% of sites with complete entries indicate some form of duct sealing IDP entries for pre/post duct efficiency are based on visual inspection and a BPI table relating observable characteristics to efficiency levels – CAT didn’t use these reported efficiency values in SEEM inputs Values not likely to be consistent with measured leakage values in RBSA For the record, reported efficiency levels averaged 62% (pre) and 87% (post) – CAT’s analysis assumes 9% heating energy savings (based on Avista evaluation of prescriptive MH duct sealing program) for all homes that received duct sealing For SEEM inputs, used RBSA averages for duct tightness – Separate averages for pre-92 and post-92 homes (same as RBSA missing-value imputation) – Duct tightness inputs do not change pre/post – Post-case calibrated-SEEM values reduced by 0.69*0.09 = 6.2% to account for duct sealing. Detail: Reduced post-case uncalibrated SEEM until post-case calibrated SEEM was 6.2% lower Result: Graphs below look exactly as they would if we’d found post-case duct leakage values that reduce calibrated SEEM by 6.2% 11 – Existing MH Calibration

12. Initial Comparison: Phase 1 Trend 12 – Existing MH Calibration Differences SEEM.69 – VBDD for IDP sample versus trend captured in RBSA Phase I curve values SEEM.69/ft 2

13. Initial Comparison: Phase 1 Trend 13 – Existing MH Calibration Differences SEEM.69 – VBDD for IDP sample versus trend captured in RBSA All but two of these had HPs installed Phase I curve values SEEM.69/ft 2

14. Initial Comparison: Phase 1 Trend 14 – Existing MH Calibration Differences SEEM.69 – VBDD for IDP sample versus trend captured in RBSA All but two of these had HPs installed SEEM.69/ft 2

15. Initial Comparison: Residuals 15 – Existing MH Calibration IDP differences SEEM.69 – VBDD after Phase I adjustment Mean difference much closer to zero for most x- value ranges. All but one of these had HPs installed SEEM.69/ft 2

16. Apples to apples: Phase II “Calibrated SEEM” means Phase I and Phase II adjustments have been made. All figures and graphs describe “okay-VBDD” homes For these homes, Phase II reduces electric heating kWh by about 5.5%, pre and post For unfiltered homes, the Phase-II adjustment would be about twice as large. 16 – Existing MH Calibration

17. Initial Comparison: Averages 17 – Existing MH Calibration 85 “okay-VBDD” sitesVBDD Calibrated SEEM (RTF- approved) Constant- factor * Calibrated SEEM ** Uncalibrated SEEM.69 ** Heating kWh (Pre)9,93610,418 18,157 Heating kWh (Post)6,851 6,463 5,2249,065 Difference3,085 3,956 5,1949,091 * Constant-factor Calibrated SEEM applies the base-case Phase I adjustment factor to raw SEEM output pre and post. ** To account for duct sealing, Constant-factor Calibrated SEEM and Uncalibrated SEEM.69 both use SEEM.69*(1 - 0.062) for post-case SEEM output.

18. Initial Comparison: Averages 18 – Existing MH Calibration

19. A Closer Look 19 – Existing MH Calibration Sites with ER heat pre and heat pump post (many have Wx measures in addition to HP conversion) Sites with electric- resistance heat pre and post Excludes 2 homes that had HPs prior to program participation

20. A Closer Look 20 – Existing MH Calibration Sites with ER heat pre and heat pump post (many have Wx measures in addition to HP conversion) Sites with electric- resistance heat pre and post Excludes 2 homes that had HPs prior to program participation

21. A Closer Look 21 – Existing MH Calibration Sites with ER heat pre and heat pump post (many have Wx measures in addition to HP conversion) Sites with electric- resistance heat pre and post Excludes 2 homes that had HPs prior to program participation

22. A closer look: Weatherization Based on Wx-only portion of IDP sample… Approved calibration estimates savings about 9% lower than VBDD Difference is not statistically significant Approved Phase I trend looks reasonable against pre-/post- data 22 – Existing MH Calibration 47 weatherization-only sites, pre and post

23. CAT Recommendation (part 1) Recommendation: Accept the current electric- resistance calibration for existing MH – Reasoning: Results are consistent with pre-/post- Wx data shared by Idaho Power – Consequence: Would allow proven savings for weatherization in homes with electric resistance heat Decisions on actual measures taken separately. 23 – Existing MH Calibration

24. A Closer Look: Heat pump retrofits 24 – Existing MH Calibration 32% lower28% lower

25. A Closer Look: Heat pump retrofits 25 – Existing MH Calibration 32% lower28% lower 30% lower

26. A Closer Look: Heat pump retrofits 26 – Existing MH Calibration 32% lower28% lower 30% lower 32% lower ?!?

27. CAT Recommendation (part 2) Recommendation: The RTF does not have reliable calibrated model for estimating heating energy in manufactured homes with heat pumps. – Reasoning: IDP heat pump sample gives reason for doubt, but does not provide sufficient insight for proven-worthy calibration Provides little insight into HP savings in non-weatherized homes May not generalize (it’s only 36 low-income sites in one program) – Consequence: MH measures related to heat pumps (equipment conversion, duct sealing, and weatherization in homes with existing heat pumps) should become Planning measures. Staff should develop a research strategy to improve consumption and savings estimates for MH measures that are related to heat pumps MH weatherization workbook should be split out into Proven measures (electric resistance heat) and Planning measures (heat pumps), similar to single-family. Decisions on actual measures taken separately. 27 – Existing MH Calibration

28. Decisions “I, _______, move that the RTF’s currently approved calibration does reliably estimate heating energy and savings in manufactured homes with electric resistance heat.” “I, _______, move that the RTF does not have a reliable calibrated model for estimating heating energy in manufactured homes with heat pumps.” 28 – Existing MH Calibration

29. NEW HOMES CALIBRATION 29

30. New Construction Calibration Timeline August 2014: RTF approved MH Calibration (Phases I and II) October 2014: RTF directed staff to re-think the calibration for new construction – Do new construction home properties differ enough from existing to warrant a separate calibration? June 2015 (Today): – Review existing new construction manufactured homes calibration based on comparison with reported billing analysis of NEEM homes 30 – New-Construction MH Calibration

31. New Construction: Data SEEM.69 models constructed as per NEEM spec and compared to average billing data SEEM.69: NEEM specs provided basis for models – Note: NEEM specs do not cover all building properties required by SEEM MH calibration standardized inputs used for SEEM input fields not covered by NEEM specs Duct leakage and infiltration values from the 2006 NEEM MH study (Summary of 2006 NEEM MH: Field Data and Billing Analysis – Ecotope ) 31 – New-Construction MH Calibration

32. New Construction: Data Contd. VBDD: Billing data studied for 78 homes across all 4 states in the same 2006 NEEM study – VBDD billing analysis results available for electric resistance and heat pump homes Authors of the report advised that the HP numbers may not be reliable 32 – New-Construction MH Calibration

33. How does this compare to our existing curve? 33 – New-Construction MH Calibration Single SEEM model constructed with NEEM specs is used to generate all six data points across 3 heating zone and 2 heating system types A range of new construction efficiencies would provide better basis for comparison Although the new construction data points align well with existing calibration curve, we don’t have enough variety of data to conduct a good statistical comparison no pre post data possible for new construction measures

34. Staff Recommendation Insufficient data available to conduct reliable calibration – lack of data on homes with varying efficiencies to validate existing home calibration – Note: reliable calibration required to develop proven measures Final call for billing data; measures sunset in November! – More data analysis required before recommending proven for MH new construction measures In the absence of more data, staff will develop planning measures with a research strategy using existing MH calibration curve 34

35. BACKUP SLIDES 35

36. VBDD Heating Energy Calculations Required minimum of 5 heating-season billing periods pre and post Most sites had significantly more – 91 of 106 had at least 8 pre and 8 post – 79 had at least 10 pre and 10 post Billing periods with at least 50 HDDs (under site-specific base) counted as “heating- season” 36 – Additional: Existing MH

37. Phase I Curve Comparison: Trend 37 – Existing MH Calibration Trend in the SEEM.69 - VBDD differences with pre/post data Phase I curve value ID Power homes, pre and post, “okay-VBDD” sample Wx-only sites:

38. Phase I Curve Comparison: Trend 38 Additional: Existing MH Phase I curve value ID Power homes, pre and post, “okay-VBDD” sample HP-install sites: Trend in the SEEM.69 - VBDD differences with pre/post data

39. Phase I Curve Comparison: Residuals 39 Additional: Existing MH Pre/post program data after Phase I adjustment ID Power homes, pre and post, “okay-VBDD” sample Wx-only sites:

40. Phase I Curve Comparison: Residuals 40 – Existing MH Calibration Pre/post program data after Phase I adjustment ID Power homes, pre and post, “okay-VBDD” sample HP-install sites:

41. Consumption estimates, pre and post 41 -- Additional: Existing MH

42. Comparing IDP and RBSA U-factors For each shell component, calculated RBSA averages by vintage cohort (pre-1977, 1977- 85, 1986-92, 1993-99, post-1999) Compared to IDP pre-Wx values to check consistency between IDP and RBSA U-factor conventions IDP seems to be using higher U-factors for windows and doors, lower for ceilings, about right for Uo. 42 -- Additional: Existing MH

43. Comparing IDP and RBSA U-factors 43 -- Additional: Existing MH Pre-case Uo values in the Idaho Power data set are similar to Uo values in the RBSA (see previous slide), and this is true of all vintage cohorts up to the mid- 1990s. A more detailed analysis found IDP window and door U-factors somewhat higher than RBSA figures (for most cohorts), and ceiling U- factors somewhat lower than those of the RBSA.

44. New Construction: Note on Phase I Curve Comparison 44 Additional: New Construction MH Nothing compares, nothing compares… to pre-post NC points should fall somewhere in this region Would not be able to reproduce phase 1 filters for this. Only meaningful if SEEM input conventions consistent across types… – NC vs. RBSA points – Baseline NC vs. NEEM, etc. – Lots of assumptions here