Goodbye, “Last Measure In” Hello, “Option X”? Staff Proposal for how to deal with the last measure in conundrum Regional Technical Forum October 15, 2013

Last Measure In - History Problem: Simple versus Accurate – Programs want simple deemed measures for weatherization and heat pumps. – With simple deemed measures, we don’t know The order in which measures get installed. The starting point from which the measures get installed Solution: – Last Measure In While admittedly conservative, LMI allows for a reasonable savings estimate. – Keep Weatherization separate from Heat Pumps Weatherization measure savings dependent on existing heating system 2

Then came along the Guidelines… 3

Guidelines “The UES for each measure should be computed under the assumption that all other measures it significantly interacts with are already implemented. Interaction is significant if the RTF determines that it is likely to account for more than 10% of the measure savings. The other measures assumed to be present should be consistent with expected typical conditions at the end of the measure’s effective useful life. This “last-in” requirement may create a downward bias in the short-term savings estimate for a measure.” 4 Savings, Section Interactions between Measures

Faithful implementation of the Guidelines appears to require that improvements in the HVAC system compete with shell improvements as a measure Simple logic: – Heat Pumps “significantly interact” with all the weatherization measures. – So, they should be included in the analysis. In order to get the savings right, the RTF needs to agree on a forecast of the future mix of heat pumps. – But how far out in the future should we look? Guidelines say EUL, but that’s probably too far. – What about first year savings? Heat Pumps don’t play well with Last Measure In. 5

Heat Pump’s Effect on Wx Savings sq.ft. prototype; Heating Zone 1 Fully Weatherized

… and SEEM “Calibration”. 7

Calibration Calibration required a change in thermostat setting between poorly insulated and better insulated homes: This creates a discontinuity in the relationship between UA and space heating use/savings. – Now a measure could save significantly more if it were the last measure in. For an example, see savings for Walls in the “Measure Order” slide in the additional slides section.Measure Order 8

ΔT-stat Setting’s effect depends on house UA So, insulation measures that begin with R-0 and cause an increase in T-stat setting save less if they’re done “first” (in a High UA house), versus “last” (in a Low UA house). This makes the last-in approach the opposite of conservative for these measures. 9

ΔT-stat Setting Affects other Measures, too In houses with an R-0 component, savings from other components will be affected based on their order with respect to the R-0 component. Example House: Attic R-0, Wall R-11, Floor R-11; Measure: Floor R-11 to R-25. – Scenario 1: Floors first Baseline and Efficient-case T-stat Setting: 66.5°F Lower savings from floors – Scenario 2: Floors last (Last Measure In) Baseline and Efficient-case T-stat Setting: 73.5°F Higher savings from floors 10

Measure Order Now let’s change the measure order… 1344 sq.ft. prototype; Heating Zone 1 11

So Now What? 12

Primary Options for Calculating Savings Last-measure in – Simple – Inaccurate T-stat settings Heating system conversions Requires a forecast SEEM’s complex interactions Savings Based on Existing Conditions at each House – Complex Requires an audit of the house (just like we used to do) – Most Accurate Both methods have many possible methods and sub-options; they can also be combined in different ways. 13

Some Options 14

Options Overview 15

Option 1A – LMI Status Quo Last measure in with all measures installed, except heating system known as a part of measure definition. – Separate attic insulation UES’s for a house with zonal heat and a house with a heat pump. Pro – Simple – Familiar – Deals with heating system accurately for weatherization measures Con – Doesn’t deal well with uninsulated components’ interactions. – Doesn’t adhere to guidelines with respect to last measure in Assumes all measures are installed, not expected measures at EUL 16

Options Overview 17

Option 1B – LMI assuming 85% of Cost Effective Measures Include all measures, including heating system, in last-measure in assumption at a rate of 85%. – One attic insulation UES, independent of existing heating system. Pro – Simple – Lines up with Council Planning Assumption – Attempts to line up with Guidelines with respect to last measure in Con – Assumed high penetration of heat pump conversions will cause a large underestimate of actual saving if penetration is not achieved. – Doesn’t deal well with uninsulated components’ interactions. – Many Others 18

Options Overview 19

Option 1C – LMI with Measure-specific Saturations Same as 1B, except estimate a saturation at end of measure’s EUL (different than 85%) for each measure component. Pro – Still simple – Lines up with Guidelines with respect to LMI. – Improves reliability (over 1B) of near-term savings by reducing error caused by heat pump penetration assumption Con – Savings reliability is sensitive to the heat pump saturation estimate – Doesn’t deal well with uninsulated components’ interactions. – Still requires a forecast – What should the correct forecast period be? Guidelines currently say at end of measure life, but that could be revised. 20

Options Overview 21

Option 1D – LMI on an Annual Basis Like Option 1C, but each measure would have a saturation forecast curve. This would allow unique savings to be calculated for each year of the measure life. Pro – Still simple program delivery – Reasonable savings estimate accuracy each year (assuming forecast is correct) Further improves reliability (over 1c) of near-term savings by reducing error caused by heat pump penetration assumption – Adheres to guidelines with respect to LMI But, Which savings estimate would the RTF use? 1 st year, average over the EUL, end of EUL, something else? – Deals with uninsulated components’ interactions. Con – Savings estimate accuracy depends on reliability of forecast – More involved analysis 22

Options Overview 23

Option 2A – Existing Conditions for Each House Base savings on existing conditions at site. Pro – Savings are most accurate (average across all participants) Eliminates any under counting and double-counting Accurately deals with uninsulated measure interactions and heating systems. – Lines up better with billing analyses, standard protocols, and custom measures Con – Very involved program delivery Requires identification of detailed as-found conditions at house – Requires many more UES values (or Standard Protocol) for each combination of existing conditions – Data collection/reporting errors could be large – Doesn’t adhere to guidelines w.r.t. LMI 24

Options Overview 25

Option 2B – Existing Conditions for RBSA Houses Calculate savings for each relevant characteristic scenario in the RBSA dataset, take the weighted average. – Example Measure: Attic Insulation R-0 to R-38 Filter RBSA dataset for houses with R-0 attic insulation Determine percentage of houses with specific characteristic scenarios – Example: » 5% have Zonal heat, Walls at R-0, Floors at R-0, etc. » 12% have Zonal heat, Walls at R-11, Floors at R-0, etc. » Etc. Pro – Does not impact program delivery – Savings fairly accurate because known distribution of starting points Con – Doesn’t deal with multiple measures installed simultaneously Heat pumps and weatherization – Component saturations need to be updated before they move too much Need new RBSA data. 5 year schedule adequate for Weatherization? heat pumps? – More involved analysis than status quo, many many measures and permutations. – Relies on Program house characteristics matching RBSA – Some data (infiltration, duct tightness) not available on each house – Doesn’t adhere to guidelines w.r.t. LMI 26

Options Overview 27

Option 3 – Use Existing Conditions for RBSA houses (2B), but prorate measure savings using ratio of LMI (1A) to “full package savings” Similar to Option 2B (Existing Conditions for RBSA Houses), except for within each characteristic scenario, calculate savings for the entire “full measure package” (all the way to Attic R-38, Wall R-11, Floor R-25, etc.), then prorate each measures’ LMI savings based on the ratio of the “LMI full measure package savings” to “actual full measure package savings”. Weatherization measures would be defined for each existing heating system type 28

Option 3: Explained With Numbers 29 Note: Only one Characteristic Scenario is shown here for simplicity. Instead, for each measure, the ratio “actual/LMI” would be determined for each applicable characteristic scenario and weighted according to its frequency of occurrence in the RBSA. Note 2: Heat Pump conversion measures would be calculated in the same manner, but they would receive their own Actual/LMI ratio’s.

Option 3 Pro – Deals with multiple measures installed at the same time, or over time. – Early investigation by staff shows this should work well for weatherization measures with a known heating system Con – Difficult to explain – More complicated analysis New measures may require re-running analysis to develop new ratios – Accuracy can depend on how close houses get to the full measure package. – Doesn’t adhere to guidelines w.r.t. LMI 30

Options Overview 31

Option 4 – Use Option 3 for weatherization measures, and Option 2A for heat pump measures For weatherization measures, with measure specification tied to existing heating system type: – Use Option 3 – Use Existing Conditions for RBSA houses (2B), but prorate savings using ratio of LMI (1A) to “full package savings” For heating system conversion measures: – Use option 2A – Use existing conditions for each house. Audit house at each site and use the audit data to calculate a house-specific savings value. Pro’s – Keeps weatherization program operation simple – Deals with multiple measures and interactions between measures – Keeps accurate heat pump savings Con’s – For weatherization, accuracy depends on definition of “full package” – For heat pumps, significant increase in program complexity Heat pump contractors are the likely ones to provide house insulation levels and other necessary characteristics (“necessary” would still need to be defined) – Doesn’t adhere to guidelines w.r.t. LMI 32

Options Overview 33

Ok… There’s no Perfect Solution How do we assess reliability so we can pick the least imperfect Option? (And once we do that, is it sufficiently reliable, or should we revert to program impact evaluation?) 34

Comparing Methods Given: – We know Options 3 and 4 won’t give us “Actual” savings every time. Question: – On average, how well will each option give us “actual” savings? Problem: – What is “average”? We don’t know: which measures will be installed, and which houses they’ll be installed in. – Assumption For the purposes of the Comparison exercise, let’s assume: All possible measure combinations have an equal likelihood of being installed. 35

Setting Up the Comparison Each Characteristic Scenario, has its own suite of possible measures, measure orders, and stopping points. For each possible measure installation order and stopping point, we can calculate the Ratio: Option’s Savings to “Actual” Savings. Then, we can generate a histogram of the results, for each option… 36

Characteristic Scenario #1: Zonal, Attic R-19, Wall R-11, Floor R-11, Windows U

Characteristic Scenario #1: Zonal, Attic R-19, Wall R-11, Floor R-11, Windows U

Characteristic Scenario #1: Zonal, Attic R-19, Wall R-11, Floor R-11, Windows U

We do this for each of the 91 Characteristic Scenarios, Apply the RBSA Weight for each Scenario, 40 And Generate the following Histograms…..

41

42

43

Note: High and Low extremes have been cut off and are not included in the average (these are likely places where the “actual” savings are incorrect. 44 Weighted Averages Option 4: 1.00 Option 3: 0.98 LMIsq: 0.96

Staff Recommendation 45 Option 3 – Reliable While the comparison method is not perfect, it provides some confidence that Option 3 is likely to provide reliable savings, on average. – Doable Program implementation will be mostly unchanged (“average heating system” case will be removed). (Early discussions imply Option 4 could be a deal-breaker for programs.) It requires more complicated analysis than LMIsq (Option 1A), but it’s probably worth the extra effort. Option 1A – LMI Status Quo Option 1B – LMI assuming 85% of cost-effective measures Option 1C – LMI with measure-specific saturations Option 1D – LMI on an annual basis Option 2A – Use Existing Conditions for each house Option 2B – Use Existing Conditions for RBSA houses Option 3 – Use Existing Conditions for RBSA houses (2B), but prorate savings using ratio of LMI (1A) to “full package savings” Option 4 – Use Option 3 for weatherization measures, and Option 2A for heat pump measures

Decision “I __________ move the RTF use Option ___ in estimating savings for residential heating system and weatherization measures.” Note: The guidelines are currently being reviewed. Today’s decision will be taken into consideration when proposing edits to the section of the guidelines that deals with last measure in. 46

Extra Slides 47

Measure Order Now let’s change the measure order… 1344 sq.ft. prototype; Heating Zone 1 48

Sunday vs SEEM and LMI Sunday said: A weatherization measure saves less when it’s installed in a more efficient house. – This made sense. As the house gets more efficient, internal and solar gains meet a larger percentage of its heating needs. – This means LMI is always conservative 49

SEEM Hourly simulation, models the effects of: Solar gains, ground contact, crawlspace and attic buffer spaces, thermal mass, infiltration, radiant heat transfer, and duct leakage. 50 SundaySEEM

Let’s Compare Sunday: (again) SEEM: 51 “Last Measure In” is no longer predictably conservative.