Savings Are What You Define Them To Be: The “Baseline” Issue Tom Eckman Manager, Conservation Resources Northwest Power and Conservation Council

Defining Savings and Baselines Savings - The difference in energy use between the baseline and post (after measure delivery) periods, which is caused by the installation/use of a measure or is attributable to one or more “mechanisms” (e.g, programs, codes and standards, etc). Savings - The difference in energy use between the baseline and post (after measure delivery) periods, which is caused by the installation/use of a measure or is attributable to one or more “mechanisms” (e.g, programs, codes and standards, etc). Baselines Can Be: Baselines Can Be: – Current/common practice = baseline is used if the measure affects systems, equipment or practices that are at the end of their useful life. In this case baseline use is defined by the recent typical choices of eligible end users in purchasing new equipment and services. – Pre-conditions = baseline is used when the measure-affected equipment or practice still has remaining useful life. In this case baseline use is defined by typical existing conditions. 2

Baseline Types Current Practice Baseline Savings Pre-Condition Baseline Savings

Baseline Types – The “Special One” Because it Comes in Two-Parts Current Practice Baseline Savings for EUL Early Retirement Baseline Savings for RUL Persistent savings is only equivalent to savings over “current practice,” and there’s a risk in might be less

Savings Are A Function of Perspective Resource Planning Perspective – Savings are the difference between forecast energy use and efficient case energy use Resource Planning Perspective – Savings are the difference between forecast energy use and efficient case energy use Program Planning and Evaluation Perspective – Savings are the difference between the energy use that would have occurred “absent the program” and the energy use of program participants* Program Planning and Evaluation Perspective – Savings are the difference between the energy use that would have occurred “absent the program” and the energy use of program participants* *Including spillover and market effects

slide 6 Resource Planning Perspective Focuses on Total Energy Use Reduction From All “Forces”

Program Planning and Evaluation Perspective Attempts to Determine Attribution To Each “Force” Source: Jayaweera, PhD, Tina and Hossein Haeri, PhD. “Methodology for Quantifying Market-Induced, Non-Programmatic Savings.” Prepared by Cadmus for BPA. April 1,

The Relationship Between Newtonian Physics and “Attribution” Newton’s First Law: Corpus omne perseverare in statu suo quiescendi vel movendi uniformiter in directum, nisi quatenus a viribus impressis cogitur statum illum mutare. Newton’s First Law: Corpus omne perseverare in statu suo quiescendi vel movendi uniformiter in directum, nisi quatenus a viribus impressis cogitur statum illum mutare. Translation 1 by Dave Berry – A body at rest will remain at rest until 8:45 p.m. The night before the science-fair project is due, at which point the body will come rushing to the body’s parents, who are already in their pajamas, and shout, “I just remembered the science fair is tomorrow and we gotta go to the store right now!” Translation 1 by Dave Berry – A body at rest will remain at rest until 8:45 p.m. The night before the science-fair project is due, at which point the body will come rushing to the body’s parents, who are already in their pajamas, and shout, “I just remembered the science fair is tomorrow and we gotta go to the store right now!” Translation 2 - An object remains at rest or in motion unless acted upon by an external “force” Translation 2 - An object remains at rest or in motion unless acted upon by an external “force”

A Power Planner’s View of the Derivation of Attribution & “Net-to-Gross” Ratios

slide 10 A Power Planner’s View of Attribution and Net-To-Gross We Don’t Care Who Was Responsible for Screwing in the CFLWe Don’t Care Who Was Responsible for Screwing in the CFL We Do Need to Know ThatWe Do Need to Know That –It got installed –How much electricity it will use –When it will use electricity –How long it will function

Implications for EM&V There is no such thing as “naturally occurring” efficiency improvements There is no such thing as “naturally occurring” efficiency improvements An external “force” must be exerted to move inefficient conditions to efficient conditions An external “force” must be exerted to move inefficient conditions to efficient conditions “Net-to-gross” evaluations attempt to quantify the magnitude of each “force” “Net-to-gross” evaluations attempt to quantify the magnitude of each “force” The choice of a “baseline” is not an assignment of “attribution” The choice of a “baseline” is not an assignment of “attribution” – But it does matter!

Should The Resource Planning and Program Planning “Perspectives” Use Different Baselines? No – If the reported savings are used to determine the need for additional resources No – If the reported savings are used to determine the need for additional resources Maybe – If the reported savings are used to determine compliance with an “EERS” (Energy Efficiency Resource Standard) which was set independent of resource need Maybe – If the reported savings are used to determine compliance with an “EERS” (Energy Efficiency Resource Standard) which was set independent of resource need Yes – I can’t think of any logical reason Yes – I can’t think of any logical reason

Why Both Perspectives Should Use Current/Common Practice Baselines Consistency with Load Forecast Consistency with Load Forecast – The energy use of an appliance or piece of equipment that form the basis for load forecast should also be the basis for determining the remaining potential for conservation – Determining whether all or only a portion of the savings attributed to a program actually impact the need for resources cannot be done without comparison to what was explicitly or implicitly assumed in forecasting that need

Why Both Perspectives Should Use Current/Common Practice Baselines Experimental Design - Determining what is occurring prior to program launch is a better measure of what would have occurred absent the program (i.e., the counterfactual) than evaluations made after the program has influenced the market Experimental Design - Determining what is occurring prior to program launch is a better measure of what would have occurred absent the program (i.e., the counterfactual) than evaluations made after the program has influenced the market

Why Both Perspectives Should Use Current/Common Practice Baselines Establishing common practice baselines requires information on current market conditions prior to program or project design and implementation. Establishing common practice baselines requires information on current market conditions prior to program or project design and implementation. – As a result, assessments of both the need for and design of market intervention programs can be conducted prior to implementation. – This reduces the probability of initiating programs where standard practices is already efficient and increases the likelihood of the targeting of niche markets that are not.

Why Both Perspectives Should Use Current/Common Practice Baselines Use of a common practice baseline generally results in much smaller differences between gross and net savings (i.e., higher NTG ratios). As a result, regulatory processes, such as cost recovery proceedings, tend to be less acrimonious

Why Both Perspectives Should Use Current/Common Practice Baselines Use of Common Practice baselines tests whether incremental savings and further investments in energy efficiency are cost- effective to pursue, regardless of who gets credit for them. Use of Common Practice baselines tests whether incremental savings and further investments in energy efficiency are cost- effective to pursue, regardless of who gets credit for them.

Use of Common Practice Baselines Does Not Protect Against Self-Selection Bias However, the market research or other data analysis that must be carried out in order to estimate a common practice baseline can and should inform program designs so they more effectively target consumers who are not already adopting the efficient technology or practice But, Poorly Designed Programs Are Still Possible!

Use of Common Practice Baselines Does Not Capture Non-Participant Spillover On the other hand, very few NTG evaluations capture non-participant spillover, so whether this is better or worse than the current majority paradigm isn’t clear. On the other hand, very few NTG evaluations capture non-participant spillover, so whether this is better or worse than the current majority paradigm isn’t clear.

Use of Common Practice Baselines Does Not Give Credit For Market Effects of Prior Programs On the other hand, very few NTG evaluations capture market effects of prior programs, so whether this is better or worse than the current majority paradigm isn’t clear. On the other hand, very few NTG evaluations capture market effects of prior programs, so whether this is better or worse than the current majority paradigm isn’t clear.