Download presentation
Presentation is loading. Please wait.
Published byGriffin Cooper Modified over 9 years ago
1
Measuring the Effectiveness of Your Energy Efficiency Efforts Carol Mulholland American Public Power Association National Conference June 16, 2009
2
Contents Why Implement Energy Efficiency Programs? How Do You Know They Work? Evaluation Overview Process Evaluation Impact Evaluation
3
Why Implement Energy Efficiency Programs? Rising fuel costs Rising demand for electricity More concern over emissions leading to difficulty building new power plants More global competition for energy More profitable from utility business perspective Environmental (global warming) concerns
4
Policy Changes We Are Seeing Today Cost Recovery Program spending on the rise (elec/gas) Looking at DSM as alternatives or deferral of T&D upgrades RPS and EPS on the rise First forward capacity market Local politicians in the GHG arena Potential for carbon tax
5
Cost of EE vs. Price of Electricity: July 2005 Source: State of Delaware, Sustainable Energy Utility Task Force Briefing Book, http://www.seu-de.org/documents.html
6
Energy Efficiency Uncertainties Motivation -Regulatory, capacity, T&D or energy affordability? Focus-Internal (system needs) external (customer needs)? Policy Context-State Energy Efficiency Portfolio Standards, Federal Carbon tax? Market -Size and make-up of the market for EE by sector? Financials -Cost and benefits, avoided costs, compared to what?
7
Contents Why Implement Energy Efficiency Programs? How Do You Know They Work? Evaluation Overview Process Evaluation Impact Evaluation
8
How Many of You... ...Are counting on energy efficiency/DSM programs to offset increased demand? ...Are counting on energy efficiency to help offset clip peaks? ... Are implementing energy efficiency/DSM programs because someone has told you to? ... Have detailed program operations manuals that explain how your utility's energy efficiency programs work? ...Maintain a database of all program expenditures and estimated savings?
9
If you answered “YES” to the questions in BLUE, you need to evaluate your programs to find out whether you’re getting the results you expect. If you answered “YES” to the questions in GREEN, you’re already part way there.
10
Contents Why Implement Energy Efficiency Programs? How Do You Know They Work? Evaluation Overview Process Evaluation Impact Evaluation
11
The Importance of Program Evaluation Definition: Evaluation is a systematic process for an organization to obtain information on its activities, its impacts, and the effectiveness of its work, so that it can improve its activities and describe its accomplishments.* Benefits: An effective evaluation will help you learn about your successes [and shortcomings], share information with key audiences, and improve your services.* Other reasons: To assign a value to energy efficiency as a potential asset To verify impacts for capacity and carbon trading To compare EE investments against alternative investments for meeting demand (integrated resources planning)
12
Why Evaluate? Evaluation: Measures program impacts Provides valuable information about program operations Offers a positive customer contact opportunity Shows what you’ve accomplished to all interested parties, including City Councils and Utility Boards Meets regulatory requirements, for those who have them
13
What are you measuring? Energy savings Demand Savings Customer satisfaction Technology penetration
14
The Way it Should be However, the role of evaluation can go well beyond simply documenting savings to actually improving programs and providing a basis for future savings estimates If applied concurrently with program implementation, evaluations can provide information in real time to allow for course correction if needed In summary, evaluation fosters more effective programs and justifies increased levels of energy-efficiency investment. Perhaps this was best put by Economist John Kenneth Galbraith, “Things that are measured tend to improve.”
15
Interaction Process Evaluation Market Assessment Impact Evaluation
16
When To Evaluate? ProgramImplementationProgramImplementation Program Design ProgramEvaluationProgramEvaluation is what makes evaluation useful The feedback loop…
17
Evaluate Early and Often Evaluation should be an ongoing process Feedback loop – measure change over time Use the information – What you discover may cause you to alter your strategy as you go. Make mid-course corrections as necessary. Interpretation is the most important step – Be careful how the data you have collected is interpreted (and by whom). You should carefully control the outcomes of evaluation such that raw data does not get misinterpreted by non-professionals.
18
Contents Why Implement Energy Efficiency Programs? How Do You Know They Work? Evaluation Overview Process Evaluation Impact Evaluation
19
Process Evaluation Assesses the process a program undergoes during implementation Documents program goals and objectives from a variety of perspectives Describes program strengths and weaknesses so that success is highlighted and improvements can be made
20
Designing a process evaluation Identify target audiences Determine what type of research to undertake with each (e.g. Interviews? Surveys? Focus groups?) Lay out the evaluation timeline, taking into consideration both internal and external deadlines Important to seek out independent 3 rd party for an effective evaluation – even more than impact work To encourage candor on the part of delivery agents and customers alike To provide credibility with outside parties that an unbiased view will be offered
21
Market Assessment and Market Transformation Market Assessment looks at the broader market for EE products and services within which a program operates It is typically done BEFORE or AS PART OF program design It can be done as an EVALUATION ACTIVITY, when looking at the broader market effects of an EE program (e.g, sometimes rebate programs may increase product availability and drive product prices down, resulting in…) Market Transformation Market transformation is a goal for EE programs that seek to overcome significant barriers to adoption in the marketplace
22
Market Assessment Goals To determine the size of the program market To assess the market’s characteristics To find out what impact, if any, the energy efficiency programs had
23
Barriers to Market Transformation High prices Low availability Low awareness Lack of education as to use of product or its merits Poor infrastructure for proper installation and support of product
24
Content Why Implement Energy Efficiency Programs? How Do You Know They Work? Evaluation Overview Process Evaluation Impact Evaluation
25
What do we mean by impacts? Demand and energy savings, market effects, and the environmental and economic costs and benefits that result from an energy services program. Savings are typically reported on a gross and net basis. “Net energy and demand savings” adjusts the gross savings for a variety of factors.
26
Methods of Estimation Program Impacts Energy –Engineering Simple Engineering Simulation Models –Statistical Simple Pre/Post (difference of means) Regression (Acct for weather) Comparison Group (Quasi Experimental Design) Detailed Regression Demand –Existing Load Factors –Secondary Load Shapes –End-Use Metering –Simulations Data Collection –Surveys –Billing Data –Metering –Site Visits
27
What Do You Measure in Impact Evaluation? Tracking system gross energy and demand savings Adjusted gross energy and demand savings Realization rates Program verified gross energy and demand savings 4. Program Verified 3. Realized 2. Adjusted Gross 1. Tracking System Gross Energy and Demand Savings
28
What Do You Measure in Impact Evaluation? Net-to-Gross or Attribution Factors are typically measured through customer surveys. These factors are then applied to program verified gross savings to estimate the net energy and demand savings attributed to the Program’s activities. Net-to-gross factors include: Free ridership Spillover BECAUSE these can cancel each other out, some studies do not bother to measure either one!
29
What Do You Measure in Impact Evaluation? Economic costs and benefits net energy and demand savings, program costs, participant costs, societal costs and benefits – all are estimated to calculate benefit-cost ratios for various cost-effectiveness tests Environmental benefits Sum up electricity savings (and/or on-site fuel savings, e.g., natural gas) multiply those savings by emission factors to produce pounds of emissions saved or avoided emission factors are expressed as pounds of pollutant per MWh of electricity or per therm of natural gas
30
Approach to Impact Evaluation A typical impact evaluation requires the following activities or elements: Gather/review program background and savings information Develop sampling/analysis approach Conduct surveys and interviews Conduct project site data collection Analyze data to develop measure and project level gross savings Develop gross and net energy and demand savings by program
31
Final Comments Set up the evaluation when you set up the program Get your vendors to do your data tracking, provide monthly status reports (the program will evaluate itself!) Make sure your processes are seamless and transparent, so someone else can easily get what they need Remember this is art, not science!
32
Carol Mulholland Carol.mulholland@cadmusgroup.com
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.