Experience you can trust. NTG Estimation for California IOUs’ Upstream Lighting Program CALMAC Meeting Pacific Energy Center July 18, 2007 Tami Rasmussen, KEMA Inc.

2 Organization of Presentation  Program characteristics  NTG assessment objectives  Methods considered – Overview of each method – Caveats – Whether method was used/if not, under what contexts method may be useful – Keys to success for method used

3 Program Characteristics  Upstream program – lighting retailers and manufacturers receive the incentives (point-of-sale or buydown) – Consumers may or may not be aware that the price of the product they are buying has been reduced by the program  Prior California IOU lighting programs have created broad and cumulative market effects  Prior and current programs have influenced the market beyond the specific products rebated

4 NTG Assessment Objectives  Prior Upstream Lighting program evaluations focused on characterizing the market and establishing linkages between program interventions and market change – Attribution was assessed qualitatively  study required an explicit estimate of NTG – Estimate must include only free-ridership – spillover may be estimated separately but must be excluded from final NTG estimate

5 NTG Methods Considered – Net Effects  Overview of method – Net effects = Difference between baseline sales and total CFL sales – Baseline sales = sales that would have occurred in absence of the program (typically estimated on a per capita basis for a similar state or region that does not have a program) – Total CFL sales – often difficult to estimate – NTGR = ratio of program sales to net effects – This method has been used recently in the Northeast and Wisconsin

6 Net Effects Method  Caveats: – Difficult if not impossible to reliably estimate total CFL sales for California – Baseline sales estimates for CA will be overstated (net effects/NTGR will be understated) because any region or state without a program will have been influenced by CA’s programs – Method will implicitly capture cumulative effects of prior CA programs (reflected in total CFL sales) leading to overstatement of net effects/NTGR – Results include both free-ridership and spillover

7 Net Effects Method  Method may be useful: – In regions where programs are small-scale, such that the national market is not effected (so baseline estimates are not biased) – Where total CFL sales may be reliably estimated – During the first year or two of a program, where cumulative effects are not a significant issue – In contexts where it is acceptable to use a NTGR that includes both free-ridership and spillover

8 NTG Methods Considered – Regional Regressions  Overview of method: – Model CFL sales (or market share/penetration) for various states/regions across the U.S. including California – Use the parameter estimates to estimate CFL sales in California in absence of the program – Used in the Northeast to estimate program attribution of ENERGY STAR appliance rebates – Incorporates both free-ridership and spillover

9 Regional Regressions  Caveats: – Significant CFL data quality and availability issues across the U.S. – California programs have influenced CFL sales/market shares across the U.S. leading to an understatement of the program’s influence in California (like the net effects method) – Not enough observations (obs=states/regions) since few states/regions collect and/or estimate CFL sales

10 Regional Regressions  Method may be useful: – For CFLs if CFL sales data become more available across the U.S. – For other products

11 NTG Methods Considered – Participant Self-Report  Overview of method: – Surveying participants (retailers and manufacturers in this case) regarding the influence of the program on their behavior – May address free-ridership and spillover separately – Used for NYSERDA Upstream Motors program

12 Participant Self-Report  Caveats: – A handful of lighting suppliers handle the majority of CFL sales in the state – Key decision-makers are often hard to contact and may be unwilling to provide information they consider proprietary

13 Participant Self-Report – Selected Method  Algorithm used for 1 of 3 CF product categories

14 Participant Self-Report – Selected Method  Algorithm used to combine results

15 Participant Self-Report  Keys to success: – To set up analysis framework:  Anticipate how program influence will likely vary by CF product and market actor type  Obtain detailed program tracking data – To ensure representative sample:  Obtain current contact information for participating suppliers  Have program managers encourage participants to respond to evaluators  Identify and focus on “must-have” interviews (based on % of rebates)

16 Participant Self-Report  Keys to success, continued: – To address possible bias:  Sought FR estimates from both manufacturers and retailers (mostly buydown – retailers did not receive incentives)  Assessed consistency of FR estimates across and within various categories

Experience you can trust. End sheet Thank you for your attention.