DNV GL © 2014 Ungraded Wednesday, March 9, 2016 SAFER, SMARTER, GREENER DNV GL © 2014 Impact Evaluation of Upstream and Residential Downstream Lighting Programs 1 ED_I_LTG_4 California Public Utilities Commission, Energy Division March 9, 2016

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Table of Contents  Introduction  Measure Quantity Adjustments  Gross Savings  NTGR and Net Savings  Alternate Gross Savings and NTGR  Recommendations 2

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Introduction 3

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Ex Ante Savings Claims 4 Ex ante net annual savings from upstream and residential downstream lighting measures, IOU Total PortfolioUpstream/ Residential Downstream Lighting Energy (GWh) Peak Demand Reduction (MW) Energy (GWh) Peak Demand Reduction (MW) Energy (GWh) Peak Demand Reduction (MW) PG&E1, %7% SCE1, %16% SDG&E %11% Statewide3, %12% Source: Tracking Data Introduction  9%-18% of each IOU’s ex ante net annual energy savings were from upstream/residential downstream lighting measures

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Ex Ante Savings Claims 5 Source: Tracking Data Statewide ex ante upstream and residential downstream lighting net savings, IOU / Lighting Measure Category Ex Ante Upstream and Residential Downstream Lighting Savings EnergyDemandGas Impacts GWh % of GWh MW % of MW Therms % of Therms Upstream - Evaluated %66.196%-7.095% Upstream - Passed Through12.93%1.83%-0.23% Downstream - Passed Through9.22%1.12%-0.12% Grand Total – All IOUs % % % Introduction  Evaluation focused on upstream measures, which accounted for 96% of energy and demand savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Program Overview 6 Share of upstream lighting program CFL shipments by retail channel and program period, and Source: Tracking Data Introduction  The program was smaller than  LED lamp share of program increased, while basic spiral CFL share decreased

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Program Overview 7 Evaluated Upstream Lighting Measure Group Average Incentive Amount Overall PG&ESCESDG&E MSB CFL basic spiral ≤ 30 W$0.34$0.48$0.61$0.49 MSB CFL A-lamp ≤ 30 W$2.05$2.32$1.20$2.08 MSB CFL reflector ≤ 30 W$1.45$2.52$1.43$2.14 MSB CFL globe ≤ 30 WN/A$2.77$1.47$2.53 MSB CFL high-wattage (> 30 W)$2.65$2.92$1.13$2.89 LED A-lamp, all wattages$6.16$8.59$5.00$6.59 LED reflector, all wattages$3.56$6.04$3.47$4.75 Overall$2.04$3.70$1.19$2.41 Source: Tracking Data Average incentive amount per lamp by evaluated upstream lighting measure group and IOU, Introduction  Basic spiral discounts were relatively low, LED lamp discounts were relatively high

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Program Overview 8 Quarterly lamp shipments to home improvement stores by upstream lighting program measure group, Source: Tracking Data Introduction  Availability of program-discounted lamps varied throughout the program period Example

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Market Context 9 Lamp Replacement Category Measure Group IOU Discounted Retail Price A-lamp Replacements MSB CFL basic spiral ≤ 30 W$0.59$3.27 MSB CFL A-lamp ≤ 30 W$0.93$5.45 Incandescent A-lampN/A$1.90 Halogen A-lampN/A$2.14 LED A-lamp, all wattages$6.92$11.16 Reflector Lamp Replacements MSB CFL reflector ≤ 30 W$1.77$5.79 Incandescent reflectorN/A$4.59 LED reflector, all wattages$7.84$18.70 Globe Lamp Replacements MSB CFL globe ≤ 30 W$1.32$5.50 Incandescent globeN/A$2.36 Source: bulbstockdata.com Average IOU-discounted and non- IOU-discounted price per lamp across retail stores by replacement lamp category, winter Introduction  In the presence of the program, CFLs cost the least  Without the program, inefficient lamps cost the least

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Market Context 10 Source: bulbstockdata.com Share of lamps in California retail stores by technology, winter vs. winter Introduction  Share of efficient/inefficient stock was consistent  Within efficient, LED share grew and CFLs dropped  Within inefficient, halogen share grew

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Measure Quantity Adjustments 11

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Three Adjustments  evaluation relied on values 12 Adjustment MethodValue Shipment quantity verification Review lamp manufacturer shipment invoices 100% Res/Non-Res Split % of program lamps purchased by res vs. nonres customers Review results from CMST and CLASS IOU Non Res PG&E 7%93% SCE 6%94% SDG&E 6%94% Leakage % of program lamps purchased by non-IOU customers Review results from shopper intercept surveys 0% Quantity Adjustments

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Gross Savings 13

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Results 14 Statewide Ex ante and ex post gross savings and realization rates,  Gross realization rates are mostly >100% Gross Savings Measure Group Ex AnteEx Post Realization Rates GWhMW Therms (10,000) GWhMW Therms (10,000) GWhMWTherms MSB CFL basic spiral ≤ 30 W %170%167% MSB CFL A-lamp ≤ 30 W % 141% MSB CFL reflector ≤ 30 W %132%128% MSB CFL globe ≤ 30 W %105%81% MSB CFL high-Watt (> 30 W) %104%106% LED A-lamp, all wattages % 158% LED reflector, all-wattages %141%160% Overall %125%128%

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Results 15 Statewide Ex ante and ex post gross savings and realization rates,  Gross realization rates are mostly >100% Gross Savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Results  Gross realization rates are high because three key parameter values are higher in ex post than ex ante: 1.Delta watts 2.Installation rates 3.Nonresidential allocation in res/non-res split for many measure groups* * Exception: SCE LED lamps 16 Gross Savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Delta Watts  Ex Post calculated higher delta watts than ex ante  Lamps in the program had fairly low wattage  Ex ante approach – Used wattage-reduction ratio for delta watts, which yields lower deltas for lower-wattage lamps  Ex post approach – Subtracted average program lamp wattages from average baseline, which yields higher deltas for lower-wattage lamps – baselines: – CFL baseline = incandescent lamps – LED baseline = CFLs + incandescent lamps 17 Gross Savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Installation Rates  Ex post used higher installation rates than ex ante  The definition changed to include all lamps that are or will eventually be installed, eliminating the need for carryover going forward 18 SDG&E ex ante and ex post res and non-res installation rates, Gross Savings Evaluated Upstream Lighting Measure Group ResidentialNonresidential Ex AnteEx PostEx AnteEx Post MSB CFL basic spiral ≤ 30 W76%97%76%97% MSB CFL A-lamp ≤ 30 W73%97%71%97% MSB CFL globe ≤ 30 W77%97%78%97% MSB CFL reflector ≤ 30 W76%97%74%97% MSB CFL high-wattage (> 30 W)76%97%79%97% LED A-lamp, all wattages100%99%100%99% LED reflector, all wattages100%99%100%99% Example

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Res/Non-Res Split  Ex post generally used higher nonresidential allocation compared to ex ante, which increases ex post gross energy savings* 19 Gross Savings IOU Ex AnteEx Post ResidentialNonresidentialResidentialNonresidential PG&E94%6%93%7% SCE99%1%94%6% SDG&E95%5%94%6% Example Basic spiral CFL ≤ 30 W ex ante and ex post res/non-res split * Exception: SCE LED lamps

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 NTGR and Net Savings 20

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Results 21 Statewide net savings realization rates, NTGR & Net Savings  Net savings realization rates are mostly between 60% and 100%

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Results 22 Statewide ex ante and ex post net savings and realization rates, NTGR & Net Savings  Net savings realization rates are mostly between 60% and 100% Measure Group Ex AnteEx Post Realization Rates GWhMW Therms (10,000) GWhMW Therms (10,000) GWhMWTherms MSB CFL basic spiral ≤ 30 W %78%75% MSB CFL A-lamp ≤ 30 W % 164% MSB CFL reflector ≤ 30 W %60%57% MSB CFL globe ≤ 30 W %133%103% MSB CFL high-Watt (> 30 W) %86%88% LED A-lamp, all wattages %82%80% LED reflector, all-wattages %52%59% Overall %89%88%

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Overview of Approach 23 NTGR & Net Savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Model-Based NTGR 24 NTGR & Net Savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Model-Based NTGR 25 NTGR & Net Savings  Lamp choice model predicts the probability with which a consumer would purchase different lamps within a replacement lamp category  Model-based NTGR is the difference in the measure group’s sales share with the program versus without the program as a percent of the with-program share

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Model-Based NTGR 26 NTGR & Net Savings Modelled share of sales for MSB CFL A-Lamp ≤ 30 W, discount channel Source: DNV GL Lamp Choice Model Example

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Model-Based NTGR 27 NTGR & Net Savings Lamp Replacement Category Measure Group A-lamp Replacements MSB CFL basic spiral ≤ 30 W MSB CFL A-lamp ≤ 30 W Incandescent A-lamp Halogen A-lamp LED A-lamp, all wattages Reflector Lamp Replacements MSB CFL reflector ≤ 30 W Incandescent reflector LED reflector, all wattages Globe Lamp Replacements MSB CFL globe ≤ 30 W Incandescent globe Three modelled replacement lamp categories

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Model-Based NTGR 28 NTGR & Net Savings  Additional model considerations – Scenarios with program – No discount (no program) – Constrained (no program) – Program activity type – Separately model choice sets with different combinations of discounted measure groups present – Weight by program shipments – Respondent counts – Rigorous data is ≥ 15 shopper intercept survey respondents

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Overview of Approach 29 NTGR & Net Savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Supplier-Based NTGR 30 NTGR & Net Savings  Two supplier sources: – Manufacturers – 16 participating lighting manufacturers – Represented ~99% of program-discounted lamps – Retail lighting buyers – 6 lighting buyers for large retail chains – Represented ~30% of program-discounted lamps  NTGR based on this question: – By what percentage would your sales have changed in absence of the program?

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Weighting of Results 31 NTGR & Net Savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Weighting of Results 32 NTGR & Net Savings  Weighted Model and Supplier NTGR – Model-based NTGR (70%) – Actual shopper insights that reflect customer characteristics – Simulations based on observed in-store prices and lamp availability – Used only with robust intercept survey sample sizes (n≥ 15) – Supplier-based NTGR (30%) – Comprehensive coverage of the lighting market – Established survey methodology used in California and elsewhere

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Imputation 33 NTGR & Net Savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Imputation 34 NTGR & Net Savings  Imputation factor – Average final NTGR divided by average supplier NTGR – Based on robust cases – Calculated for every measure group – When model data was insufficient (n < 15), used an imputation factor from a similar measure group  Impute from supplier NTGR – Supplier NTGR for each retail channel multiplied by the respective imputation factor

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Final Measure Group NTGR 35 NTGR & Net Savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Final Measure Group NTGR 36 NTGR & Net Savings  Final measure group NTGR is the weighted average of the channel-level NTGRs  Weights are based on the lamp quantities shipped to each respective channel

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Ex Post Net Savings 37 NTGR & Net Savings Statewide ex ante and ex post net savings and realization rates,  Net savings realization rates are between 60% and 100% Measure Group Ex AnteEx Post Realization Rates GWhMW Therms (10,000) GWhMW Therms (10,000)GWhMWTherms MSB CFL basic spiral ≤ 30 W %78%75% MSB CFL A-lamp ≤ 30 W % 164% MSB CFL reflector ≤ 30 W %60%57% MSB CFL globe ≤ 30 W %133%103% MSB CFL high-Watt (> 30 W) %86%88% LED A-lamp, all wattages %82%80% LED reflector, all-wattages %52%59% Overall %89%88%

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Demonstration of Approach 38 Final IOU-level NTGR for MSB CFL basic spiral ≤ 30 W, weighted by channel, Channel Channel NTGR Shipments Total Program Lamps PG&ESCESDG&E Discount66%250,335061,952188,383 Drug Store35%38,2022,61234,2401,350 Grocery - Chain7%16, ,852 Grocery - Independent55%2, Hardware26%86,21686, Home Improvement22%2,013,099541,868586,862884,369 Mass Merchandise33%406, ,482 Membership Club15%751,543410,71210,638330,193 Total Shipments 3,564,4931,042,169696,6961,825,628 Overall NTGR 20%26%27% NTGR & Net Savings Example  MSB CFL basic spiral ≤ 30 W

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 MSB CFL basic spiral ≤ 30 W 39 IOUAverage Discount PG&E $0.34. SCE $0.48. SDG&E $0.61. Overall $0.49. Source: Tracking Data Timing of program lamp shipments, Channel distribution of program lamp shipments, Average discount by IOU, NTGR & Net Savings N=3,763,679

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 MSB CFL basic spiral ≤ 30 W 40 Modelled NTGR by channel NTGR & Net Savings Channel Modelled NTGR Discount57% Drug* Grocery - Chain* Grocery - Independent* Hardware14% Home Improvement8% Mass Merchandise* Membership Club5%

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 MSB CFL basic spiral ≤ 30 W 41 Modelled and supplier NTGR by channel NTGR & Net Savings Channel Modelled NTGR Supplier NTGR Discount57%84% Drug*63% Grocery - Chain*12% Grocery - Independent*100% Hardware14%54% Home Improvement8%53% Mass Merchandise*59% Membership Club5%41%

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 MSB CFL basic spiral ≤ 30 W 42 Modelled, supplier, and blended NTGR by channel NTGR & Net Savings Channel Modelled NTGR Supplier NTGR Model + Supplier NTGR Discount57%84%66% Drug*63%* Grocery - Chain*12%* Grocery - Independent*100%* Hardware14%54%26% Home Improvement8%53%22% Mass Merchandise*59%* Membership Club5%41%15%

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 MSB CFL basic spiral ≤ 30 W 43 Modelled, supplier, and blended NTGR by channel and imputation factor NTGR & Net Savings Channel Modelled NTGR Supplier NTGR Model + Supplier NTGR Imputation Factor Discount57%84%66% 0.55 Drug*63%* Grocery - Chain*12%* Grocery - Independent*100%* Hardware14%54%26% Home Improvement8%53%22% Mass Merchandise*59%* Membership Club5%41%15%

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 MSB CFL basic spiral ≤ 30 W 44 Channel Modelled NTGR Supplier NTGR Model + Supplier NTGR Imputation Factor Final NTGR Discount57%84%66% % Drug*63%*35% Grocery - Chain*12%*7% Grocery - Independent*100%*55% Hardware14%54%26% Home Improvement8%53%22% Mass Merchandise*59%*33% Membership Club5%41%15% Modelled, supplier, blended, and imputed NTGR by channel NTGR & Net Savings

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 MSB CFL basic spiral ≤ 30 W 45 Modelled, supplier, blended, and final NTGR by channel NTGR & Net Savings Channel Modelled NTGR Supplier NTGR Model + Supplier NTGR Imputation Factor Final NTGR Discount57%84%66% % Drug*63%*35% Grocery - Chain*12%*7% Grocery - Independent*100%*55% Hardware14%54%26% Home Improvement8%53%22% Mass Merchandise*59%*33% Membership Club5%41%15%

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Demonstration of Approach 46 Final IOU-level NTGR for MSB CFL basic spiral ≤ 30 W, weighted by channel, Channel Channel NTGR Shipments Total Program Lamps PG&ESCESDG&E Discount66%250,335061,952188,383 Drug Store35%38,2022,61234,2401,350 Grocery - Chain7%16, ,852 Grocery - Independent55%2, Hardware26%86,21686, Home Improvement22%2,013,099541,868586,862884,369 Mass Merchandise33%406, ,482 Membership Club15%751,543410,71210,638330,193 Total Shipments 3,564,4931,042,169696,6961,825,628 Overall NTGR 20%26%27% NTGR & Net Savings Example

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Alternate Gross Savings and NTGR 47

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Alternate Gross Savings 48 Alternate Gross Savings and NTGR

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Alternate Gross Savings 49 Alternate Gross Savings and NTGR  Delta Watts baseline differences – CFL, conventional: installed incandescent – CFL, alternate: installed mix of incandescent lamps and CFLs – LED, conventional: installed mix of incandescent lamps and CFLs (no alternate necessary)

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Recommendations 50

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Recommendations  Refine targeting for LED lamp incentives – Big box NTGR are relative low for most measure groups – The presence of LEDs in these channels has increased rapidly while pricing has declined at the market level – Review the cost-effectiveness of offering incentives for LED lamps in big box channels and (if not cost- effective) consider directing incentives toward non- big box channels 51 Recommendations

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Recommendations  Refine targeting for CFL incentives – CFL NTGR are somewhat lower than in the prior evaluation, but still potentially cost-effective – Examine the cost effectiveness of different measure groups in each retail channel 52 Recommendations

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Recommendations  Examine projections of lamp pricing and market conditions – Review baselines – Explore the effectiveness of offering discounts on competing technologies 53 Recommendations

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 Thank You!  Jenna Canseco  Andrew Stryker  Stephanie Whalley  Chris Dyson  Anton Zyarko  Claire Palmgren  Mimi Goldberg  Jeorge Tagnipes  Ralph Prahl  Many others 54

DNV GL © 2014 Ungraded Wednesday, March 9, 2016 SAFER, SMARTER, GREENER Thank you for your attention 55 Jon TaffelJenna Canseco DNV GL (Oakland, CA)DNV GL (Oakland, CA) ConsultantHead of Section, Markets—West x x44121