California Energy Commission Presentation of Proposed Grouping of 2006 Energy Efficiency programs for Evaluation Planning Mike Messenger CEC Evaluation Staff July 20, 2005

California Energy Commission Problem Statement- How to group programs across utility service areas and program strategies to facilitate timely completion of Process, Impact and Market assessment within a 3 year cycle and staffing constraints

California Energy Commission Criteria for Grouping/Scheduling Be able to estimate/validate performance basis for major programs for each area Fine enough detail to provide useful process insights to improve strategies Reduce number of evaluation contracts Allow for evaluation of the effectiveness of marketing and potential measurement of spillover impacts Stage process, impact and market assessment over three year cycles; don’t want all impact in first year for example

California Energy Commission Proposed Grouping Strategies Evaluated By Program Strategy By End use and Customer segment By Market Segment By Utility Service Area By Customer Segment Tentative Winner- Strategy and Major Market Segments

California Energy Commission Proposed Grouping by Strategy Strategy grouping Program Names Residential Audits, Small commercial direct install and Home Energy Ratings,Home Energy Surveys, Commercial Direct Install, Third party time of sale programs, feedback programs Residential and Small Commercial Rebates to Customers, Dealers or Upstream actors and financing Residential EE rebates, small commercial express rebates, cfl upstream rebates, other appliance upstream programs, PGE Mass Market Residential New Construction Rebates and design assistance Residential new construction, advanced homes, programs targetted to increase code compliance Residential comprehensive HVAC and DHW programs Comprehensive HVAC, Programs targeted to solely reduce critical peak, and future domestic hot water programs Residential marketing, training, and education programs to customers ( includes flex your power, utility inserts, kiosks, others) Flex your power, other statewide campaigns in multiple languages, local information campaigns, sales kiosk, 20/20 programs? Codes and Standards Advocacy

California Energy Commission Proposed Grouping by Strategy- Non Residential and Cross cutting Non Residential Audits and Benchmarking Non Residential Audits and any new Benchmarking programs, Envinta program,Retro commissioning, Integrated School based audits? Non Residential Prescriptive rebates- Express Efficiency Express Efficiency, Business Incentive, Upstream Motors and or HVAC system rebates Non Residential Custom rebates and Performance Contracting (includes agricultural and industrial process programs) Comprehensive HVAC, Agricultural and Industrial process, standard performance contracting Non Residential Savings by Design and New Construction Savings by Design Non Residential marketing, training, and education programs for architects, contracts and installers. ( includes Energy Centers) All Energy and technology Centers, all training programs targeted at building operating professionals Local government Partnerships and Sustainable Communities Emerging Technologies- Statewide evaluation Emerging Technologies and link to PIER and other research

California Energy Commission Proposed Evaluation Study Sequence by Strategy Grouping-Residential Residential Audits, Small commercial direct install and Home Energy Ratings Market Assessment- Res RetrofitImpactProcess Residential and Small Commercial Rebates to Customers, Dealers or Upstream actors and financing Market Assessment- Res RetrofitImpactProcess Residential New Construction Rebates and design assistance Process Market Assessment- New constructionImpact Residential comprehensive HVAC and DHW programs Process and some impact work Market Assessment- New constructionImpact Residential marketing, training, and education programs to customers ( includes flex your power, utility inserts, kiosks, others) Market Assessment- Res RetrofitProcess Impact- 3 year longitudinal study?

California Energy Commission Proposed Evaluation Study Sequence by Strategy Grouping-Non Residential Strategy Grouping Non Residential Audits and Benchmarking Impact Market Assessment- Non Residential RetrofitProcess Non Residential Prescriptive rebates- Express Efficiency Impact Market Assessment- Non Residential RetrofitProcess Non Residential Custom rebates for processes and equipment, Performance Contracting Impact Market Assessment- Non Residential RetrofitProcess Non Residential Savings by Design and New Construction Process for levels of compliance, new techniquesImpact Market Assessment- Non res New construction Non Residential marketing, training, and education programs for architects, contracts and installers. ( includes Energy Centers) Process Market Assessment- Non Residential RetrofitProcess

California Energy Commission Proposed Scheduling for Cross Cutting Programs Strategy Grouping Local government Partnerships and Sustainable Communities ProcessImpact if feasible Market Assessment- Local government programs Emerging Technologies- Statewide evaluation Market Assessment- effectiveness of program Process Impact?? Or more market assessment Codes and Standards Advocacy Process and some M & V- duct sealing Market Assessment- New constructionImpact

California Energy Commission Evaluation Objectives for Market Assessments ( 4 total) Provide an estimate of the total energy and peak savings achieved in this market over the last 2? Years for each of the four utility areas and a breakdown of the relative share of each of the major strategies operating in the market. ( include spillover?) Provide an assessment of the extent that these programs have caused changes in market structure and their sustainability

California Energy Commission Evaluation Objectives for Impact Assessments Estimate/Validate ex ante estimates of program strategy savings using protocols Provide insights about why there may be differences between ex ante and ex post values Estimate impacts by end use and measure if feasible

California Energy Commission Evaluation Objectives for Process Evaluations Evaluate effectiveness of marketing and program designs Evaluate if program theory is valid Provide suggestions for improvement both in program processes and methods to reach and motivate customers