Formation of a Market Analysis Subcommittee Charlie Grist and Jennifer Light RTF Policy Advisory Committee February 19, 2016

Presentation Overview Key Objectives: Get a head nod from the PAC on the value of the RTF developing a Market Analysis Subcommittee Leave with understanding of potential resource implications and outstanding questions to be addressed by staff and PAC through initial exploration phase of this work What we will Cover: Value of market analysis research for the region Genesis of RTF role in this work Anticipated scope of review and resource implications Outstanding questions 2

Markets are Changing, both with and without Program Activity Programs are not the sole driver of efficiency adoption Product turnover cycles are adoption opportunities Understanding these market changes is becoming increasingly important to program planners and implementers – How much slower are the meters turning as a result of programs and changing markets? – In which markets are programs likely to have the biggest impact? – When can programs quit one market and move to the next? 3

What is Market Analysis Research? 4 Market Analysis Research Mix of Efficient & Inefficient Product Sales Identify Delivery Channels & Decision Events Impact of Efficiency Standards & Codes Track Changes Over Time

Example: Conversion From T12 to T8 Tubes 5 Only about 10% of efficient T5 & T8 tubes sold were reported in programs in 2010

Differential Uptake of LED 6

Council interest: Reliable estimate of progress against the Plan baseline Bonneville interest: Rigorous analysis and increased credibility RTF interest: Robust analysis and data inform measure baselines Why Bring this to the RTF? 7 Momentum savings can be big – important to get it right The underlying data collection and analysis is critical to load forecasting, conservation planning Program operation, and RTF measure development The RTF was set up to provide independent, technical expertise to the region on estimating energy savings

Role of an Independent Reviewers Method Review Support during the initial Planning phases of the project Review and inform analytical approach, including how to go about data collection (ex: where to go to collect data) and what data to collect May help to inform prioritization of projects Model Review Support throughout the In Flight phases of the project Review and inform draft models, including feedback on key inputs, errors, and other questions that come up model development Final Review Support in Finalizing the model Review and provide professional judgement on soundness of model results and identification of considerations for future research/continual improvement 8

Estimate of Resource Lift per Model Existing Model Review – Method review: 6 hr – Model review: 15 hr – Final review: 3 hr (8 hr for reviewer not following process) New Model Review – Method review: 12 hr – Model review: 21 hr – Final review: 3 hr (8 hr for reviewer not following process) Standards Review – Approximately 2/3 the lift of market model review 9 (ROUGH)

Anticipated Models/Standards 2016 Market Models Residential Lighting Non-Residential Lighting Residential HVAC Agriculture Standards Assume 6* 2017 Market Models Residential Lighting Non-Residential Lighting Residential HVAC Commercial HVAC (new) Lighting Controls (new) Standards Assume 6* 10 *Bonneville estimate of 12 standards over the next 2 years, see additional slides for potential standards included

Subcommittee Lift 2016 Hours Estimate Resource Lift (per person)Annual Percent of Annual Hours Primary Contract Analyst23811% Cross Cutting4 Existing Models132 Standards102 New Models0 Subcommittee Participate (full review)1969% Subcommittee Participant (partial review)1256% Additional RTF Staff Lift (remaining staff)2968% Cross Cutting16 Internal Team Review144 Model Review24 RTF Manager1125% 2017 Hours Estimate Resource Lift (per person)Annual Percent of Annual Hours Primary Contract Analyst29314% Cross Cutting0 Existing Models99 Standards102 New Models92 Subcommittee Participate (full review)24012% Subcommittee Participant (partial review)1578% Additional RTF Staff Lift (remaining staff)32911% Cross Cutting8 Internal Team Review136 Model Review60 RTF Manager1256% 11 (ROUGH) Assumptions Full review assumes review of all market models/standards Partial review assumes review of half the market models/standards plus 8 hours for judgment on other set of models/standards Primary contract analyst requires additional time for meeting prep Additional staff resources assumes another analyst review of 1 model in 2016 and 2 models in 2017 RTF Manager resource assumes meeting participation and monitoring of work

Detail by CategoryContract RFP Contract Analyst Team RTF Manager Subtotal Funders Sub- section % of total Regional Coordination (Research and Data Development) Regional Research Coordination $0$75,000$0$75,000 End Use Load Data Library Development and Maintenance $15,000$10,000$0$25,000 Coordination with Momentum Savings Efforts $0$30,000$0$30,000 Coordination with Power Plan $0$20,000$0$20,000 Subtotal Regional Coordination (Research and Data Development) $15,000$135,000$0$150,000 9% Total 2016 Work Plan$508,000$1,020,000$135,000$1,663,000100% 2016 Work Plan Already includes Some Support of Market Analysis Research 12 $30,000 represents: 3% of total contract analyst budget ~225 hours for a single contract analyst 11% of time for a single contract analyst 2%

How Does Work Plan Align with 2016 (ROUGH) Estimate? Resource Lift by PersonAnnual Percent of Annual Hours Primary Contract Analyst23811% Cross Cutting4 Existing Models132 Standards102 Subcommittee Participate (full review)1969% Subcommittee Participant (partial review)1256% Additional RTF Staff Lift2968% Cross Cutting16 Internal Team Review144 Model Review24 RTF Manager1125% 13 This aligns with the current work plan estimate This requires minimal shift in focus as internal team review would take place during regularly scheduled meetings RTF Member Resources Challenge: Currently assume hrs for member contribution This would double member lift RTF does provide minimal support for some members, but this category in work plan is already 94% allocated Note: The full RTF Work Plan is only 82% allocated as of 2/11/16 There is flexibility in work plan to support some additional subcommittee time (either “supported” members or contractors)

Blue Ribbon Panel Suggestion Given the lift for review, there was a suggestion to pay independent consultants to participate in subcommittee – Melding of a Blue Ribbon Panel with RTF Subcommittee Outstanding questions: – Who pays for these experts? (RTF may be able to support some experts) – What is the “job description” for these experts? – What is the selection criteria and who selects/recruits these experts? – What is the relationship between these consultants and the RTF members? 14

PAC Discussion Does the PAC agree there is value in the RTF informing and supporting regional market analysis research? Does the PAC support staff in exploring ways to allocate more of the RTF budget to formalizing a subcommittee process? – This would include support of non-contract analyst participation on the subcommittee As the RTF explores/finalizes a process for market analysis review, does the PAC want to provide guidance on how broadly/narrowly we scope the efforts? – Should review be limited to only markets related to RTF measures? – Should staff not exceed a designated support level (to be defined) in this initial stage? – Are there aspects of the review that should not be included in the subcommittee review? 15

Additional Slides 16

Federal Standards 17