1. Electronic TRM Proposal STAKEHOLDER PRESENTATIONS FALL 2015

2. Executive Summary Fall 2015 Introduction to the Cal TF EE Measure Development, Updating and Maintenance: Status Quo vs. Policy Goals  Compared to longstanding CPUC goals  Compared to state policy goals for EE  Compared to EPA Clean Power Plan (CPP) draft guidance for counting EE for CPP compliance Technical Reference Manual Research and Best Practices Path Forward  Electronic TRM  Proposed implementation process 2 Electronic TRM Proposal

3. The California Technical Forum (Cal TF) What is the Technical Forum? A group of in-state and out-of-state technical experts that work in a collaborative and transparent way to review new and updated energy efficiency measures and other technical information related to the integrated demand-side management portfolio. Fall 2015 Electronic TRM Proposal 3

4. Cal TF: A Broad Collaborative Fall 2015 Electronic TRM Proposal CPUC Office of Ratepayer Advocates 4

5. The Collaborative Supports the Technical Experts Fall 2015 Electronic TRM Proposal Technical Forum Cal TF Administrator Sponsoring Entity PAC IOUs POUs Others Funding Entities Peer reviews, comments Scopes work, monitors Facilitates, supports Administrator Contract Co-funding Agreements (Future Vision) 5

6. Technical Forum Members Selected through a competitive RFQ process with a ~50% selection rate.  CPUC provided input on selection criteria 2015 TF Composition:  35 members  450+ years combined industry experience  Technical expertise in wide range of energy efficiency technologies  30% from outside of California Fall 2015 Electronic TRM Proposal 6

7. EE Measure Development: Commission Goals Compared to Status Quo Fall 2015 Collaborative (D13-09-023 at 56)  Only utilities and CPUC staff involved in measure development  Process is adversarial rather than collaborative Transparent and Well-Documented (D10-04-029 at 30 and D01-11-06 at 20)  Extensive Cal TF staff DEER documentation work has yielded very little clear documentation  Most DEER measure parameters cannot be traced to sources; DEER measures parameters generally not reproducible Uses Best Available Information (D11-07-030 at 8)  CPUC measure approval often delayed until more data is collected Balances Accuracy, Precision, Timeliness, Cost, and Certainty (D11-07-030 at 8)  DEER is very complex - Leads to false precision  New measure development is lengthy and expensive  Measure values are not fixed from cycle to cycle – Over a dozen changes to DEER this year alone 7 Electronic TRM Proposal

8. EE Measure Development: State Policy Goals Compared to Status Quo Fall 2015 Use credible, statewide consistent values for forecasting and planning  POUs were unable to continue using DEER  Too complex, opaque, hard-to-use and understand Increase inter-agency and regional coordination  CEC and CPUC use different modeling tools to calculate energy savings.  CEC uses EnergyPlus for Title 24 compliance and CPUC uses DOE-2.2 for developing DEER energy savings values Energy efficiency as a resource  Efficiency can’t be a credible resource if IOUs and POUs use different approaches and values for calculating savings from energy efficiency Double energy efficiency savings by 2030  Protracted review and approval of new/updated measures (e.g. LEDs) will make state goals for energy efficiency more difficult to achieve 8 Electronic TRM Proposal

9. Fall 2015 On August 3, 2015, the US EPA released draft guidelines to ensure EE measures are “quantifiable and verifiable” for Clean Power Plan compliance purposes. Status quo does not conform to EPA draft guidance for EE:  DEER measure parameters are not transparent or well- documented; Workpapers are not publically available  2.4.2. Applicable Guidance: “Based on measure definitions, applicable conditions, assumptions, calculations, and references that are well documented in work papers that are publically available.”  California workpaper developers use conservative values to speed workpaper approval by CPUC staff  2.6.2 Applicable Guidance: “Should be designed neither to provide optimistic savings estimates… nor to provide conservative estimates.”  The system is not collaborative  2.8.2. Applicable Guidance: “Participate in collaborative and joint research to improve breath and quality.” EE Measure Development: Draft US EPA Clean Power Plan Guidance Compared to Status Quo 9 Electronic TRM Proposal

10. Technical Reference Manual Research Fall 2015 Technical Reference Manuals (TRMs) are the functional equivalent of DEER in other jurisdictions; they contain EE measures, measure parameters and measure documentation. Reviewed over 20 TRMs from jurisdictions across the country to identify best practices for measure development and measure repositories Interviewed developers and users in with strong TRMs  Massachusetts  New York  Pennsylvania and Mid-Atlantic  Illinois  Texas Reviewed prior literature/analysis on TRMs  Most analyses are about 5 years old  TRMs have evolved considerably since then Identified best practices for:  Process – process for developing and updating EE measures  Structure – structure for maintaining measures and associated documentation  Content – technical guidelines, directives and practices for developing/updating EE measures 10 Electronic TRM Proposal

11. TRM Best Practices – Process Fall 2015 Technical collaboratives open to the public Predictable and regular update processes  Existing measures must be updated regularly Participation by regulatory staff is key  Speeds issue resolution  Speeds regulatory review  Fosters technical understanding between regulators and other stakeholders  Builds regulator confidence in results Results of collaborative consensus-building process generally adopted by decision makers with little change  Regulators maintain final approval authority but can depend on robust process and results to inform decision-making. Regulatory Commissions, not staff, approve final values 11 Electronic TRM Proposal

12. TRM Best Practices – Structure Fall 2015 Standard format for each measure characterization, including:  Narrative explanation of measure  Base and measure case technical specifications  Energy and demand savings algorithms  Other key parameters (measure life, costs, etc.)  Pertinent implementation details (e.g. exclusions) All measure parameters clearly linked to measure Measure is well-documented and values are reproducible  Citations to primary sources, not other TRMs  Primary sources maintained and readily available Embedded calculators and look up tables Non-measure specific tools and information included as appendices  NY TRM has excellent descriptions of building prototypes 12 Electronic TRM Proposal

13. TRM Best Practices – Content Fall 2015 Written guidelines for addressing recurring technical issues  NW RTF and Mid-Atlantic state use process language  Use of reproducible methods, diligent review of all sources…  PA and IL use more specific data hierarchies  NW RTF has several guidelines on measure complexity, statistical significance, other Careful consideration of modeling vs. engineering equations vs. field data  No “one size fits all”; consider pros and cons of different approaches Key parameters (from modeling or engineering equations) should be validated with real data!  Field conditions and human behavior may alter forecasted savings  Collect data through implementation or early EM&V  Identify and implement use of AMI data (e.g. EnergySavvy) and other tools (DOE Building Performance Database) 13 Electronic TRM Proposal

14. DEER Alternative: Statewide Electronic TRM – Key Features Fall 2015 All measures are fully documented with a workpaper  All measure parameters are clearly documented and linked  Source documents are clearly cited and hosted in the tool Modeled values are linked to models  Uses EnergyPlus for measures that should be modeled Automates measure updates when inputs change—weather files, code updates, etc.  Also, includes clear update and revision histories for each measure Clear and documented workflow management  Identifies who has updated and/or reviewed a measure Generates all key outputs for CPUC ex ante database, utility and CPUC cost-effectiveness analysis and reporting, CEC forecasting Keyword searchable Permits restricted access to protect confidential information Open source repository for EE measures 14 Electronic TRM Proposal

15. Fall 2015 Proposed Implementation Process California Ex Ante Measures Approximately 187 California Ex Ante Measures Approximately 187 Overlapping Measures (DEER, non-DEER, POU) 36 Overlapping Measures (DEER, non-DEER, POU) 36 DEER Measures (No WP, POU Overlap) 17 DEER Measures (No WP, POU Overlap) 17 IOU Non-DEER WPs; includes consolidating common IOU measures (No DEER Overlap ) 125 IOU Non-DEER WPs; includes consolidating common IOU measures (No DEER Overlap ) 125 POU TRM Measures (No DEER/WP Overlap ) 9 POU TRM Measures (No DEER/WP Overlap ) 9 Create WP Create Single WP per Measure Detailed Cal TF Review – Specialized Subcommittees Detailed Cal TF Review – Specialized Subcommittees Full TF Review and Approval Full TF Review and Approval Populate Statewide Electronic TRM Electronic TRM Proposal

16. Conclusion: Benefits of the Electronic TRM Fall 2015 Electronic TRM Proposal 16 Rigorous, well-documented EE measures Common repository for all California measures Statewide consistent energy savings values Collaborative, consensus-based, public and inclusive process Use of open source electronic TRM and tools

17. Appendix Fall 2015 Electronic TRM Proposal 17 The move to “Electronic TRMs” Status Quo vs. Alternative Comparison EnergyPlus and Open Studio:  Beyond Single Point Analysis  Comparison Against Multiple Baselines

18. Appendix: The Move to Electronic TRMs Fall 2015 Key benefits to be gained:  Improves documentation  Ability to embed tools and supporting documents  Enables more detailed revision histories  Reduces cost and increases efficiency of data management  Through APIs, automatic download of values into utility tracking and planning databases  Highly transparent workflow management for new and updated measures.  Keyword searchable Available tools include:  VEIC  Nexant iEnergy  Energy Platforms  Frontier  U.S. Department of Energy Platforms 18 Electronic TRM Proposal

19. Status Quo Cal TF Alternative Fall 2015 Electronic TRM Proposal Rolling Portfolio mechanics needed to accommodate two sets of yearly updates  Q2-Q3: CPUC Staff-led DEER update  Q4: IOU workpaper updates  Q1: CPUC Staff review of updated workpapers Only one repository to update and adopt Reduced time and resources spent in translation—DEER to workpapers and back to DEER  Less room for computational errors  i.e. Freezer savings values for ARP  Reduce duplication of values 19 Appendix: Repository Updates and Management

20. Fall 2015 Electronic TRM Proposal New Measure Development and Existing Measure Updates Two Stages of Peer Review TRM Finalized and Submitted to Commission TRM Update Bus Stop 9/1 TRM Update Bus Stop 9/1 TRM Plan Development and Updates Two Stages of Peer Review Legend External Contractors, PA Staff TF peer review with participation from CPUC Staff Cal TF staff Final Commission approval Current Rolling Portfolio Two-Repository Update Process Proposed Electronic TRM Single Repository Update Process

21. Status Quo Cal TF Alternative Fall 2015 Electronic TRM Proposal Commissioners were forced to review individual savings parameters for ARP Struggles to determine right amount of uncertainty delayed important decision by 6 weeks Conclusions were limited by available data and need to settle on single point value Open Studio and EnergyPlus enable analysis that considers uncertainty in base cases  Parametric analyses can quantify uncertainty to enable more data- driven decisions Commission would be presented with replicable consensus recommendations  Supporting documentation in an accessible format for in-depth review if needed (in cases of non- consensus items) 21 Appendix: Savings Estimate Development and Updating

22. Status Quo Cal TF Alternative Fall 2015 Electronic TRM Proposal No single forum for open discussion of complex technical issues  i.e. CPUC staff review of Advanced Power Strips was forced to re-open issues already discussed by TF months before  Issues could have been settled in half the time if discussed in a single forum One forum for peer review, discussion, and guidance from CPUC Staff and consultants Open, transparent process  Participation by broad range of stakeholders  Comparison Exhibit for disputed values memorializing disputes and rationale for each position, facilitating CPUC analysis and resolution of disputed issues  All documentation available to all participants, public 22 Appendix: Measure Review

23. 23 EnergyPlus and Open Studio: Beyond Single Point Analysis 0.200.250.300.350.40 Window-to-Wall Ratio Probability Density Function (PDF) Infiltration % Change PDF -40-2020040 OpenStudio and EnergyPlus enable analysis that considers uncertainty in the baseline building. This illustrative example is built with DOE’s medium office prototype building model and two typical sources of variability. Red lines denote “nominal” (DEER- like) model results absent variability

24. 24 0.200.250.300.350.40 Window-to-Wall Ratio Probability Density Function (PDF) Infiltration % Change PDF -40-2020040 Pre-1980 Baseline PDF 555560565570 EUI (MJ/m 2 ) Scalable use of commodity cloud computing enables rapid creation of a family of baseline buildings that includes uncertainty. EnergyPlus and Open Studio: Beyond Single Point Analysis

25. 25 Energy Recovery Ventilation Measure + Solid State Lighting Measure + 0.200.250.300.350.40 Window-to-Wall Ratio Probability Density Function (PDF) Infiltration % Change PDF -40-2020040 Pre-1980 Baseline PDF 555560565570 EUI (MJ/m 2 ) 440460500560 EUI (MJ/m 2 ) 480520540 Potential analysis now includes uncertainty in estimated savings that can aid in deemed/custom determinations. EnergyPlus and Open Studio: Beyond Single Point Analysis

26. 26 Pre-1980 Baseline PDF 90.1-2004 Baseline PDF 90.1-2010 Baseline PDF 555560565570 440460480 339341342340 EUI (MJ/m 2 ) Pre-1980 90.1-2004 90.1-2010 440460500560 EUI (MJ/m 2 ) 480520540 440460 EUI (MJ/m 2 ) 480 380400420 325330 EUI (MJ/m 2 ) 335 310315320 340 Baseline Baseline + Energy Recovery Ventilation Baseline + Solid State Lighting The system also enables easy comparison of savings predictions relative to a range of baselines. EnergyPlus and Open Studio: Comparison Against Multiple Baselines