1. EM&V Quarterly stakeholder meeting December 12, 2017
IOU EM&V Project and Budget Updates
Pacific Gas & Electric – Lucy Morris
San Diego Gas & Electric – Jesse Emge
Southern California Edison – Shahana Samiullah
Southern California Gas – Loan Nguyen

2. SoCalGas Non-Study Costs For EM&V Activities
SCG
2016
2017 Q1 Q2 Q3 Q4
Staff Labor ¹
$77,380
$56,641
$47,662
$64,773
$41518
$34,820
$16,337
EM&V Admin
$24,383
$21,341
$13,209
Travel/Training ¹
$4,583
$1,016
$2,443
$1,360
$455
$1,100
$1,503
Technical advisors
$8,184
$32,749
$37,952
$62,434
$20,433
$22,992
$35,483
Overhead ¹
$27,630
$85
$-110
$333
$-74 $0
$-112
Total
$117,778
$90,491
$87,947
$128,899
$86,715
$80,253
$71,633
¹ Annual costs can swing due to accruals and reversals. Quarterly expense for advisors can swing due to time of invoice is billed and paid.
Staff Labor
Travel/
Training
Technical advisors
Overhead
Salary, vacation and sick, and payroll taxes
Travel, conference registrations, training costs, membership
Technical advisors supporting study development or review, PCG2 participation, and RTRs
Exclude: Other overhead expenses such as incentive compensation, pension and benefits, worker’s comp, personal liability and property damage insurance, third- party general & administrative and fixed cost loaders are paid outside of EE budget through the GRC.
Include: Affiliate inter-utility billing.

3. PG&E Non-Study Costs For EM&V Activities
2016
2017 Q1 Q2 Q3 Q4
Staff Labor ¹
$479,533
$449,181
$348,080
$529,777
$415,802
$364,801
$352,704
EM&V Admin
$20,790
$15,626
$20,797
Travel/Training ¹
$4,050
$16,377
$8,431
$14,933
$1,464
$1,945
$8,082
Technical advisors
$52,444
$84,109
$54,176
$58,514
$14,552
$101,965
$33,569
Overhead ¹
$110,104
$153,134
$117,084
$179,581
$81,813
$72,329
$80,781
Total
$646,131
$702,801
$527,771
$782,805
$513,631
$541,040
$475,136
EM&V Admin cost estimated by calculating the % of total timesheet hours that are spent on EM&V Admin. That percentage (Q1: 5%; Q2: 4%; Q3: 6%) is then multiplied by the Total Staff Labor (including salary, benefits, vacation and sick time) cost.
¹ spend estimates calculated using activity percentages of bundled standard rate.
Staff Labor
Travel/
Training
Technical advisors
Overhead
Salary, benefits, vacation and sick time
Excludes: Pension (paid outside EE budget)
Travel, conference registrations, training costs
Technical advisors supporting study development or review, PCG2 participation, and RTRs
Includes: Payroll taxes; Office costs: IT-telecoms equipment and licenses, office supplies, IT-telecoms support (standing monthly charge), facilities charges (standing monthly charge); General charge-ins: Senior Management, legal, data security and analytics; Professional memberships and support (Calmac, EVO);

4. SDG&E Non-Study Costs For EM&V Activities
2016
2017 Q1 Q2 Q3 Q4
Staff Labor ¹
$91,555
$131,671
$174,051
$111,622
$61,637
$73,133
$69,355
EM&V Admin -
$25,717
$30,514
$28,937
Travel/Training ¹
$2,826
$1,624
$581
$2,039
$1,708
$3,604
Technical advisors
$21,675
$13,583
$18,186
$17,555
$13,081
$37,612
$51,436
Overhead ¹
 $372
$233
$150 
$460
$880
$797
$555
Total
$116,428
$146,887
$190,682
$131,676
$101,896
$143,763
$153,886
¹ Annual costs can swing due to accruals and reversals. Quarterly expense for advisors can swing due to time of invoice is billed and paid.
Staff Labor
Travel/
Training
Technical advisors
Overhead
Salary, vacation and sick, and payroll taxes.
Travel, conference registrations, training costs, membership
Technical advisors supporting study development or review, PCG2 participation, and RTRs
Exclude: Other overhead expenses such as incentive compensation, pension and benefits, worker’s comp, personal liability and property damage insurance, third-party general & administrative and fixed cost loaders are paid outside of EE budget through the GRC.
Include: Affiliate inter-utility billing.

5. SCE Non-Study Costs For EM&V Activities
2016
2017 Q1 Q2 Q3 Q4
Staff Labor ¹
$1,301,756
$483,913
$628,704
$751,785
$110,549
$133,260
$153,495
EM&V Admin
$30,953
$ 37,180
$ 42,825
Travel/Training ¹
$17,243
$6,327
$7,509
$9,598
$1,048
$ 5,489
$ 4,270
Technical advisors
$71,514
$9,224
$31,124
$16,399
$6,912
$ 5,926 $
Overhead ¹
$24,574
$4,676
$9,578
$18,505
$4,368
$ 7,291
$ 9,846
Total
$1,415,087
$504,140
$676,914
$796,287
$122,877
$151,966
$168,205
Staff Labor
Travel/
Training
Technical advisors
Overhead
Salary, vacation and sick time.
Excludes: Pension/Benefits (paid outside EE budget)
Travel, conference registrations, training costs
As needed technical advisors supporting PCG2 participation, and RTRs
Includes: IT costs (software, web/phone, remote access, teleconference). Office supplies, office equipment (computers, phones).

6. IOU Evaluation Update: Project Status from 2013 – December 2017
Includes studies from and cycles.
Note, numbers don’t reflect how many studies are in the pre-scoping phase (i.e. planned in the EM&V plan or as ad-hoc study need, but not at the stage of developing a full SOW); many more studies in pipeline.
Old Link to PSR is problematic; Best way to access is through CA EE Stats page -->Evaluation --> Evaluation Status and Budgets;

7. New Whitepapers
Final Whitepaper: Development of Order-Independent Waterfall Graphics to Enable Comprehensive Understanding of Impact Evaluation Results Three types of waterfall graphics: gross, net, and hybrid graphics were developed to serve the needs of all evaluation stakeholders. A methodology is presented to ensure that evaluation findings are normalized -- individual findings are neither over nor under emphasized. The resulting visuals offer a concise, complete, and accurate way to communicate evaluation findings, highlight program achievements, provide insights for program planning, and reveal opportunities for program improvement. The paper includes a link to a web-based tool to automatically generate customized waterfall graphics.
Observations on Chapter 8 of the Uniform Methods Project: A Discussion of Comparison Groups for Net and Gross Impacts This white paper explores the methodological issues evaluators face in conducting impact evaluations of multiple-measure programs, and how they are somewhat different than when evaluating traditional single-measure programs. The authors question some of the recommendations for evaluating whole-building programs in Chapter 8 of the Uniform Methods Project (UMP) (currently being revised) and some of the implicit assumptions underlying that chapter.
Mitigating Self-Selection Bias in Billing Analysis for Impact Evaluation This white paper describes methods to estimate the net savings of energy efficiency programs using customer-level consumption data analysis, also known as billing analysis for net savings. The specific focus is on statistical techniques for mitigating self-selection bias.

8. Completed Studies LGP Comprehensive Process Evaluations
process evaluations of nine LGPs (San Luis Obispo Energy Watch, Sierra Nevada Energy Watch, San Bernardino County, Los Angeles County, Riverside County, San Mateo County Energy Watch, Monterey Bay Area Governments Energy Watch, City of Chula Vista, and Valley Innovative Energy Watch).
Calmac Study IDs SCG SCG
California EM&V Framework Refresh Needs Assessment
In 2013, in Rulemakeing: R , the CPUC indicated “a broader reexamination of Evaluation Measurement & Verification (EM&V) is in order.” This report provides the findings of a needs assessment conducted on behalf of the four California IOUs in response to R The goal of this needs assessment was to identify whether there are ways to improve the usefulness and usability of the Framework and ensure its applicability to meet the energy efficiency evaluation needs of California given the changing policy and industry environments.
Calmac Study ID SCE