1. AESP MARKET RESEARCH, EVALUATION, AND GREEN HOUSE GAS TOPIC COMMITTEE MEETING April 22, 2015 First quarter TC Meeting, on behalf of the AESP

2. Agenda Update on National Conference TC meeting TC Leadership Team Hot topics Opportunities to participate in TC Next steps Special topic: international efforts related to evaluation (Bill Miller) Wrap up

3. National Conference Update

4. Nominated Leadership Team Laura Schauer, Chair Vice Chairs Sue Hanson (Sue.Hanson@tetratech.com), Tetra TechSue.Hanson@tetratech.com Linda Dethman (linda.dethman@reseachintoaction.com) and Paul Schwartz (paul.schwarz@researchintoaction.com), Research Into Actionlinda.dethman@reseachintoaction.compaul.schwarz@researchintoaction.com Bryan Ward (Bryan.Ward@camdusgroup.com), CadmusBryan.Ward@camdusgroup.com Wendy Todd (Wendy.todd@dnvgl.com), DNV GL Lisa Obear (Lobear@emiconsulting.com), EMI ConsultingLobear@emiconsulting.com Martha Thompson (Thompson@evergreenecon.com), Evergreen EconomicsThompson@evergreenecon.com

5. Share, Network, and Engage: Increasing the Value to Members Asked the question: What can our topic committee do for you? Be a resource Create a forum for keeping an ear to the ground on policy and other issues Serve as a learning community for each other Engage each other in hot topics real-time This first quarterly meeting is in response to that brainstorm! What else would you like to see?

6. Hot Topics

7. Hot topic categories There were no shortage of ideas this year! It was a GREAT brainstorm session. Market research and data Big data and EM&V 2.0 EM&V methodological issues Communication and reporting Market issues Economic impacts Emerging issues

8. Market research and data, Big Data / EM&V 2.0 Market research and data Market research techniques to see how customers respond – tools and technologies Mixed mode research How to deal with increasing data privacy issues Others? Big Data / EM&V 2.0 Larger datasets – ways to do measurements that are good, fast, and cheap enough How to assess quality of measurements with big data

9. EM&V Methodological Issues Using evaluation to support and validate potential for emerging technologies and pilot programs How to use evaluation to respond when things don’t turn out as expected Evolution of billing data methods – we haven’t evolved in our methods much over the years, what is coming up and how should we be evolving? How to establish and implement experimental (and quasi- experimental) study designs Role of in field performance measurement in our evaluations

10. Communication and Reporting How to effectively communicate EM&V results and findings (including various reporting tools such as dashboards) Formalizing the feedback loop EM&V best practices (or even just better practices) – evaluator certification was briefly raised Combining qualitative and quantitative methods and results for reporting Using E&V as a “listening” tool

11. Market Issues Market assessment of the value proposition of DSM Obtaining and using sales and market data in evaluation (as well as planning) Customer engagement and segmentation Market effects of whole building programs Codes, market effects, and how those issues interact with other programs

12. Economic Impacts Cost-effectiveness testing – calculability and metrics (this may be a topic covered in DSM Business, or where we can collaborate with DSM Business) Non-energy benefits Assessing effective useful lives

13. Emerging Issues Assessing commercial behavior change programs Evaluating integrated resource programs (DR/EE/water/other) Smart homes, tools, and technologies Measuring impact of DSM on climate change Behavioral programs and demand response What does EM&V look like in the future?

14. What do we do with these? Getting engaged!

15. How to become engaged Brown bags (Need to deliver 3/year) Brainstorm ideas Help develop content Identify and/or recruit presenters Strategies articles (need to draft 2/year, first published July) Contribute ideas Lead drafting of the article Offer to review draft Participate in ad-hoc surveys

16. How to become engaged Mission statement review team Currently, “To advance market research, evaluation and greenhouse gas in support of energy services activities in competitive and regulated markets and to rapidly disseminate related information to AESP International’s membership.” Participate in other ways! Lead a hot topic presentation during a quarterly meeting Review awards and award nominations Raise hot topics as they arise Attend quarterly meetings Organize regional TC meetings Etc.!

17. Questions for you to consider How would you like to participate in the TC? How can the TC be of the most value to you? What other hot topics should we consider? Please pass on thoughts to me!

18. Next steps TC leadership team will organize brown bag topics and strategies articles and begin scheduling those events We’d love your input on this! Considering a sign-up genius to organize volunteers for brown bags. Look for that email in the next few weeks. Next quarterly meeting will be ~July. Have something you’d like to share with the group? Please let me know! Looking for 2-3 information sharing topics for this next meeting.

19. Special topic: International effort to establish evaluation criteria

20. Wrap-up