1. Flex Alert | 11.12.2014 | 1 Presentation to the DRMEC Dulane Moran, Research Into Action November 12, 2014 2013 Flex Alert Process Evaluation Findings

2. Flex Alert | 11.12.2014 | 2 Introduction and Overview Program Overview Project Scope and Research Questions Methods and Data sources Findings Staff and stakeholders Customers Media analyses Program comparison Conclusions and Recommendations

3. Flex Alert | 11.12.2014 | 3 2013 Flex Alert Overview

4. Flex Alert | 11.12.2014 | 4 Flex Alert.org

5. Flex Alert | 11.12.2014 | 5 Flex Alert is… Triggered by CAISO (Conservation Notification) Descendent of Flex Your Power Typically before emergency notifications Statewide public appeal Supported by a mixture of paid and earned media Two primary activities: Issuing public notifications (earned media) Conducting on-going public awareness campaign (paid media)

6. Flex Alert | 11.12.2014 | 6 2013 Reflected Differences North/South $10 million budget, primarily allocated to media purchase Allocation of media $$ reflected concerns about SONGS closure – focused on Southern California Three Flex Alert events in 2013 – all called for Northern California April 16 July 1 & 2 February 6, 2014 event driven by cold weather elsewhere

7. Flex Alert | 11.12.2014 | 7 Flex Alert and local voluntary programs Local voluntary IOU programs make similar requests, often with the prospect of bill credits ProgramAprilMayJuneJulyAugustSeptFeb Flex Alert16*1, 26 PG&E Smart Rate7, 281, 2, 19199, 10 SCE SPD228, 305, 9 SDG&E RYU317 * PG&E activated a wide variety of demand response programs on April 16 in response to late night vandalism at a sub-station. *April and February events called until midnight and 10pm respectively

8. Flex Alert | 11.12.2014 | 8 Project Scope and Research Questions

9. Flex Alert | 11.12.2014 | 9 Process Evaluation Research Topics General program management and communication Roles and effectiveness Event day processes and coordination Ad and web swaps Notification lead times Messaging strategy and effectiveness Community groups Non-English speakers General awareness and (event) understanding Media analyses Program comparison

10. Flex Alert | 11.12.2014 | 10 Research Approach and Data Sources ActivityData sources Obtain staff and stakeholder opinions 8 in-depth interviews – included contacts at CAISO, CPUC, Governor’s Office, the IOUs and McGuire Survey Flex Alert NetworkEmail survey of 208 orgs. 38 completes Document role of earned [unpaid] media 11 interviews with media professionals Flexalert.org web analytics Media monitoring data Twitter data General awareness survey800 English Web 382 Non-English Web & Phone Post-event survey*Not fielded for lack of event Community-based organizationsEmail and phone survey. 40 organizations. *Survey is programmed and ready for fielding

11. Flex Alert | 11.12.2014 | 11 Findings

12. Flex Alert | 11.12.2014 | 12 Staff & Stakeholders Divided on Value Respondents were divided on the value of the paid media component Broad support for the idea of a statewide public notification system Leverage the media attention earned by an event Coordination and communication satisfactory IOUs still removed from the process No events during campaign

13. Flex Alert | 11.12.2014 | 13 Varied Organizational Roles Call events Coordinate weekly calls Manage paid media Web switch-outs Notify FAN Receive and respond to requests IOUs CAISO Implementer Media Orgs Customers Messaging Receive notifications Recruit community organizations through summer readiness campaigns Community organizations and FAN Contacts distribute event notifications to constituents Reflect web of communication and diffusion of responsibility

14. Flex Alert | 11.12.2014 | 14 General Survey IOUs CAISO Implementer Media Orgs Customers Messaging

15. Flex Alert | 11.12.2014 | 15 General Survey Approach StrataSample Size WebPhoneResults Weighted English PG&E400 SCE200 SDG&E200 Chinese 95 Korean 80 Vietnamese 87 Spanish120 Unless specified, results presented are for English-speaking contacts All survey items asked about recognition (not recall) of Flex Alert and its features

16. Flex Alert | 11.12.2014 | 16 Relatively High Flex Alert Awareness No spontaneous, completely unaided awareness question; term Flex Alert was provided in a list

17. Flex Alert | 11.12.2014 | 17 Lower Awareness in some Language Groups

18. Flex Alert | 11.12.2014 | 18 2013 Awareness Evidence of Past Marketing

19. Flex Alert | 11.12.2014 | 19 Evidence of Flex Alert & Local DR Confusion ReportedReality 15% Opted in for Flex Alert notifications <0.1% 9% Heard through utility text or email 0% 17% Heard through mail 0% 67% Earning bill credits important motivation No bill credits All English-speaking respondents (n=800)

20. Flex Alert | 11.12.2014 | 20 Flex Alert Understanding is Mixed Most English speakers identify “Flex Alert” as about energy Half recall Flex Alerts occur in afternoons and on one or several days Aided recognition (items read); Flex Alert aware respondents

21. Flex Alert | 11.12.2014 | 21 Flex Alert Understanding is Mixed Somewhat lower understanding among non-English groups Aided recognition (items read); Flex Alert aware respondents

22. Flex Alert | 11.12.2014 | 22 Generally Positive Attitudes 49% Responded to Flex Alert requests 49% Likely to sign up for email notifications 83% Rated avoiding power outages important motivation 92% Somewhat or very likely to respond in future 39% Checked energy use on IOU website 76% Took other actions to reduce energy use All English-speaking respondents (n=800)

23. Flex Alert | 11.12.2014 | 23 Media Analyses IOUs CAISO Implementer Media Orgs Customers Messaging

24. Flex Alert | 11.12.2014 | 24 Earned Media Is Event Responsive Average Impressions Per Day (Millions) Outreach Type 2013 DR Season Apr 2013 Event Period Jul 2013 Event Period Feb 2014 Event Period Northern California Broadcast Ads 1.5--0.8-- Earned Media 0.31.56.81.8 Southern California Broadcast Ads 5.9--4.2-- Earned Media 0.020.20.42.5 2013 DR Season = 6/10 – 9/22 April 2013 Event Period = 4/16 – 4/17 July 2013 Event Period = 6/30 – 7/3 February 2014 Event Period = 2/6 – 2/7 Note: Figure adjusted since report filed, reduced estimated earned media impressions from web traffic for the February 2014 event.

25. Flex Alert | 11.12.2014 | 25 Estimated Impressions per Day 5.9 in SoCal

26. Flex Alert | 11.12.2014 | 26 Energy Shortages Are News Reporters and editors view Flex Alerts as newsworthy Potential power outages are important CAISO staff are seen as credible But, less control of content with earned media General information on timing General request to reduce energy use – not specific suggestions for action

27. Flex Alert | 11.12.2014 | 27 Earned Media is Lower Cost RegionCost ($ Million) Impressions (Million) Cost per Impression Paid Ad Campaign* Northern CA$0.91540.6 Southern CA$6.86241.1 Statewide$7.77781.0 Earned Media** Northern CA--40.2Not available Southern CA--8.7Not Available Statewide--48.9Not Available *Excludes on-line advertising **Includes April 2013, July 2013, and February 2014 events

28. Flex Alert | 11.12.2014 | 28 Earned Media Spiked on Event Days

29. Flex Alert | 11.12.2014 | 29 California Unique Searched ISO programs in US and Canada Two comparative programs identified (ERCOT & ISO- NE) Some similar objectives for peak messaging, however: No programs identified that use paid media to notify of events No programs identified use paid media to increase awareness of Flex Alert-type events No programs have evaluated awareness or verified program impacts

30. Flex Alert | 11.12.2014 | 30 Conclusions

31. Flex Alert | 11.12.2014 | 31 Program Management & Communication Organizational roles reflect a web of event day communication activities IOUs have limited responsibility and engagement Multiple distribution lists, notification systems, and communication avenues Diffusion of responsibility was a challenge to evaluation Weekly coordination calls improved inter-organizational communication IOUs CAISO Implementer Media Orgs Customers Messaging

32. Flex Alert | 11.12.2014 | 32 Event Day Processes and Coordination Paid media campaign does not correspond with actual event day needs Emergency scenarios not always in line with program campaign Coordination with utility demand response programs is limited Documentation of event end times remains inconsistent Monday events are challenging, but no 2013 issues emerged IOUs CAISO Implementer Media Orgs Customers Messaging

33. Flex Alert | 11.12.2014 | 33 Messaging Strategy and Effectiveness Higher level of awareness in Southern California provides evidence of paid media About half of those aware of Flex Alert understand the key components of request Evidence of confusion with local requests Reaching non-English and non- Spanish speaking populations remains challenging CBOs may have access to HTR populations, but need coaching to prepare for event notification IOUs CAISO Implementer Media Orgs Customers Messaging

34. Flex Alert | 11.12.2014 | 34 Recommendations

35. Flex Alert | 11.12.2014 | 35 Recommendations 1.Continue weekly coordination calls 2.Leverage earned media for event communication 3.Ensure that start and end times are communicated in each program-supplied message and that times are consistent 4.Improve consistency and accuracy of messages distributed by social media 5.Clarify the role of and expectations for organizations on the Flex Alert Network 6.Ensure organizations recruited through community outreach efforts receive Flex Alert notifications

36. Flex Alert | 11.12.2014 | 36 Contact: Dulane Moran Research Into Action dulane.moran@researchintoaction.com 503-287-9136

37. Flex Alert | 11.12.2014 | 37 Back up slides

38. Flex Alert | 11.12.2014 | 38 TV, Radio generally top sources of Flex Alert awareness Community groups not a notable source of awareness Evidence of Flex Alert-IOU program confusion Source of Awareness Reflects Media Aided recognition (items read); Flex Alert aware respondents

39. Flex Alert | 11.12.2014 | 39 TV, Radio generally top sources of Flex Alert awareness Community groups not a notable source of awareness Evidence of Flex Alert-IOU program confusion Some sources more prevalent for non-English groups Source of Awareness Reflects Media * Significant difference across languages Aided recognition (items read); Flex Alert aware respondents