1. Powering forward. Together. December 2014 A Sentiment-al Journey: Text and Sentiment Analysis of Online Discussion Responses Susan Corbelli and Anya Suneson Powering forward. Together.

2. Home: Sacramento Municipal Utility District Electric Utility Located in Northern California Have about 610,000 Customers –540,000 Residential –70,000 Commercial Covers territory: Sacramento County Part of Placer County

3. Why an electric utility does customer research? Monitor existing programs Develop new programs Improve customer experience Traditional methods: Telephone, mail, online surveys Focus groups Now venturing out into new methods…

4. Travel Bucket List

5. “Bucket list” of Innovative Research Methods  Ethnography  Usability testing  Online Research Community  Discussion boards  Text analytics

6. The “Vacation Home” – Online Research Communities Spent lots of time researching and planning Big investment Use regularly Requires regular maintenance Lots of people want to use

7. SMUD’s Online Research Community

8. The “Time Share” – Discussion Boards Use occasionally Short term use No long term commitment Great way to try something different!

9. Case Study: Analysis of Online Discussion Board Responses

10. Research Objectives and Methodology Objective: Determine customer energy-related needs and ways SMUD can address them Data Collection Method: Online Discussion Boards Invitation sent out to 386 SMUD Plugged In participants Online discussion was open for 2 weeks Facilitated by SMUD research specialist About 100 respondents participated in discussion

11. Online Discussion Board Example

12. Positive Customer Feedback I appreciate the opportunity to be able to share my opinions and ideas with SMUD. Since I consider SMUD to be a highly ethical company, I would like to be a participant in helping it flourish, in any way I can. I really love that my opinion matters to you. Tons of love for SMUD. That is all =) I appreciate the opportunity to give feedback and voice my concerns, as well as suggestions for future development of my energy provider.

13. Text Analysis Tools Utilized Traditional Verbatim Coding Word clouds Semantic score Sentiment Analysis

14. Text Analysis: Traditional Coding Pros: Human coder gets nuances machine can’t Cons: Labor- and time-intensive

15. The “Date Night” – Word Clouds Inexpensive Don’t require much planning or preparation Quick look Fun

16. Text Analysis: Word Clouds Pros: Quick, easy, free, visually-appealing Cons: Not sophisticated, word-level and not phrase-level

17. The “Day in Port” – Sentiment Analysis Heard “buzz” about SA Cost ranges from free to very expensive Quick look But soon realized need more time in this “destination”

18. What is Sentiment Analysis? Also called Opinion Mining Is a systematic analysis of online expressions. Focuses on evaluating attitudes and opinions on a topic of interest using machine learning techniques (Rombocas) Classifies words, text, documents according to the opinion, emotion or sentiment that they express Determines: –Subjective vs. Objective polarity –Positive vs. Negative polarity –Strength of the opinion

19. Benefits of Sentiment Analysis Can analyze large sets of data quickly Can use on many different sources of unstructured data Measures emotions which are important for marketing Data less influenced by researcher Less subjective than human coding Allows real-time analysis

20. Text Analysis: Sentiment Score Tool: Python NLTK Pros: Easy to use, fast, free Cons: Harder to explain, less actionable, further analysis requires Python programming knowledge

21. Text Analysis: Semantic Analysis Tool: Semantria Pros: Sophisticated, can churn big volumes, combines sentiment score and phrase-level categorization Cons: Costly, takes time to learn

22. Semantic Analysis Facets are the most important ideas with their accompanying attributes. Facets rely on Subject Verb Object (SVO) parsing to find trends even when there are weak or no noun phrases in your text. Affordable energy, reliable energy, fixed income, low income and solar power were some of the facets identified in our text

23. Discussion boards were a great way to have more of a conversation with our customers Text analysis methods can help save time and offer new insights Sentiment analysis appears to be useful for finding themes and important ideas in verbatims – especially for certain types of questions Need more time to better understand and utilize Sentiment Analysis and available tools –May need to learn a new language? –Maybe hire a guide? –Plan to visit again! What did we learn on our journey?

24. Questions? Thank you!

25. Resources A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts Bo Pang and Lillian Lee http://www.cs.cornell.edu/home/llee/papers/cutsent.pdf LingPipe Sentiment Tutorial http://alias-i.com/lingpipe/demos/tutorial/sentiment/read-me.html Professor Dan Jurafsky & Chris Manning are offering a free online course on Natural Language Processing. http://www.nlp-class.org/ https://www.coursera.org/course/textanalytics

26. Resources http://www.online-utility.org/text/analyzer.jsp Free software utility which allows you to find the most frequent phrases and frequencies of words. Non-English language texts are supported. It also counts number of words, characters, sentences and syllables. Also calculates lexical density. Text Mining package in R: http://cran.r-project.org/web/packages/tm/vignettes/tm.pdfhttp://cran.r-project.org/web/packages/tm/vignettes/tm.pdf Data Mining (including Text) Examples: http://cran.r- project.org/doc/contrib/Zhao_R_and_data_mining.pdfhttp://cran.r- project.org/doc/contrib/Zhao_R_and_data_mining.pdf MAXQDA, free 30-day trial available at http://www.maxqda.com/downloads/demohttp://www.maxqda.com/downloads/demo UCINET, free 30-day trial available at http://www.analytictech.com/downloaduc6.htmhttp://www.analytictech.com/downloaduc6.htm

27. Resources - Word Cloud Tools www.wordle.net TIP: Double click on word to remove, can use “~” to combine words, customize color www.tagcrowd.comwww.tagcrowd.com TIP: Does stemming www.tagxedo.comwww.tagxedo.com TIP: Does stemming, many shapes https://tagul.com/

28. Free Sentiment Analysis Tools 1. Twitrratr – www.twitrratr.comwww.twitrratr.com 2. Sentiment 140 - http://www.sentiment140.comhttp://www.sentiment140.com 3. Tweetfeel – www.tweetfeel.comwww.tweetfeel.com 4. Opinmind – www.opinmind.comwww.opinmind.com 5. Social Mention – www.socialmention.comwww.socialmention.com