Powering forward. Together. December 2014 A Sentiment-al Journey: Text and Sentiment Analysis of Online Discussion Responses Susan Corbelli and Anya Suneson Powering forward. Together.

Home: Sacramento Municipal Utility District Electric Utility Located in Northern California Have about 610,000 Customers –540,000 Residential –70,000 Commercial Covers territory: Sacramento County Part of Placer County

Why an electric utility does customer research? Monitor existing programs Develop new programs Improve customer experience Traditional methods: Telephone, mail, online surveys Focus groups Now venturing out into new methods…

Travel Bucket List

“Bucket list” of Innovative Research Methods  Ethnography  Usability testing  Online Research Community  Discussion boards  Text analytics

The “Vacation Home” – Online Research Communities Spent lots of time researching and planning Big investment Use regularly Requires regular maintenance Lots of people want to use

SMUD’s Online Research Community

The “Time Share” – Discussion Boards Use occasionally Short term use No long term commitment Great way to try something different!

Case Study: Analysis of Online Discussion Board Responses

Research Objectives and Methodology Objective: Determine customer energy-related needs and ways SMUD can address them Data Collection Method: Online Discussion Boards Invitation sent out to 386 SMUD Plugged In participants Online discussion was open for 2 weeks Facilitated by SMUD research specialist About 100 respondents participated in discussion

Online Discussion Board Example

Positive Customer Feedback I appreciate the opportunity to be able to share my opinions and ideas with SMUD. Since I consider SMUD to be a highly ethical company, I would like to be a participant in helping it flourish, in any way I can. I really love that my opinion matters to you. Tons of love for SMUD. That is all =) I appreciate the opportunity to give feedback and voice my concerns, as well as suggestions for future development of my energy provider.

Text Analysis Tools Utilized Traditional Verbatim Coding Word clouds Semantic score Sentiment Analysis

Text Analysis: Traditional Coding Pros: Human coder gets nuances machine can’t Cons: Labor- and time-intensive

The “Date Night” – Word Clouds Inexpensive Don’t require much planning or preparation Quick look Fun

Text Analysis: Word Clouds Pros: Quick, easy, free, visually-appealing Cons: Not sophisticated, word-level and not phrase-level

The “Day in Port” – Sentiment Analysis Heard “buzz” about SA Cost ranges from free to very expensive Quick look But soon realized need more time in this “destination”

What is Sentiment Analysis? Also called Opinion Mining Is a systematic analysis of online expressions. Focuses on evaluating attitudes and opinions on a topic of interest using machine learning techniques (Rombocas) Classifies words, text, documents according to the opinion, emotion or sentiment that they express Determines: –Subjective vs. Objective polarity –Positive vs. Negative polarity –Strength of the opinion

Benefits of Sentiment Analysis Can analyze large sets of data quickly Can use on many different sources of unstructured data Measures emotions which are important for marketing Data less influenced by researcher Less subjective than human coding Allows real-time analysis

Text Analysis: Sentiment Score Tool: Python NLTK Pros: Easy to use, fast, free Cons: Harder to explain, less actionable, further analysis requires Python programming knowledge

Text Analysis: Semantic Analysis Tool: Semantria Pros: Sophisticated, can churn big volumes, combines sentiment score and phrase-level categorization Cons: Costly, takes time to learn

Semantic Analysis Facets are the most important ideas with their accompanying attributes. Facets rely on Subject Verb Object (SVO) parsing to find trends even when there are weak or no noun phrases in your text. Affordable energy, reliable energy, fixed income, low income and solar power were some of the facets identified in our text

Discussion boards were a great way to have more of a conversation with our customers Text analysis methods can help save time and offer new insights Sentiment analysis appears to be useful for finding themes and important ideas in verbatims – especially for certain types of questions Need more time to better understand and utilize Sentiment Analysis and available tools –May need to learn a new language? –Maybe hire a guide? –Plan to visit again! What did we learn on our journey?

Questions? Thank you!

Resources A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts Bo Pang and Lillian Lee LingPipe Sentiment Tutorial Professor Dan Jurafsky & Chris Manning are offering a free online course on Natural Language Processing.

Resources Free software utility which allows you to find the most frequent phrases and frequencies of words. Non-English language texts are supported. It also counts number of words, characters, sentences and syllables. Also calculates lexical density. Text Mining package in R: Data Mining (including Text) Examples: project.org/doc/contrib/Zhao_R_and_data_mining.pdfhttp://cran.r- project.org/doc/contrib/Zhao_R_and_data_mining.pdf MAXQDA, free 30-day trial available at UCINET, free 30-day trial available at

Resources - Word Cloud Tools TIP: Double click on word to remove, can use “~” to combine words, customize color TIP: Does stemming TIP: Does stemming, many shapes

Free Sentiment Analysis Tools 1. Twitrratr – 2. Sentiment Tweetfeel – 4. Opinmind – 5. Social Mention –