Feelings quantified – scoring and storing social media sentiment Matt Gordon
Speaker info Matt Gordon Architect Matt.Gordon@insight.com : @sqlatspeed www.sqlatspeed.com
Speaker info Matt Gordon Co-Founder FGE Professional Sports Analytics matt.gordon@finbargoeseast.com : @sqlatspeed www.sqlatspeed.com
About ME 15+ years of SQL Server experience Microsoft Data Platform MVP IDERA ACE Managed 24x7 datacenters Worked on development teams MCSE: Data Management and Analytics PASS Summit 2017 and 2018 speaker Leader of Lexington, KY PASS Local Group
How I picked my twitter handle and domain name
agenda What is sentiment analysis? Why is sentiment analysis important? How did I get involved in this? What did I build? (English Premier League Mood Table) How did I build it? Wrap-up and takeaways Q&A Takeaway 1: Logic Apps are really powerful (mention training and discount code here) and Cognitive Services are cool as well Takeaway 2: There are things we can do with data outside of our job roles that add value to our company
What is sentiment analysis? “The process of computationally identifying and categorizing opinions expressed in a piece of text, especially in order to determine whether the writer’s attitude towards a particular topic, product, etc., is positive, negative, or neutral.” Google’s definition “Using computer magic to figure out how people are feeling when they tweet about something.” My definition
What is sentiment analysis? Also known as opinion mining Concept has been around for a while but rise of social media brought it to the forefront Typically done with specialized tools whose algorithms are unknown Azure Cognitive Services API democratizes this to an extent
What is sentiment analysis? What are the Cognitive Services APIs? Sets of machine learning algorithms to solve problems in the field of AI These algorithms can be consumed through standard REST calls over the Internet Comprehensive documentation can be found here: https://bit.ly/2sxsqry
What is sentiment analysis? List of Cognitive Services APIs Vision (analyze images and videos) Speech (speech recognition and speaker identification) Language (understand sentences and intent) Knowledge (enhanced search for research) Search (applies ML to web searches) Mood table (and similar sentiment analysis) uses Text Analytics API of Language API
Why is sentiment analysis important? News cycles used to last days – now they last hours Users/customers live on social media these days Political divisions can affect and “enhance” people’s outrage Being on the wrong end of an unfortunate tweet or post can have lasting consequences Companies should baseline their social media sentiment
Why is sentiment analysis important? Dove Body Wash Facebook campaign in October 2017 Took the brand nearly two days to offer a thorough response Proper sentiment analysis would have alerted them to the issue much earlier Response “by feel” is not fast enough
Why is sentiment analysis important? Entenmann’s #NotGuilty tweet in July 2011 Tweeted on a hashtag dedicated to Casey Anthony verdict Didn’t respond to negative attention for hours Account was actually deleted for a period of time
Why is sentiment analysis important? #MyMcDonaldsStory Brands can unintentionally create issues for themselves Important to monitor campaigns where feedback has been solicited Have contingency plans for how to manage that content Delete tweets? Participate in campaign? Monitor sentiment
How did I get involved in this?
How did I get involved in this?
How did I get involved in this? Men In Blazers (Roger Bennett and Michael Davies) Popular soccer podcast TV show on NBC Sports Network Podcast in mid-October 2017 musing about a “mood table” Contacted them with a POC and it’s now become a regular feature on the podcast
How did I get involved in this?
How did I get involved in this? Brad Ball’s sentiment analysis blog got my train of thought started @sqlballs on Twitter, www.sqlballs.com is his blog Premier League Mood Table expands on single-event sentiment Ranks clubs by supporter sentiments within 10 minutes of the final whistle of a match
What did I build? Premier League Mood Table Azure SQL DB Logic app for all 20 clubs Text Analytics API of Cognitive Services Language API Azure Scheduler Job Collection*
What did I build?
What did I build?
How did I build it? Azure SQL DB Created database and table to store Twitter information Sentiment score, account, location, tweet text, etc. Queries to rank the clubs by sentiment from best to worst 1 is most positive 0 is most negative 0.5 is no sentiment detected
How did I build it? Logic Apps Logic apps are a cloud service that helps you automate and orchestrate tasks, business processes, and workflows One per club “When a new tweet is posted” trigger Feeds tweet text to Detect Sentiment action from Text Analytics API Final step is Insert Row action into Azure SQL DB Original version also streamed to Power BI
How did I build it? Azure Scheduler Job Collection Uses Azure AD to authenticate the logic apps Direct scheduling interface for enable and disable actions Provides run history Provides execution count and error count Helps manage costs *Will be deprecated on September 30, 2019 Recurrence trigger new scheduling method after that date
DEMO TIME Mood Table In-depth
How did I build it? AWS Can Do This Too! Amazon RDS for database Amazon Comprehend for text and sentiment analysis Amazon Lambda or StepFunctions for logic app-like behavior CloudWatch Events can provide scheduling
WHERE ELSE CAN WE APPLY this technology? Transfer data between two disparate systems Healthcare Sentiment analysis on knowledge base/community posts Sentiment analysis on monitored social media School violence Self-harm
What’s next? (soccer customer things) Using Python and/or rtweet to scrape past tweets (and ML to score those tweets) in order to form more cohesive and customizable sentiment pictures Working with soccer clubs to map this against fan response, souvenir sales, and game action Working with Raspberry Pi and microphone arrays to do in-stadium “sentiment”
What’s next? (soccer customer things) Event monitoring Custom hashtags or matchday/event experiences Allows for real-time monitoring of on-site fan sentiment Enables quick reactions to potential issues
What’s next? (non-soccer customer things) Azure function to send an email or initiate a workflow alerting about Tweets below a certain sentiment score Power BI live sentiment dashboard PASS Summit demo video here Geography-based reports
THE SLIDE WHERE I SHILL FOR TRAINING I MADE In-depth Logic Apps course with Cognitive Services labs https://pragmaticworks.com/Training/On-Demand-Training Click “Buy Now” button In cart, click “I have a discount coupon” and then enter code MattGordon 20% off any of their on-demand training courses and my wife will be impressed that people keep buying something I made
Wrap-up and takeaways Social media presence requires sentiment analysis Any data professional or data-curious pro can have a role in this, especially in smaller organizations Underscores how important machine learning is to understand Language API (particularly Text Analytics) and Logic Apps give you powerful tools to measure company or personal brand online I am a big dork that built something to rank soccer teams based on feelings
Questions?
Speaker info Matt Gordon Architect Matt.Gordon@insight.com : @sqlatspeed www.sqlatspeed.com
REFERENCes and resources Cognitive Services Documentation: https://bit.ly/2sxsqry Text Analytics overview: https://bit.ly/2APjgtP Human Language Technologies white papers from MS: https://bit.ly/2FfOXAS LUIS white paper from MS Research: https://bit.ly/2JJDJ68 AWS doc on building equivalent in AWS: https://amzn.to/2Iabdfs