Sentiment Analysis on Tweets

Thumbs up? Sentiment Classification using Machine Learning Techniques Classify documents by overall sentiment. Machine Learning Methods – Naïve Bayes – Maximum Entropy Classification – Support Vector Machine Features – Unigrams, Bigrams, Part of speech, Position etc.

Thumbs up? Sentiment Classification using Machine Learning Techniques Data Source – Internet Movie Database ( – Use reviews with star or numerical value as training and test data. (convert into three categories: positive, negative and neutral)

Thumbs up? Sentiment Classification using Machine Learning Techniques Machine Learning Methods: – Naïve Bayes: given document d, the class c* = arg maxcP(c|d). Assume all features are conditionally independent ni(d) is the number of times fi occurs in document d.

Thumbs up? Sentiment Classification using Machine Learning Techniques Machine Learning Methods: – Maximum Entropy: Z(d) is a normalization function. The are feature-weight parameters. Larger means fi is considered a strong indicator for class c.

Thumbs up? Sentiment Classification using Machine Learning Techniques Machine Learning Methods: – Support Vector Machine. Find a hyperplane makes the margin between two categories.

Thumbs up? Sentiment Classification using Machine Learning Techniques Features – Unigrams: A single word. – Feature frequency: frequency of a feature appears – Feature presence: 1 only when a feature appears – Bigrams: Two continues word. – Parts of Speech: Tag the word with its POS. – Adjectives: Only use adjectives in the text. – Position: The position of a word in the text. In the first quarter, last quarter or the middle half.

Thumbs up? Sentiment Classification using Machine Learning Techniques Results for different feature: – Unigrams works better than baseline, but worse than topic-based classification – Presence is better than frequency – Bigram feature does not improve performance – Adjectives are poor – POS improve slight for NB and ME, but decline for SVM – Position also does not help

Twitter Sentiment Classification using Distant Supervision Analysis sentiment for tweets – Use the machine learning methods similar to the previous paper. (Naïve bayes, Maximum Entropy and Support Vector Machine) – Use emoticons as noisy label for training data – Features: unigrams, bigrams and parts of speech

Twitter Sentiment Classification using Distant Supervision Characteristics of Tweets – Length: The maximum length is 140 characters. – Data availability: Easy to collect millions of tweets. – Language Model: High frequency of misspellings and slang. – Domain: Varity of topics.

Twitter Sentiment Classification using Distant Supervision Data source: Use query term to extract tweets. – Training Data: Use scraper to extract tweets which contain following emoticons. Stripe off emoticons in tweets. Remove tweets which contain both positive and negative emoticons. Remove retweets. Remove tweets with “:P” Remove repeated Twitter

Twitter Sentiment Classification using Distant Supervision Test Data – Search Twitter API with specific queries like kindle, aig, at&t etc. – Label those twitter manually.

Twitter Sentiment Classification using Distant Supervision Machine Learning Methods – Naïve Bayes – Maximum Entropy – Support Vector Machine

Twitter Sentiment Classification using Distant Supervision Results – Unigrams: 81.3%, 80.5%, 82.2%. – Bigrams: Overall accuracy drops for MaxEnt and SVM. Because the feature space of bigram is very sparse. (Can we just use some frequent bigram?) – Unigrams and Bigrams: 82.7%, 82.7%, 81.6%. – Parts of Speech: Not very useful.

Enhanced Sentiment Learning Using Twitter Hashtags and Smileys Use Hashtags and smiley as sentiment label Features include single word, n-gram, pattern and punctuation. K-nearest neighbors strategy classification algorithm.

Enhanced Sentiment Learning Using Twitter Hashtags and Smileys Classification features: – Word based and n-gram based features Each word appearing in a sentence serves as a binary feature with weight equal to the inverted counts of this word. Rare words have a higher weight than common words. 2-5 words as a binary n-gram feature using a similar weighting strategy. Appearing less than 0.5% do not constitute a feature.

Enhanced Sentiment Learning Using Twitter Hashtags and Smileys Classification features: – Pattern based features. Pattern: [HFW][CW slot][HFW]. (HFW: high frequency words; CW: common words)

Enhanced Sentiment Learning Using Twitter Hashtags and Smileys Punctuation-based features – Sentence length in words – Number of “!” in sentence – Number of “?” in sentence – Number of quotes in sentence – Number of capitalized words in sentence

Enhanced Sentiment Learning Using Twitter Hashtags and Smileys Classification algorithm – K-nearest neighbors like strategy. – Let ti, i= 1…k be the k vectors with lowest Euclidean distance to v. – Outlier vectors, the distance of which was more than twice the mean distance. – Label assigned to v is the label of the majority of the remaining vectors.

Enhanced Sentiment Learning Using Twitter Hashtags and Smileys Datasets – From Brendan O‘Connor, PhD student from CMU. – 475 million from May 2009 to Jan 2010 – Replace URL, Hashtags and references by URL/REF/TAG meta-words. – Hashtag-based label (like: #suck # notcute etc…) Select 50 hashtags annotated “1” or “2” by both judges. Each hashtags sampled 1000 tweets, so get 50,000 labeled tweets – Smiley-based label Select 15 smileys. Sample 1000 tweets for each smiley. – No-sentiment dataset Randomly sampled tweets with no hashtags/smileys

Enhanced Sentiment Learning Using Twitter Hashtags and Smileys Results – Multi-class classification. Assign a single label (51 hashtag and 16 in case of smileys)

Enhanced Sentiment Learning Using Twitter Hashtags and Smileys Binary classification – Providing as training/test sets only positive examples of the specific sentiment label together with non-sentiment examples. – Classification of a variety of different sentiment types.

Robust Sentiment Detection on Twitter from Biased and Noisy Data Leverage sources of noisy labels as training data.(Twendz, Twitter Sentiment and TweetFeel) 2 step sentiment analysis. – Subjective and objective – Distinguish subjective tweets as positive or negative or neutral. Features – Meta-features – Tweet syntax Features

Robust Sentiment Detection on Twitter from Biased and Noisy Data Features – Meta-features: POS. Use a POS dic Prior Subjectivity. Use subjectivity lexicon. (slang web vocabulary) – Tweet Syntax Features: Retweet Hashtag Reply Link Punctuation Emoticons Upper cases The frequency of each feature is divided by the number of the words in the tweet.

Robust Sentiment Detection on Twitter from Biased and Noisy Data Subjectivity Classifier – Clean data Remove the tweets that are disagreed Remove same user’s message Clean objective training set, remove tweets with top-n opinion words (cool, awesome) from it. – Features: Meta-info: positive polarity, strong subjective and verbs Syntax features: (link and upper case)

Robust Sentiment Detection on Twitter from Biased and Noisy Data Polarity Classifier – Combine 3 data sources. – Use kappa coefficient to measure degree of agreement between two sources. – Polarity Features (meta-info is more important) Meta-info: negative polarity, positive polarity and verbs Syntax features: emoticons and upper case

Robust Sentiment Detection on Twitter from Biased and Noisy Data Experiment – Use SVM. Compare with others’ previous work. – Subjectivity detection evaluation. – TwitterSA(cleaning) is best, also with the smallest training set

Robust Sentiment Detection on Twitter from Biased and Noisy Data Experiment – Polarity detection evaluation. TwitterSA (maxconf) is best, also with the smallest training set

Target-dependent Twitter Sentiment Classification Target dependent: Give a query, analysis the tweets sentiment about the query. query “chrome”. Tweet: I am hating chrome right about now. 3 step. – Subjective or objective – Positive or negative – Graph-based optimization. Taking related tweet into consideration Incorporating target-dependent features. Using SVM

Target-dependent Twitter Sentiment Classification Target independent features – Content features: words, punctuation, emoticons and hashtags – Sentiment lexicon features: how many positve and negative words are included in the tweet.

Target-dependent Twitter Sentiment Classification Target dependent classification – Extended targets. People may comment on some related things of the target. – Indentify all extended targets. First regard all noun phrases as extended targets Co-referring.(“oh, Jon Stewart. How I love you so.”, “you” and “Jon Stewart” are co-referring) Identifying the top K nouns and noun phrases which have strongest association with the target and regard these nouns or noun phrases as extended target.

Target-dependent Twitter Sentiment Classification Target dependent features – Wi is a transitive verb and T is its object. wi_arg2. – Wi is a transitive verb and T is its subject. wi_arg1. – Wi is a intransitive verb and T is its subject. wi_it_arg2. – Wi is an adjective or noun and T is its head. Wi_arg1 – Wi is an adjective or noun and it is connected by a copula with T. Wi_cp_arg1 – Wi is an adjective or intransitive verb appearing aline as a sentence and T appears in the previous sentence. Wi_arg – Wi is an adverb and the verb it modifies has T as its subject. Arg1_v_well – Feature modified by a negation, add a prefix “neg-” All target-dependent features are binary features. If the feature present, the entry is 1; otherwise is 0.

Target-dependent Twitter Sentiment Classification Graph-based Sentiment Optimization – Take contexts into consideration Retweet Tweet containing the same target and published by the same person Tweets replying to or replied to. – Construct a graph based on the above relationship

Target-dependent Twitter Sentiment Classification Experiment: Queries: {Obama, Google, Ipad, Lakers, Lady Gaga}. For each query, download 400 tweets.

Reference Bo Pang, Lillian Lee, Shivakumar Vaithyanathan Thumbs up? Sentiment Classification using Machine Learning Techniques Alec Go, Richa Bhayani, Lei Huang Twitter Sentiment Classification using Distant Supervision. Dmitry Davidiv, Oren Tsur and Ari Rappoport Enhanced Sentiment Learning Using Twitter Hashtags and Smileys. Coling Luciano Barbosa and Junlan Feng Robust Sentiment Detection on Twitter from Biased and Noisy Data. Coling Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, Tiejun Zhao. Target-dependent Twitter sentiment classification