Sentiment Analysis Balamurali A R IITB-Monash Research Academy {balamurali@cse.iitb.ac.in} Acknowledgment: Aditya Joshi

Artist: Leonardo da Vinci Image from wikimedia commons Source: Wikipedia Smile of Mona Lisa Is she smiling at all? Is she happy? What is she smiling about? What is she happy about? Mona Lisa 16th century Artist: Leonardo da Vinci

Sentiment analysis (SA) Task of tagging text with orientation of opinion This is a good movie. This is a bad movie. The movie is set in Australia. Subjective Objective

Sentiment Analysis Robotics NLP Search, Reasoning, Expert Learning Disciplines which form the core of AI- inner circle Fields which draw from these disciplines- outer circle. Planning Computer Vision NLP Expert Systems Robotics Search, Reasoning, Learning Sentiment Analysis

Motivation & Introduction Outline Motivation & Introduction Applications Approaches to SA SA @ CFILT

Motivation & Introduction Outline Motivation & Introduction Applications Need of SA: Why is SA needed? Variants of SA: What forms does it exist in? Challenges of SA: Why is SA not trivial? Approaches to SA SA @ CFILT

User-generated content Web 2.0 empowers the user of the internet They are most likely to express their opinion there Temporal nature of UGC: ‘Live Web’ Can SA tap it?

SA: Where? Blogs Review websites Social networks User conversations Conversations between users on one of the above Multiple review websites offering specific to general-topic reviews Some SPs: mouthshut, burrrp, bollywoodhungama A website, usually maintained by an individual with regular entries of commentary, descriptions of events. Some SPs: Blogger, LiveJournal, Wordpress Websites that allow people to connect with one another and exchange thoughts

SA: How much? Size of blogosphere Reference : www.technorati.com/state-of-the-blogosphere/ SA: How much? Size of blogosphere Through the ‘eyes’ of the blog trackers Technorati : 112.8 million blogs (excluding 72.82 million blogs in Chinese as counted by a corresponding Chinese Center) A blog crawler could extract 88 million blog URLs from blogger.com alone 12,000 new weblogs daily

SA: How much opinion? Chart created using : www.technorati.com/chart/

Flavours of SA Subjective/Objective Emotion analysis SA with magnitude Entity-specific SA Feature-based SA Perspectivization “The camera is the best in its price range. However, a pathetically slow interface ruins it for this cell phone.” “The Leftists were arrested yesterday by the police.” “India defeated England in the cricket match badly.” “dude.. just get lost.” “Whoa! Super!!” “The movie is good.” “People say that the movie is good.” “This movie is awesome.” “Taj Mahal was constructed by Shah Jahan in the memory of his wife Mumtaz.” “Taj Mahal is a masterpiece of an architecture and symbolizes unparalleled beauty.”

Challenges of SA Domain dependent Sarcasm Thwarted expressions Negation Implicit polarity Time-bounded “The camera of the mobile phone is less than one mega-pixel – quite uncommon for a phone of today.” “This phone allows me to send SMS.” “This phone has a touch-screen.” “I did not like the movie.” “Not only is the movie boring, it is also the biggest waste of producer’s money.” “Not withstanding the pressure of the public, let me admit that I have loved the movie.” Sarcasm uses words of a polarity to represent another polarity. Example: “The perfume is so amazing that I suggest you wear it with your windows shut” the sentences/words that contradict the overall sentiment of the set are in majority Example: “The actors are good, the music is brilliant and appealing. Yet, the movie fails to strike a chord.” Sentiment of a word is w.r.t. the domain. Example: ‘unpredictable’ For steering of a car, For movie review,

SA Challenges: Sample Review 1 (This, that and this) ‘Touch screen’ today signifies a positive feature. Will it be the same in the future? FLY E300 is a good mobile which i purchased recently with lots of hesitation. Since this Brand is not familiar in Market as well known as Sony Ericsson. But i found that E300 was cheap with almost all the features for a good mobile. Any other brand with the same set of features would come around 19k Indian Ruppees.. But this one is only 9k. Touch Screen, good resolution, good talk time, 3.2Mega Pixel camera, A2DP, IRDA and so on... BUT BEWARE THAT THE CAMERA IS NOT THAT GOOD, THOUGH IT FEATURES 3.2 MEGA PIXEL, ITS NOT AS GOOD AS MY PREVIOUS MOBILE SONY ERICSSION K750i which is just 2Mega Pixel. Sony ericsson was excellent with the feature of camera. So if anyone is thinking for Camera, please excuse. This model of FLY is not apt for you.. Am fooled in this regard.. Audio is not bad, infact better than Sony Ericsson K750i. FLY is not user friendly probably since we have just started to use this Brand. Comparing old products The confused conclusion From: www.mouthshut.com

SA Challenges: Sample Review 2  Hi,    I have Haier phone.. It was good when i was buing this phone.. But I invented  A lot of bad features by this phone those are It’s cost is low but Software is not good and Battery is very bad..,,Ther are no signals at out side of the city..,, People can’t understand this type of software..,, There aren’t features in this phone, Design is better not good..,, Sound also bad..So I’m not intrest this side.They are giving heare phones it is good. They are giving more talktime and validity these are  also good.They are giving colour screen at display time it is also good because other phones aren’t this type of feature.It is also low wait. Lack of punctuation marks, Grammatical errors Wait.. err.. Come again From: www.mouthshut.com

Motivation & Introduction Outline Motivation & Introduction Applications Need of SA: Why is SA needed? Variants of SA: What forms does it exist in? Challenges of SA: Why is SA not trivial? Approaches to SA SA @ CFILT Basics: What is classification Approaches: What are ways to do SA

Task Definition Marking reviews as positive or negative at the document level Lexicon-based classifiers ML-based classifiers

What is classification? A machine learning task that deals with identifying the class to which an instance belongs A classifier performs classification Classifier SVM Classifier Naïve Bayes Classifier MaxEnt Classifier Combination ( Perceptive inputs ) ( Age, Marital status, Health status, Salary ) ( Textual features : Ngrams ) Test instance Attributes (a1, a2,… an) Discrete-valued Class label Category of document? {Politics, Science, Biology} Steer? { Left, Straight, Right } Issue Loan? {Yes, No}

Classification learning Training phase Testing phase Learning the classifier from the available data ‘Training set’ (Labeled) Testing how well the classifier performs ‘Testing set’

Testing phase Methods: Holdout (2/3rd training, 1/3rd testing) Cross validation (n – fold) Divide into n parts Train on (n-1), test on last Repeat for different permutations

Approaches to SA and Text granularity Based on text granularity Document level Sentence level Phrase level Word level ………..Approaches to SA will differ

Machine Learning based Generic Approaches Document level sentiment classifier SA Machine Learning based Supervised Unsupervised Rule based Lexicon based

Motivation & Introduction Outline Motivation & Introduction Applications Need of SA: Why is SA needed? Variants of SA: What forms does it exist in? Challenges of SA: Why is SA not trivial? Approaches to SA SA @ CFILT Basics: What is classification Approaches: What are ways to do SA 1) Rule based 2) Machine Learning based

Rule based System: Resources for SA SentiWordNet WordNet synsets marked with three types of scores: positive, negative, objective I am feeling happy. I am feeling happy.

Seed-set expansion in SWN also-see Seed words antonymy Ln Lp The sets at the end of kth step are called Tr(k,p) and Tr(k,n) Tr(k,o) is the set that is not present in Tr(k,p) and Tr(k,n)

Building SentiWordnet Classifier alternatives used: Rocchio (BowPackage) & SVM(LibSVM) Different training data based on expansion POS –NOPOS and NEG-NONEG classification Total eight classifiers For different combinations of k and classifiers Synsets not in the expanded seed set are used as test synsets Score is average of scores returned by the classifiers 25 25

Rule based System: An Example C-FEEL-IT An entity-based opinion search engine on Twitter How it works? User enter a search string to get its “public vibe” Tweets are fetched based on search string Based on sentiment lexicon, mark each tweet with sentiment score using majority rule Categorize each tweet into sentiment categories using a threshold value

C-FeeL-IT: Preprocessing and heuristics Feeds from twitter used to obtain: 50 tweets In English About the keyword Normalization done using: Mapping between chat lingo to dictionary words1 Mapping between emoticons and direct sentiment prediction1 Extensions of words replaced by contracted forms Negation handling 1 http://chat.reichards.net/

C-FeeL-IT: Resources used •SentiWordNet  (Andrea & Sebastani,2006) •Subjectivity clues  (Weibi et al, 2004) •Taboada (Taboada & Grieve, 2004) •Inquirer (Stone et al, 1966) Andrea Esuli and Fabrizio Sebastiani. 2006. SentiWordNet: A publicly available lexical resource for opinion mining. In Proceedings of LREC-06, Genova, Italy Janyce Wiebe, Theresa Wilson, Rebecca Bruce, Matthew Bell, and Melanie Martin. 2004. Learning subjective language. Computional Linguistics, 30:277–308, September. Maite Taboada and Jack Grieve. 2004. Analyzing Appraisal Automatically. In Proceedings of the AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications, pages 158–161, Stanford, US. Philip J. Stone, Dexter C. Dunphy, Marshall S. Smith, and Daniel M. Ogilvie. 1966. The General Inquirer: A Computer Approach to Content Analysis. MIT Press. Reference given in Notes section

C-FeeL-IT: Demo Available at: http://www.clia.iitb.ac.in:8080/cfeelit-2/

Motivation & Introduction Outline Motivation & Introduction Applications Need of SA: Why is SA needed? Variants of SA: What forms does it exist in? Challenges of SA: Why is SA not trivial? Approaches to SA SA @ CFILT Basics: What is classification Approaches: What are ways to do SA 1) Rule based 2) Machine Learning based

Supervised System Training Data Feature Engineering Learner Apply on Test Data Evaluate Evaluation metrics: Accuracy, Recall, Precision, F-Score SVM, Naïve Bayes, MaxEnt, Ensemble etc Things to consider: Select suitable domain: product, travel, politics, movies etc Select the text granularity Popular features: Term presence/term frequency, unigram/bigram , adjectives

Supervised System: Our System Existing approaches do not consider ‘sense/meaning’ of the word However, a word may have: 1 2 3 sentiment bearing and non sentiment bearing senses senses with opposing polarity been abstracted Bag-of-words features - Pang et al. (2002), Martineau & Finn(2009), Paltoglou &Thelwall (2010) Syntactic features - Matsumoto et al. 2005, Kennedy & Inkpen (2006), Whitelaw et al. (2005) “He speaks a vulgar language.” “Now that's real crude behavior!” “Her face fell when she heard that she had been fired.” “The fruit fell from the tree.” “The snake bite proved to be deadly for the young boy.” “Shane Warne is a deadly spinner.”

Lexical space v/s sense space Image source: Wikimedia commons Supervised System: Our System Lexical space v/s sense space There are also fire-pits available if you want to have a bonfire with your friends . There are also_347757 fire_pits_19147259 available_4203394 if you want_21808093 to have a bonfire_17203241 with your friends_19962226 . Manually annotated Senses (M) Automatically annotated Senses (I) Lexical Space Sense Space Word Sense Disambiguation (WSD) fire_pits : 19147259 (1: POS identifier : Noun, 9147259: Wordnet Synset offset)

Supervised System: Our Approach Image source: Wikimedia commons Supervised System: Our Approach Manual Annotation A WSD Engine Corpus Automatic Sense-annotated Corpus Classifier Training Manual Sense-annotated Corpus Classifier W Only-sense Filter Classifier Training Only-sense Filter Classifier Training Classifier Training Classifier W+S(I) Classifier Training Classifier W+S(M) Classifier I Classifier M

Results: Overall Classification Image source: Wikimedia commons Results: Overall Classification Feature Representation Accuracy Pos Precision Neg Precision Pos Recall Neg Recall W 84.90 84.95 84.92 85.19 84.60 M 89.10 91.50 87.07 85.18 91.24 W+S(M) 90.20 92.02 88.55 87.71 92.39 I 85.48 87.17 83.93 83.53 87.46 W+S(I) 86.08 85.87 86.38 86.69 85.46 Senses give better overall accuracy Negative Recall increases

Motivation & Introduction Outline Motivation & Introduction Applications Cross-lingual SA Cross-domain SA Opinion Spam SA for tweets Need of SA: Why is SA needed? Variants of SA: What forms does it exist in? Challenges of SA: Why is SA not trivial? Approaches to SA SA @ CFILT Basics: What is classification Approaches: What are ways to do SA 1) Machine Learning based 2) Rule based

Cross-lingual SA Multilingual content on the internet growing How can the sentiment it carries be identified? Can we take help of the ‘rich cousin’ English? Sentiment Analysis System Sentiment Analysis System English document Hindi document Sentiment Label

Alternatives to Cross-lingual SA Strategies for SA for target language Use corpus in target language Translate to a ‘rich’ source language Develop resources for target language

Motivation & Introduction Outline Motivation & Introduction Applications Cross-lingual SA Cross-domain SA Opinion Spam SA for tweets Need of SA: Why is SA needed? Variants of SA: What forms does it exist in? Challenges of SA: Why is SA not trivial? Approaches to SA SA @ CFILT Basics: What is classification Approaches: What are ways to do SA 1) Machine Learning based 2) Rule based

Domain-dependence of words ‘deadly’ It was one deadly match! There are some deadly poisonous snakes in the jungles of Amazon.

General Approach Retain the ‘common-to-all-domain’ words Learn only the ‘special domain’ words Domain differences can be substantial

Motivation & Introduction Outline Motivation & Introduction Applications Cross-lingual SA Cross-domain SA Opinion Spam SA for tweets Need of SA: Why is SA needed? Variants of SA: What forms does it exist in? Challenges of SA: Why is SA not trivial? Approaches to SA SA @ CFILT Basics: What is classification Approaches: What are ways to do SA 1) Machine Learning based 2) Rule based

Opinion spam: A side-effect of UGC Reviews contain rich user opinions on products and services Anyone can write anything on the Web No quality control Result Incentives Positive opinion -> Financial gain for organization Low quality reviews, review spam / opinion Spam.

Different types of spam reviews Reference : [Jindal et al, 2008] Different types of spam reviews Type 1 (untruthful opinions) Type 2 (reviews on brands only) Type 3 (non-reviews) Advertisements Other irrelevant reviews containing no opinions e.g. questions, answers and random text Giving undeserving reviews to some target objects in order to promote/demote the object hyper spam - undeserving positive reviews defaming spam - malicious negative reviews DUPLICATES No comment on the product Comments on brands, manufacturer or sellers of the product Duplicates: Bala to say: “ Same user, multiple websites. Multiple users, same website. Multiple users, multiple websites.” It’s from nikon, what more you want.. Although you should not expect prompt shippin. (It took 3 weeks and several e-mails before I received my order.) I would order again from this merchant, just because the price was right - http://www.pricegrabber.com

Motivation & Introduction Outline Motivation & Introduction Applications Cross-lingual SA Cross-domain SA Opinion Spam SA for tweets Need of SA: Why is SA needed? Variants of SA: What forms does it exist in? Challenges of SA: Why is SA not trivial? Approaches to SA SA @ CFILT Basics: What is classification Approaches: What are ways to do SA 1) Machine Learning based 2) Rule based

Challenges with tweets Ill-formed Spelling mistakes Informal words/emoticons Extensions of words (‘happppyyyyy’) Vague topics

Motivation & Introduction Outline Motivation & Introduction Applications Cross-lingual SA Cross-domain SA Opinion Spam SA for tweets Need of SA: Why is SA needed? Variants of SA: What forms does it exist in? Challenges of SA: Why is SA not trivial? Approaches to SA SA @ CFILT Twitter based SA, Sense based SA, Cross-Lingual SA and many more….. Basics: What is classification Approaches: What are ways to do SA 1) Machine Learning based 2) Rule based

SA @ CFILT English Twitter Trend Analysis Cross-Domain Sense-based Similarity Metric Discourse Detecting Thwarting Other Languages Indian Languages Cross Lingual Hindi Marathi European Languages French Spanish German

Thank you! & Questions?

Extra Reading- Classifiers Naïve Bayes SVM Committee-based classifiers

Naïve Bayes classifiers Based on Bayes rule Naïve Bayes : Conditional independence assumption

Support vector machines Basic idea Margin “Maximum separating-margin classifier” Support vectors Separating hyperplane : wx+b = 0

Multi-class SVM Multiple SVMs are trained: True/false classifiers for each of the class labels Pair-wise classifiers for the class labels

Combining Classifiers Reference : Scribe by Rahul Gupta, IIT Bombay ‘Ensemble’ learning Use a combination of models for prediction Bagging : Majority votes Boosting : Attention to the ‘weak’ instances Goal : An improved combined model

Boosting (AdaBoost) Total set Classifier learning scheme Error Reference : Scribe by Rahul Gupta, IIT Bombay Total set Classifier learning scheme Error Classifier model M 1 Weighted vote Class Label Training dataset D Classifier model M n Sample D 1 Error Test set Weights of correctly classified instances multiplied by error / (1 – error) If error > 0.5? Initialize weights of instances to 1/d Selection based on weight. May use bootstrap sampling with replacement