1. Sentiment Analysis Balamurali A R IITB-Monash Research Academy
Acknowledgment: Aditya Joshi

2. Artist: Leonardo da Vinci
Image from wikimedia commons
Source: Wikipedia
Smile of Mona Lisa
Is she smiling at all?
Is she happy?
What is she smiling about?
What is she happy about?
Mona Lisa
16th century
Artist: Leonardo da Vinci

3. Sentiment analysis (SA)
Task of tagging text with orientation of opinion
This is a good movie.
This is a bad movie.
The movie is set in
Australia.
Subjective
Objective

4. Sentiment Analysis Robotics NLP Search, Reasoning, Expert Learning
Disciplines which form the core of AI- inner circle
Fields which draw from these disciplines- outer circle.
Planning
Computer
Vision
NLP
Expert
Systems
Robotics
Search,
Reasoning,
Learning
Sentiment Analysis

5. Motivation & Introduction
Outline
Motivation & Introduction
Applications
Approaches to SA
CFILT

6. Motivation & Introduction
Outline
Motivation & Introduction
Applications
Need of SA: Why is SA needed?
Variants of SA: What forms does it exist in?
Challenges of SA: Why is SA not trivial?
Approaches to SA
CFILT

7. User-generated content
Web 2.0 empowers the user of the internet
They are most likely to express their opinion there
Temporal nature of UGC: ‘Live Web’
Can SA tap it?

8. SA: Where? Blogs Review websites Social networks User conversations
Conversations between
users on one of the above
Multiple review websites
offering specific to general-topic
reviews
Some SPs: mouthshut, burrrp,
bollywoodhungama
A website, usually maintained
by an individual with regular
entries of commentary,
descriptions of events.
Some SPs: Blogger, LiveJournal,
Wordpress
Websites
that allow people to
connect with one another
and exchange thoughts

9. SA: How much? Size of blogosphere
Reference :
SA: How much?
Size of blogosphere
Through the ‘eyes’ of the blog trackers
Technorati : million blogs (excluding million blogs in Chinese as counted by a corresponding Chinese Center)
A blog crawler could extract 88 million blog URLs from blogger.com alone
12,000 new weblogs daily

10. SA: How much opinion?
Chart created using :

11. Flavours of SA Subjective/Objective Emotion analysis SA with magnitude
Entity-specific SA
Feature-based SA
Perspectivization
“The camera is the best
in its price range. However,
a pathetically slow interface
ruins it for this cell phone.”
“The Leftists were arrested
yesterday by the police.”
“India defeated England in the
cricket match badly.”
“dude.. just get lost.”
“Whoa! Super!!”
“The movie is good.”
“People say that the movie is good.”
“This movie is awesome.”
“Taj Mahal was constructed by
Shah Jahan in the memory of his
wife Mumtaz.”
“Taj Mahal is a masterpiece
of an architecture and
symbolizes unparalleled beauty.”

12. Challenges of SA Domain dependent Sarcasm Thwarted expressions
Negation
Implicit polarity
Time-bounded
“The camera of the mobile phone is less than one mega-pixel – quite uncommon for a phone of today.”
“This phone allows me to send SMS.”
“This phone has a touch-screen.”
“I did not like the movie.”
“Not only is the movie boring, it is also the biggest waste of producer’s money.”
“Not withstanding the pressure of the public, let me admit that I have loved the movie.”
Sarcasm uses words of
a polarity to represent
another polarity.
Example: “The perfume is so
amazing that I suggest you wear it
with your windows shut”
the sentences/words that
contradict the overall sentiment
of the set are in majority
Example: “The actors are good,
the music is brilliant and appealing.
Yet, the movie fails to strike a chord.”
Sentiment of a word
is w.r.t. the
domain.
Example: ‘unpredictable’
For steering of a car,
For movie review,

13. SA Challenges: Sample Review 1 (This, that and this)
‘Touch screen’ today signifies
a positive feature.
Will it be the same in the future?
FLY E300 is a good mobile which i purchased recently with lots of hesitation. Since this Brand is not familiar in Market as well known as Sony Ericsson. But i found that E300 was cheap with almost all the features for a good mobile. Any other brand with the same set of features would come around 19k Indian Ruppees.. But this one is only 9k. Touch Screen, good resolution, good talk time, 3.2Mega Pixel camera, A2DP, IRDA and so on... BUT BEWARE THAT THE CAMERA IS NOT THAT GOOD, THOUGH IT FEATURES 3.2 MEGA PIXEL, ITS NOT AS GOOD AS MY PREVIOUS MOBILE SONY ERICSSION K750i which is just 2Mega Pixel. Sony ericsson was excellent with the feature of camera. So if anyone is thinking for Camera, please excuse. This model of FLY is not apt for you.. Am fooled in this regard.. Audio is not bad, infact better than Sony Ericsson K750i. FLY is not user friendly probably since we have just started to use this Brand.
Comparing old products
The confused conclusion
From:

14. SA Challenges: Sample Review 2
 Hi,    I have Haier phone.. It was good when i was buing this phone.. But I invented  A lot of bad features by this phone those are It’s cost is low but Software is not good and Battery is very bad..,,Ther are no signals at out side of the city..,, People can’t understand this type of software..,, There aren’t features in this phone, Design is better not good..,, Sound also bad..So I’m not intrest this side.They are giving heare phones it is good. They are giving more talktime and validity these are  also good.They are giving colour screen at display time it is also good because other phones aren’t this type of feature.It is also low wait.
Lack of punctuation marks,
Grammatical errors
Wait.. err.. Come again
From:

15. Motivation & Introduction
Outline
Motivation & Introduction
Applications
Need of SA: Why is SA needed?
Variants of SA: What forms does it exist in?
Challenges of SA: Why is SA not trivial?
Approaches to SA
CFILT
Basics: What is classification
Approaches: What are ways to do SA

16. Task Definition
Marking reviews as positive or negative at the document level
Lexicon-based classifiers
ML-based classifiers

17. What is classification?
A machine learning task that deals with identifying the class to which an instance belongs
A classifier performs classification
Classifier
SVM Classifier
Naïve Bayes Classifier
MaxEnt Classifier
Combination
( Perceptive inputs )
( Age, Marital status,
Health status, Salary )
( Textual features : Ngrams )
Test instance
Attributes
(a1, a2,… an)
Discrete-valued
Class label
Category of document? {Politics, Science, Biology}
Steer? { Left, Straight, Right }
Issue Loan? {Yes, No}

18. Classification learning
Training
phase
Testing
phase
Learning the classifier
from the available data
‘Training set’
(Labeled)
Testing how well the classifier
performs
‘Testing set’

19. Testing phase Methods: Holdout (2/3rd training, 1/3rd testing)
Cross validation (n – fold)
Divide into n parts
Train on (n-1), test on last
Repeat for different permutations

20. Approaches to SA and Text granularity
Based on text granularity
Document level
Sentence level
Phrase level
Word level
………..Approaches to SA will differ

21. Machine Learning based
Generic Approaches
Document level sentiment classifier SA
Machine Learning based
Supervised
Unsupervised
Rule based
Lexicon based

22. Motivation & Introduction
Outline
Motivation & Introduction
Applications
Need of SA: Why is SA needed?
Variants of SA: What forms does it exist in?
Challenges of SA: Why is SA not trivial?
Approaches to SA
CFILT
Basics: What is classification
Approaches: What are ways to do SA
1) Rule based
2) Machine Learning based

23. Rule based System: Resources for SA
SentiWordNet
WordNet synsets marked with three types of scores: positive, negative, objective
I am feeling happy.
I am feeling happy.

24. Seed-set expansion in SWN
also-see
Seed words
antonymy Ln Lp
The sets at the end of kth step are called Tr(k,p) and Tr(k,n)
Tr(k,o) is the set that is not present in Tr(k,p) and Tr(k,n)

25. Building SentiWordnet
Classifier alternatives used: Rocchio (BowPackage) & SVM(LibSVM)
Different training data based on expansion
POS –NOPOS and NEG-NONEG classification
Total eight classifiers
For different combinations of k and classifiers
Synsets not in the expanded seed set are used as test synsets
Score is average of scores returned by the classifiers 25 25

26. Rule based System: An Example
C-FEEL-IT
An entity-based opinion search engine on Twitter
How it works?
User enter a search string to get its “public vibe”
Tweets are fetched based on search string
Based on sentiment lexicon, mark each tweet with sentiment score using majority rule
Categorize each tweet into sentiment categories using a threshold value

27. C-FeeL-IT: Preprocessing and heuristics
Feeds from twitter used to obtain:
50 tweets
In English
About the keyword
Normalization done using:
Mapping between chat lingo to dictionary words1
Mapping between emoticons and direct sentiment prediction1
Extensions of words replaced by contracted forms
Negation handling 1

28. C-FeeL-IT: Resources used
•SentiWordNet  (Andrea & Sebastani,2006)
•Subjectivity clues  (Weibi et al, 2004)
•Taboada (Taboada & Grieve, 2004)
•Inquirer (Stone et al, 1966)
Andrea Esuli and Fabrizio Sebastiani SentiWordNet: A publicly available lexical resource for opinion mining. In Proceedings of LREC-06, Genova, Italy
Janyce Wiebe, Theresa Wilson, Rebecca Bruce, Matthew Bell, and Melanie Martin Learning subjective language. Computional Linguistics, 30:277–308, September.
Maite Taboada and Jack Grieve Analyzing Appraisal Automatically. In Proceedings of the AAAI
Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications, pages 158–161, Stanford, US.
Philip J. Stone, Dexter C. Dunphy, Marshall S. Smith, and Daniel M. Ogilvie The General Inquirer: A Computer Approach to Content Analysis. MIT Press.
Reference given in Notes section

29. C-FeeL-IT: Demo
Available at:

30. Motivation & Introduction
Outline
Motivation & Introduction
Applications
Need of SA: Why is SA needed?
Variants of SA: What forms does it exist in?
Challenges of SA: Why is SA not trivial?
Approaches to SA
CFILT
Basics: What is classification
Approaches: What are ways to do SA
1) Rule based
2) Machine Learning based

31. Supervised System
Training Data
Feature Engineering
Learner
Apply on Test Data
Evaluate
Evaluation metrics: Accuracy, Recall, Precision, F-Score
SVM, Naïve Bayes, MaxEnt, Ensemble etc
Things to consider:
Select suitable domain: product, travel, politics, movies etc
Select the text granularity
Popular features: Term presence/term frequency, unigram/bigram ,
adjectives

32. Supervised System: Our System
Existing approaches do not consider ‘sense/meaning’ of the word
However, a word may have: 1 2 3
sentiment bearing and non sentiment bearing senses
senses with opposing polarity
been abstracted
Bag-of-words features - Pang et al. (2002), Martineau & Finn(2009), Paltoglou &Thelwall (2010)
Syntactic features - Matsumoto et al. 2005, Kennedy & Inkpen (2006), Whitelaw et al. (2005)
“He speaks a vulgar language.”
“Now that's real crude behavior!”
“Her face fell when she heard that she had been fired.”
“The fruit fell from the tree.”
“The snake bite proved to be deadly for the young boy.”
“Shane Warne is a deadly spinner.”

33. Lexical space v/s sense space
Image source: Wikimedia commons
Supervised System: Our System
Lexical space v/s sense space
There are also fire-pits available if you want to have a bonfire with your friends .
There are also_ fire_pits_ available_ if you want_ to have a bonfire_ with your friends_
Manually annotated Senses (M)
Automatically annotated Senses (I)
Lexical Space Sense Space
Word Sense Disambiguation (WSD)
fire_pits :
(1: POS identifier : Noun, : Wordnet Synset offset)

34. Supervised System: Our Approach
Image source: Wikimedia commons
Supervised System: Our Approach
Manual
Annotation
A WSD Engine
Corpus
Automatic Sense-annotated Corpus
Classifier Training
Manual Sense-annotated Corpus
Classifier W
Only-sense
Filter
Classifier Training
Only-sense
Filter
Classifier Training
Classifier Training
Classifier W+S(I)
Classifier Training
Classifier W+S(M)
Classifier I
Classifier M

35. Results: Overall Classification
Image source: Wikimedia commons
Results: Overall Classification
Feature
Representation
Accuracy
Pos Precision
Neg Precision
Pos Recall
Neg Recall W
84.90
84.95
84.92
85.19
84.60 M
89.10
91.50
87.07
85.18
91.24
W+S(M)
90.20
92.02
88.55
87.71
92.39 I
85.48
87.17
83.93
83.53
87.46
W+S(I)
86.08
85.87
86.38
86.69
85.46
Senses give better overall accuracy
Negative Recall increases

36. Motivation & Introduction
Outline
Motivation & Introduction
Applications
Cross-lingual SA
Cross-domain SA
Opinion Spam
SA for tweets
Need of SA: Why is SA needed?
Variants of SA: What forms does it exist in?
Challenges of SA: Why is SA not trivial?
Approaches to SA
CFILT
Basics: What is classification
Approaches: What are ways to do SA
1) Machine Learning based
2) Rule based

37. Cross-lingual SA Multilingual content on the internet growing
How can the sentiment it carries be identified?
Can we take help of the ‘rich cousin’ English?
Sentiment
Analysis
System
Sentiment
Analysis
System
English
document
Hindi
document
Sentiment Label

38. Alternatives to Cross-lingual SA
Strategies for SA for target language
Use corpus in target language
Translate to a ‘rich’ source language
Develop resources for target language

39. Motivation & Introduction
Outline
Motivation & Introduction
Applications
Cross-lingual SA
Cross-domain SA
Opinion Spam
SA for tweets
Need of SA: Why is SA needed?
Variants of SA: What forms does it exist in?
Challenges of SA: Why is SA not trivial?
Approaches to SA
CFILT
Basics: What is classification
Approaches: What are ways to do SA
1) Machine Learning based
2) Rule based

40. Domain-dependence of words
‘deadly’
It was one deadly match!
There are some deadly poisonous snakes in the jungles of Amazon.

41. General Approach Retain the ‘common-to-all-domain’ words
Learn only the ‘special domain’ words
Domain differences can be substantial

42. Motivation & Introduction
Outline
Motivation & Introduction
Applications
Cross-lingual SA
Cross-domain SA
Opinion Spam
SA for tweets
Need of SA: Why is SA needed?
Variants of SA: What forms does it exist in?
Challenges of SA: Why is SA not trivial?
Approaches to SA
CFILT
Basics: What is classification
Approaches: What are ways to do SA
1) Machine Learning based
2) Rule based

43. Opinion spam: A side-effect of UGC
Reviews contain rich user opinions on products and services
Anyone can write anything on the Web
No quality control
Result
Incentives
Positive opinion ->
Financial gain for
organization
Low quality reviews,
review spam / opinion
Spam.

44. Different types of spam reviews
Reference : [Jindal et al, 2008]
Different types of spam reviews
Type 1 (untruthful opinions)
Type 2 (reviews on brands only)
Type 3 (non-reviews)
Advertisements
Other irrelevant reviews containing no opinions
e.g. questions, answers and random text
Giving undeserving reviews to some
target objects in order
to promote/demote the object
hyper spam - undeserving positive reviews
defaming spam - malicious negative reviews
DUPLICATES
No comment on the product
Comments on brands, manufacturer or
sellers of the product
Duplicates: Bala to say: “ Same user, multiple websites. Multiple users, same website. Multiple users, multiple websites.”
It’s from nikon, what more you want..
Although you should not expect prompt shippin.
(It took 3 weeks and several s before I received my order.)
I would order again from this merchant,
just because the price was right -

45. Motivation & Introduction
Outline
Motivation & Introduction
Applications
Cross-lingual SA
Cross-domain SA
Opinion Spam
SA for tweets
Need of SA: Why is SA needed?
Variants of SA: What forms does it exist in?
Challenges of SA: Why is SA not trivial?
Approaches to SA
CFILT
Basics: What is classification
Approaches: What are ways to do SA
1) Machine Learning based
2) Rule based

46. Challenges with tweets
Ill-formed
Spelling mistakes
Informal words/emoticons
Extensions of words (‘happppyyyyy’)
Vague topics

47. Motivation & Introduction
Outline
Motivation & Introduction
Applications
Cross-lingual SA
Cross-domain SA
Opinion Spam
SA for tweets
Need of SA: Why is SA needed?
Variants of SA: What forms does it exist in?
Challenges of SA: Why is SA not trivial?
Approaches to SA
CFILT
Twitter based SA, Sense based SA, Cross-Lingual SA and many more…..
Basics: What is classification
Approaches: What are ways to do SA
1) Machine Learning based
2) Rule based

48. CFILT
English
Twitter
Trend Analysis
Cross-Domain
Sense-based
Similarity Metric
Discourse
Detecting Thwarting
Other Languages
Indian Languages
Cross Lingual
Hindi
Marathi
European Languages
French
Spanish
German

49. Thank you! & Questions?

50. Extra Reading- Classifiers
Naïve Bayes
SVM
Committee-based classifiers

51. Naïve Bayes classifiers
Based on Bayes rule
Naïve Bayes : Conditional independence assumption

52. Support vector machines
Basic idea
Margin
“Maximum separating-margin classifier”
Support vectors
Separating hyperplane : wx+b = 0

53. Multi-class SVM Multiple SVMs are trained:
True/false classifiers for each of the class labels
Pair-wise classifiers for the class labels

54. Combining Classifiers
Reference : Scribe by Rahul Gupta, IIT Bombay
‘Ensemble’ learning
Use a combination of models for prediction
Bagging : Majority votes
Boosting : Attention to the ‘weak’ instances
Goal : An improved combined model

55. Boosting (AdaBoost) Total set Classifier learning scheme Error
Reference : Scribe by Rahul Gupta, IIT Bombay
Total set
Classifier
learning
scheme
Error
Classifier
model
M 1
Weighted
vote
Class Label
Training
dataset D
Classifier
model
M n
Sample
D 1
Error
Test
set
Weights of
correctly classified instances multiplied
by error / (1 – error)
If error > 0.5?
Initialize weights of instances to 1/d
Selection based on weight. May use bootstrap sampling with replacement