1. Determining Negation Scope and Strength in Sentiment Analysis SMC 2011 Paul van Iterson Erasmus School of Economics Erasmus University Rotterdam paulvaniterson@gmail.com Bas Heerschop Erasmus School of Economics Erasmus University Rotterdam basheerschop@gmail.com Flavius Frasincar Erasmus School of Economics Erasmus University Rotterdam frasincar@ese.eur.nl Uzay Kaymak Erasmus School of Economics Erasmus University Rotterdam kaymak@ese.eur.nl October 12, 2011 Alexander Hogenboom Erasmus School of Economics Erasmus University Rotterdam hogenboom@ese.eur.nl

2. Outline Introduction Sentiment Analysis Accounting for Negation Framework Evaluation Conclusions Future Work SMC 2011 2

3. Introduction (1) Need for information monitoring tools for tracking sentiment in today’s complex systems The Web offers an overwhelming amount of textual data, containing traces of sentiment SMC 2011 3

4. Introduction (2) Existing sentiment analysis approaches are based on word frequencies There is a tendency of involving various other aspects of content in automated sentiment analysis Accounting for negation seems promising, but how to model the influence of negation keywords on the conveyed sentiment? SMC 2011 4

5. Sentiment Analysis Sentiment analysis is typically focused on determining the polarity of natural language texts Applications in summarizing reviews, determining a general mood (consumer confidence, politics) Common approach to sentiment analysis: –Creation of lexicon (list of words and their sentiment scores) –Utilization of lexicon to determine sentiment in text Sentiment analysis approaches differ on several distinguishing characteristic features, e.g., –Analysis level and focus –Handling of syntactic variations, amplification, and negation 55 SMC 2011

6. Accounting for Negation (1) Common approach: exploitation of negation keywords Challenge lies in finding the negation scope Sophisticated approaches involve complex rules, compositional semantics, or machine learning Many existing sentiment analysis frameworks use rather simple conceptualizations of negation scope 66 SMC 2011

7. Accounting for Negation (2) Let us consider the following positive sentence: –Example:Luckily, the smelly poo did not leave awfully nasty stains on my favorite shoes! Rest of Sentence (RoS): –Following:Luckily, the smelly poo did not leave awfully nasty stains on my favorite shoes! –Around:Luckily, the smelly poo did not leave awfully nasty stains on my favorite shoes! First Sentiment-Carrying Word (FSW): –Following:Luckily, the smelly poo did not leave awfully nasty stains on my favorite shoes! –Around:Luckily, the smelly poo did not leave awfully nasty stains on my favorite shoes! 77 SMC 2011

8. Accounting for Negation (3) Let us consider the following positive sentence: –Example:Luckily, the smelly poo did not leave awfully nasty stains on my favorite shoes! Next Non-Adverb (NNA): –Following:Luckily, the smelly poo did not leave awfully nasty stains on my favorite shoes! Fixed Window Length (FWL): –Following (3):Luckily, the smelly poo did not leave awfully nasty stains on my favorite shoes! –Around (3):Luckily, the smelly poo did not leave awfully nasty stains on my favorite shoes! 88 SMC 2011

9. Framework (1) Lexicon-based sentence-level sentiment scoring by using SentiWordNet Optional support for sentiment negation Individual words are scored in the range [-1,1] Word scores are used to classify a sentence as positive (1) or negative (-1) 99 SMC 2011

10. Framework (2) Score sentences in test corpus for their sentiment For an arbitrary sentence: –Retrieve all words (simple and compound) –Retrieve each words’ Part-Of-Speech (POS) and lemma –Disambiguate word senses (Lesk algorithm) –Retrieve words’ sentiment scores from lexicon –Negate sentiment scores of negated words, as determined by means of one of the considered approaches, by multiplying the scores with an inversion factor (typically negative) –Calculate sentence score as sum of words’ scores –Classify sentence as either positive (score ≥ 0) or negative (score < 0) 10 SMC 2011

11. Evaluation (1) Implementation in C#, Microsoft SQL Server database, SharpNLP-based POS tagger, WordNet.Net API for lemmatization and word sense disambiguation, SentiWordNet sentiment lexicon Corpus of 930 positive and 1,355 negative manually classified English movie review sentences (60% training set, 40% test set) 11 SMC 2011

12. Evaluation (2) Baseline: sentiment without accounting for negation Alternatives: negation scoping with RoS, FSW, NNA, and FWL (window sizes ranging from 1 to 4) Optimized inversion factor for best alternative to a value in the range [-2, 0] (hill-climbing on training set) 12 SMC 2011

13. Evaluation (3) MethodDirectionWindowInversionAccuracyMacro F1 Baseline--49.9%49.4% FSWFollowing-52.3%52.0% FWLFollowing151.3%51.0% FWLFollowing252.7%52.4% FWLFollowing352.0%51.7% FWLFollowing452.2%52.0% FWLFollowing2-1.2753.5%53.3% 13 SMC 2011

14. Conclusions Recent sentiment analysis methods consider more and more aspects of content other than word frequencies Our corpus-based evaluation of several common negation scoping methods shows that only some perform significantly better than our baseline of not accounting for negation FWL with a window of 2 words following a negation keyword yields the highest increase in accuracy (5.5%) and macro-level F1 (6.2%) compared to the baseline An optimized inversion factor of -1.27 rather than -1 yields an accuracy increase of 7.0% and a macro-level F1 increase of 8.0% compared to the baseline SMC 2011 14

15. Future Work Let the negation scope detection method depend on the position of a negation keyword Deeper understanding of semantics in order to cope with, e.g., context-dependent interpretations Distinct sentiment inversion factors for negated positive and negative words SMC 2011 15

16. Questions? Alexander Hogenboom Erasmus School of Economics Erasmus University Rotterdam P.O. Box 1738, NL-3000 DR Rotterdam, the Netherlands hogenboom@ese.eur.nl SMC 2011 16