1. Automatic Domain Adaptive Sentiment Analysis Phase 1 Justin Martineau

2. Outline Introduction Problem Definition Thesis Statement Motivation Background and Related Work Challenges Approaches Research Plan Approach Evaluation Timeline Conclusion

3. Problem Definition Sentiment Analysis is the automatic detection and measurement of sentiment in text segments by machines. 3 Sub Tasks Objective vs. Subjective Topic Detection Positive vs. Negative Commonly applied to web data Very Domain Dependent 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

4. Sentiment Analysis Example 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

5. Thesis Statement This dissertation will develop and evaluate techniques to discover and encode domain- specific, domain-independent, and semantic knowledge to improve both single and multiple domain sentiment analysis problems on textual data given low labeled data conditions. 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

6. Motivation: Private Sector Market Research Surveys Focus Groups Feature Analysis Customer targeting (Free samples etc…) Consumer Sentiment Search Compare pros and cons Overall opinion of products/services 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

7. Motivation: Public Sector Political Alternative Polling Determine popular support for legislation Choose campaign issues National Security Detect individuals at risk for radicalization Determine local sentiment about US policy Determine local values and sentimental icons Portray actions positively using local flavor Public Health Detect potential suicide victims Detect mentally unstable people 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

8. Challenges Text Representation Unedited Text Sentiment Drift Negation Sarcasm Sentiment Target Identification Granularity Domain Dependence 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

9. Domain Dependence 1 Domain Dependent Sentiment The same sentence can mean two very different things in different domains Ex: “Read the book.” <= Good for books, bad for movies Ex: “Jolting, heart pounding, You’re in for one hell of a bumpy ride!” Good for movies and books, bad for cars. Sentimental word associations change with domain Fuzzy cameras are bad, but fuzzy teddy bears are good. Big trucks are good, but big iPods are bad. Bad is bad, but bad villains are good. 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

10. Domain Dependence 2 Endless Possibilities 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

11. Domain Dependence 3 Organization and Granularity 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

12. Theory of the Three Signals Authors communicate messages using three types of signals Domain-Specific Signals Domain-Independent Signals Semantic Signals More specific signals are generally more powerful than more generic signals 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

13. Domain-Specific Signals Dependent on problem and domain Considered more useful by readers Tells what is good or bad about topic Domain knowledge determines sentiment orientation Very strong in context, but weak or misleading out of context Can cause over generalization error when overvalued New domain-specific signal words are ignored in CDT Fuzzy teddy bears Sharp pictures Sharp knives Smooth rides New ideas Fast servers Fast cars Slow roasted burgers Slow motion Small cameras Big cars 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

14. Proposed Approach Sentiment Search is more than just a classification problem Detecting and Using the three signals Dynamic Domain Adapting Classifiers Generic Feature Detection using unlabeled data Semantic Feature Spaces 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

15. Dynamic Domain Adapting Classifiers A (preferably domain-independent) model is built using computationally intense algorithms before query time on a set of labeled data. Users interact at a query box level Query results define the domain of interest Domain specific adaptations are calculated compares how the domain of interest is different from known cases uses semantic knowledge about word senses and relations must be fast algorithm: users are waiting Domain specific adaptations are woven into the domain independent model resulting model is temporary used to classify documents as positive, negative, or objective Sentimental search results are processed for significant components and presented for human consumption 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

16. Overview Sentiment Classifier Query Results Define a new Domain Context Specific Model Lucene Index Query Dynamic Domain Adapter General Model + - Labeled data from known domain Semantic Knowledge Sentimental Search Results Component Analysis Business Intelligence Key: User Level, Source Data, Knowledge,Labeled Data Algorithms, Search Results 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

17. Subjective Context Scoring Multiply: PMI(Word,Context) IDF Co-occurance with know generic sentiment seed words times their bias (From movie reviews) Seeds: bad,worst,stupid,ridiculous, terrible,poorly great,best,perfect,wonderful, excellent,effective

18. Rocchio Baseline Rocchio - Query Expansion algorithm for search Similar goals to ours, find more relevant words Does not account for sentiment The new query is a weight sum of Matching document vectors Query vector Non-matching document vectors (negative value).

19. Papa John’s According to TFIDF

20. Papa John’s According to Subjective Context

21. George Bush According to TFIDF

22. George Bush According to Subjective Context

23. iPod according to Rocchio

24. iPod according to TFIDF Positive Sentiment In Movie Reviews Negative Sentiment in Movie Reviews

25. Sentimental Context Components: PMI(Word,Context) TF IDF Log( Actual Co Occur of Word,Seed, context / Prob by chance) Values: Abnormality to other docs Popular words in context Rare words in the corpus Words that occur with sentiment words in the query documents

26. iPod according to Sentimental Context

27. iPod Nike according to Sentimental Context

28. iPod+Nike According to Apple

29. iPod Audio according to Sentiment Context

30. iPod Shuffle According to Sentiment Context

31. iPod Warranty According to Sentimental Context

32. iPod Battery according to Sentiment Context

33. iPod nano battery According to Sentimental Context

34. Google Hits (Battery Related): iPod battery good ~ 13.5 Mill iPod battery bad ~ 900 K iPod nano battery good ~ 3 Mill iPod nano battery bad ~ 785 K iPod shuffle battery good ~ 1.6 Mill iPod shuffle battery bad ~ 230 K iPod shuffle battery price good ~ 2.6 Mill (not a typo) iPod shuffle battery price bad ~ 230 K iPod battery price good ~ 13.5 Mill iPod battery price bad ~ 850 K iPod nano battery price good ~ 3 Mill iPod nano battery price bad ~ 785 K

36. Summary Interesting problem with many potential applications Domain dependence is the core challenge The keys to success are: Vast quantities of unlabeled data Semantic knowledge from freely available sources Semantics must guide and influence but not overrule the statistics 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion

37. Questions?

38. BACKUP SLIDES

39. PMI - Pointwise Mutual Information a.k.a. Specific Mutual Information Do 2 variables occur more often with each other than chance? 1. Intro - 2. Related Work - 3. Research Plan - 4. Conclusion