Sentiment Analysis of Blogs by Combining Lexical Knowledge with Text Classification (ACM KDD 09’) Prem Melville, Wojciech Gryc, Richard D. Lawrence Date: 07/07/09 Speaker: Hsu, Yu-Wen Advisor: Dr. Koh, Jia-Ling

Outline Introduction Baseline Approaches Pooling Multinomials Empirical Evaluation Conclusion & Future Works

Introduction Most prior work in sentiment analysis use knowledge-based approaches, that classify the sentiment of texts based on dictionaries defining the sentiment-polarity of words, and simple linguistic patterns. Recently, there have been some studies that take a machine learning approach, and build text classifiers trained on documents that have been human-labeled as positive or negative. not adapt well to different domains require much effort in human annotation of documents. 最前的工作情緒分析使用知識為基礎的做法，即分類的情緒文本詞典的基礎上確定的情緒極的話，簡單的語言模式。 最近，已有一些研究，採取機器學習方法，建立文本分類器訓練的文件已被人稱為積極的還是消極的。

We present a new machine learning approach that overcomes these drawbacks by effectively combining background lexical knowledge with supervised learning. We construct a generative model based on a lexicon of sentiment-laden words, and a second model trained on labeled documents. The distributions from these two models are then adaptively pooled to create a composite multinomial Naïve Bayes classifier that captures both sources of information. 我們提出了一種新的機器學習方法，克服了這些缺點的有效結合背景詞彙知識與監督學習。

Baseline Approaches Lexical classification Feature supervision Given a lexicon of positive and negative terms, one straightforward approach to using this information is to measure the frequency of occurrence of these terms in each document. Feature supervision To use a lexicon along with unlabeled data in a semi-supervised learning setting. There have been few approaches to incorporating such background knowledge into learning. 詞彙分類  由於詞彙的積極和消極方面，一個直接的方法，利用這一信息來衡量的頻率，這些條款發生在每一個文件。 功能監管 使用詞典隨著標籤的數據，半監督學習的設置。有幾個辦法，把這種背景知識納入學習

Pooling Multinomials The multinomial Naïve Bayes classifier commonly used for text categorization relies on three assumption (1)documents are produced by a mixture model (2) there is a one-to-one correspondence between each mixture component and class (3) each mixture component is a multinomial distribution of words, i.e., given a class, the words in a document are produced independently of each other. （ 1 ）文件是由一個混合模型 （ 2 ）有一個一對一的對應關係每個混合物的組成部分和階級 （ 3 ）每個混合物的組成部分是一個多項分佈的話，即考慮了一類中，在文件中產生相互獨立。

the likelihood of a document (D) : the probability of the class : the probability of the document given the class the words of a document are generated independently liklihood : the sum of total probability over all mixture components

compute the class membership probabilities of each class the class with the highest likelihood is predicted

*combining probability distributions linear opinion pool K: the number of experts : the probability assigned by expert to word occurring in a document of class :the weights sum to one

logarithmic opinion pool Z : a normalizing constant :weights satisfy restrictions that assure that is a probability distribution weights of individual experts :the error of expert k on the training set

*A generative background-knowledge model Definitions: – the vocabulary, i.e., set of words in our domain – set of positive terms from the lexicon that exists in V – set of negative terms from the lexicon that exists in V – set of unknown terms, i.e. – size of vocabulary, i.e. |V| – number of positive terms, i.e. |P| – number of negative terms, i.e. |N |

Property 1:

Property 2: For this to be true for all values of α and β

Property 3: Since a positive document is more likely to contain a positive term than a negative term, and vice versa, we would like: (r as the polarity level) Property 4: Since each component of our mixture model is a probability distribution, we have the following constraint on the conditional probabilities for each class : 由於積極的文件很可能包含了積極的長期負面任期比，反之亦然，

Empirical Evaluation Data sets Lotus blogs: IBM Lotus collaborative software, 34 positive and 111 negative. Political candidate blogs: Posts focusing on politics about Obama and Clinton, 49 positive and 58 negative. Movie reviews: Apart from the blog data that we generated, we also used the publicly available data of movie reviews, 1000 positive and 1000 negative reviews

Result

Evaluating sensitivity of Pooling Multinomials to the polarity level parameter.

Conclusion we develop an effective framework for incorporating lexical knowledge in supervised learning for text categorization. we apply the developed approach to the task of sentiment classification — extending the state-of-the-art in the field which has focused primarily on using either background knowledge or supervised learning in isolation. 我們建立一個有效的框架，把詞彙知識的監督學習的文本分類。 我們的方法適用於發達國家的任務，情緒分類-延長國家最先進的領域已主要集中在使用背景知識或監督學習孤立。 使用背景知識的監督學習是一個辦法的負擔減輕許多例子標記在目標域中。

Future Works There has been a flurry of recent work in the area of transfer learning that could be applied to extend a background knowledge-based model to incorporate data from different domains. The fundamental challenge in such transfer learning is accounting for the training and test sets being from different distributions.