I NFORMATION R ETRIEVAL S ENTIMENT A NALYSIS N AMED E NTITY R ECOGNITION Mesut KAYA

O UTLINE Introduction (Information Retrieval) Sentiment Analysis Named Entity Recognition

I NFORMATION R ETRIEVAL The area of study concerned with searching for documents, for information within the documents, and for metadata about documents.

S ENTIMENT A NALYSIS Bo Pang & Lilian Lee and Shivakumar Vaithyanathan (2002). Thumbs up? Sentiment Classification using Machine Learning Techniques. Proceedings of the Conference on Empirical Methods in Natural Language Processing ( EMNLP ). pp.79–86.

O UTLINE Introduction Application Areas, Domains Difficulties Paper

I NTRODUCTION Opinions are important, because whenever people need to make a decision they want to hear others’ opinions. Sentiment analysis or opinion mining is Natural Language Processing, Computational Linguistics and Text Mining Technique: the computational study of opinions, sentiments and emotions expressed in text. It aims to determine the attitude of a speaker or a writer with respect to some topic.

I NTRODUCTION Sentiment Classification: doc-level Classify a document (e.g review) based on the overall sentiment expressed by opinion holder. Sentence Level Sentiment Classification: 1. Subjectivity Classification: Subjective or Objective Objective: e.g “I bought an iPhone a few days ago” Subjective: e.g “It is such a nice phone” 2. Sentiment Classification: Positive or Negative Positive: e.g “It is such a nice phone” Negative: e.g “This phone is not useful”

A PPLICATION A REAS, D OMAINS Business Intelligence Applications e.g: Financial Sentiment Analysis. Recommender Systems: To summarize user input(comments) Message Filtering Politics News Blogs

D IFFICULTIES In general sentiment analysis is context sensitive(domain dependent). e.g: “go read the book” may be positive for book domain but negative for a movie domain The order in which different opinions are presented can result in a completely opposite overall sentiment analysis. e.g: This film should be brilliant. It sounds like a great plot, the actors are first grade, and the supporting cast is good as well, and Stallone is attempting to deliver a good performance. However, it can’t hold up.

P APER Trying 3 different Machine Learning methods (Naive Bayes Classifier, Maximum Entropy, SVM) to classify movie reviews as positive or negative.

D ATA USED Movie reviews are used because: There are large online collections of such reviews Reviewers often summarize their overall sentiment with ratings. So no need to hand-label data for supervised learning. (The data is converted into positive,negative, neutral from stars)

M ETHOD To compare automatic methods(ML) with rule based methods. First, they simply produce a list of certain words, people tend to express strong sentiments. They conclude that, they must explore corpus- based techniques rather than relying on prior intuitions. So they use Machine Learning methods to compare automatic methods with rule-based method.

M ACHINE L EARNING M ETHODS Goal: Examine whether it sufficies to treat sentiment classification simply as a special case of topic-based categorization(pos - neg) or special sentiment categorization methods need to be developed. Naive Bayes Classification Maximum Entropy Classification SVM

D ATA P REPARATION Let {f1,…,fm} be a predefined set of m features that can appear in a document d; such as “still”. Let n i (d) be number of times f i occurs in document d. Then d is represented as: d := (n1(d), n2(d),..., nm(d))

N AIVE B AYES C LASSIFICATION Assign a document d to the class c where: c = argmaxc P(c|d) Add one smoothing is used. Naive Bayes is optimal for certain problem classes with highly dependent features.

M AXIMUM E NTROPY C LASSIFICATION Z(d) is a normalization function fi,c is feature/class function for feature fi and class c:

M AXIMUM E NTROPY C LASSIFICATION ƛ i,j : feature weight parameters. Large ƛ i,j means fi is considered a strong indicator for class c. They used 10 iterations of iterative scaling algorithm for parameters.

S UPPORT V ECTOR M ACHINE Let c j € {1,-1} be correct class of document dj such that: α j is obtained by solving a dual optimization problem. dj vectors that α j is greater than 0 are support vectors, since they are only documents contributing to w vector. Classification of test instances consists of determining which side of w’s hyperplane they fall on.

E VALUATION & R ESULTS Usage of unigram and bigram features since ME is expensive in the number of features.

E VALUATION & R ESULTS Naive Bayes worst, SVM best. They have good results nearly 80% accuracy. However, sentiment analysis is easier for reviews. For sentence level sentiment classification hybrid methods must be used. Using ML methods with domain, language specific rule-based methods with intuitions.

N AMED E NTITY R ECOGNITION Jenny Rose Finkel and Christopher D. Manning Nested Named Entity Recognition. In Proceedings of EMNLP, 2009.

O UTLINE Introduction Approaches Problem Domains Current Challenges Paper

I NTRODUCTION Named Entity Recognition is a subtask of IR that seeks to locate and classify atomic elements in text into predefined categories such as the name of persons, organizations, locations, expressions of times, quantities, percentages etc. e.g: “Jim bought 300 shares of Acme Corp. in 2006.” Jim bought 300 shares of Acme Corp. in 2006

A PPROACHES NER systems have been created that use linguistic grammer-based techniques as well as statistical methods. Hand-crafted grammar-based systems typically obtain better precision, but at the cost of lower recall and months of work by experienced computational linguistics. Statistical NER systems typically require a large amount of manually annotated training data.

P ROBLEM D OMAINS Extraction from journalistic articles Processing of military dispatches and reports Automatic content extraction (ACE): weblogs and text transcripts from conversational telephone speech conversations. Entity identification in the moleculer biology, bioinformatics, and medical NLPcommunities. e.g: names of genes and gene products

C URRENT C HALLENGES Despite the high F1 numbers reported on the MUC-7 (newswire articles for North American News Text Corpora) dataset, the problem of Named Entity Recognition is far from being solved. The main efforts are directed to reducing the annotation labor, robust performance across domains, scaling up to fine-grained entity types, dealing with nested named entities.

P APER Jenny Rose Finkel and Christopher D. Manning. Nested Named Entity Recognition. Computer Science Department, Stanford University.

G OAL Nested Named Entity Detection. Many named entities contain other named entities inside them. e.g: “Bank of China”, “University of Washington” both organizations and locations(nested).

M ETHOD Nested Named Entity Recognition as parsing. Represent each sentence as a constituency tree, with each named entity corresponding to a phrase in the tree, along with a root node which connects the entire sentence. Parts of speech are modeled as preterminals, and the words themselves as the leaves.

M ETHOD

Each node is then annotated with both its parent and grandparent labels, which allows the model to learn how entities nest. Trees are binarized in a right-branching manner, and then features are built over the labels, unary rules, and binary rules. First-order horizontal Markovization is used, which allows retaining some information about the previous node in the binarized rule.

M ETHOD Annotated and binarized subtree.

M ETHOD Part of speech tags are jointly modeled with the named entities. The possible part of speech tags are deterimned based on distributional similarity clusters. Because the parts of speech are annotated with the parent (and grandparent) labels, they determine what, if any, entity types a word can be labeled with. Many words, such as verbs, cannot be labeled with any entities.

F EATURES U SED Features over entire entities, features which directly model nested entities, and joint features over entities and parts of speech. 1) Local Named Entity Features: Local named entity features are over the label for a single word. 2) Pairwise Named Entity Features: Pairwise features are over the labels for adjacent words. 3) Embedded Named Entity Features: Embedded named entity features occur in binary rules where one entity is the child of another entity.

F EATURES U SED 4) Whole Entity Features: the entire phrase; the preceding and following word; the preceding and following distributional similarity tags; and the preceding distributional similarity tag with the following word. 5) Local Part of Speech Features 6) Joint Named Entity and POS features: For the joint features POS features are replicated, but included the parent of the POS, which either is the innermost entity type, or would indicate that the word is not in any entities

E XPERIMENTS Two sets of experiments are performed, the first set over biomedical data, and the second over Spanish and Catalan newspaper text. Experiments are designed to show that our model works just as well on outermost entities, the typical NER task, and also works well on nested entities.

E XPERIMENTS

C ONCLUSION A discriminative parsing-based method for nested named entity recognition is presented, which does well on both top-level and nested entities. It is slower than common flat techniques.

R EFERENCES Bo Pang & Lilian Lee and Shivakumar Vaithyanathan (2002). Thumbs up? Sentiment Classification using Machine Learning Techniques. Proceedings of the Conference on Empirical Methods in Natural Language Processing ( EMNLP ). pp.79–86. Jenny Rose Finkel and Christopher D. Manning Nested Named Entity Recognition. In Proceedings of EMNLP, 2009.