1. Experiments of Opinion Analysis On MPQA and NTCIR-6 Yaoyong Li, Kalina Bontcheva, Hamish Cunningham Department of Computer Science University of Sheffield {yaoyong,kalina,hamish}@dcs.shef.ac.uk http://gate.ac.uk/http://gate.ac.uk/ http://nlp.shef.ac.uk/http://nlp.shef.ac.uk/

2. 2(10) Outlines We participated two tracks, English and Chinese corpora. Compared the results on the MPQA corpus and the NTCIR-6 corpus.

3. 3(10) Opinionated Sentence Recognition Uni-gram of token’s lemma and POS tf*idf representation of sentence. SVM with uneven margins as binary classifier.

4. 4(10) Opinion Holder Extraction An information extraction problem. Identify the first token and last token of an opinion holder. Two SVM binary classifiers.

5. 5(10) Experiments on MPQA Corpus Consists of 535 news articles. 360 documents were used for training and other 175 documents for testing.

6. 6(10) Results on MPQA Corpus PrecisionRecallF1 Opinionated sentence 0.6780.9140.779 Opinion holder0.6760.5600.613 There are comparable with the state of the art results published.

7. 7(10) Results on NTCIR-6 English Using the SVM models learned from the MPQA corpus. The following are the official results of the run GATE-1. PrecisionRecallF1 Opinionated sentence 0.3240.9050.477 Opinion holder0.1210.3490.180

8. 8(10) GATE-1 Results Using GATE Evaluation Tools PrecisionRecallF1 Opinionated sentence 0.2930.4960.323 Opinion holder0.1750.3140.183 Results of the opinionated sentence recognition became lower. Results of the opinion holder extraction was a slightly higher.

9. 9(10) Experiments Using NTCIR-6 English Corpus for Training and Testing 300 documents for training, and 139 documents for testing. Just use the annotations of one annotator, in the file “OAT2006 formalrun english a1.csv”. 212 opinion holders (among the 2355 opinion holders) in the file which had no match within the corresponding sentences. We made necessary changes on them to find the text.

10. 10(10) Results Using NTCIR-6 English Corpus for Training and Testing Much improved results by using the NTCIR-6 corpus for training and testing, showing that there really exist differences between the two corpora, Still worse than the results on the MPQA corpus. PrecisionRecallF1 Opinionated sentence 0.6480.6100.628 Opinion holder0.4890.3460.405

11. 11(10) Conclusions SVM with uneven margins obtained state of the art results on the MPQA corpus. On NTCIR corpus, obtained moderate results on opinionated sentence extraction, but poor results on opinion holder. Using NTCIR-6 English corpus for training and testing obtained much improved results, but were still worse than those on MPQA.