1. Manual Subjectivity Analysis

2. Preliminaries What do we mean by subjectivity?
The linguistic expression of somebody’s emotions, sentiments, evaluations, opinions, beliefs, speculations, etc.
Wow, this is my 4th Olympus camera.
Staley declared it to be “one hell of a collection”.
Most voters believe that he's not going to raise their taxes
EUROLAN July 30, 2007

3. Foci of this Talk
Lower-level linguistic expressions rather than whole sentences or documents
Developing an understanding of the problem rather than trying to implement a particular solution
EUROLAN July 30, 2007

4. Corpus Annotation Wiebe, Wilson, Cardie 2005 Annotating Expressions of Opinions and Emotions in Language Later Version: Wilson PhD 2008 (1 slide later) Word senses – is that in these slides? Bing Liu’s annotations?
The corpus annotation scheme we follow is presented in the reference above. This and other references mentioned in this talk are
Listed in the bibliography that comes with the slides.
EUROLAN July 30, 2007

5. Overview
Fine-grained: expression-level rather than sentence or document level
The photo quality was the best that I have seen in a camera.
Annotate
expressions of opinions, evaluations, emotions, beliefs
material attributed to a source, but presented objectively
EUROLAN July 30, 2007

6. Overview
Opinions, evaluations, emotions, speculations are private states.
They are expressed in language by subjective expressions.
Private state: state that is not open to objective
observation or verification.
Quirk, Greenbaum, Leech, Svartvik (1985). A
Comprehensive Grammar of the English Language.
EUROLAN July 30, 2007

7. Overview Focus on three ways private states are expressed in language
Direct subjective expressions
Expressive subjective elements
Objective speech events
Though we use the term opinion for accessibility, the goal will be to identify not only opinions, per se, but also evaluations, emotions, ….
EUROLAN July 30, 2007

8. Direct Subjective Expressions
Direct mentions of private states
The United States fears a spill-over from the anti-terrorist campaign.
Private states expressed in speech events
“We foresaw electoral fraud but not daylight robbery,” Tsvangirai said.
First, there are direct subjective expressions. They directly refer to a private state.
We also annotate cases where somebody is reported as talking about a private state of their own.
EUROLAN July 30, 2007

9. Expressive Subjective Elements [Banfield 1982]
“We foresaw electoral fraud but not daylight robbery,” Tsvangirai said
The part of the US human rights report about China is full of absurdities and fabrications
Second, there are what we call expressive subjective elements.
Expressions such as “daylight robbery” do not themselves refer to mental activity or acts of communication. But they are understood as implying a particular mental state of somebody.
Compare:
“We foresaw difficulties with the electoral process but not to this extent”, Tsvangirai said.
The part of the US human rights report about China contains many statements that we were unable to verify.
EUROLAN July 30, 2007

10. Objective Speech Events
Material attributed to a source, but presented as objective fact
The government, it added, has amended the Pakistan Citizenship Act 10 of 1951 to enable women of Pakistani descent to claim Pakistani nationality for their children born to foreign husbands.
Finally, we also annotate reports of speech that present a statement in objective form.
EUROLAN July 30, 2007

11. EUROLAN July 30, 2007

12. Nested Sources
“The report is full of absurdities,’’ Xirao-Nima said the next day.
Our annotations also take into account who the source of an opinion is.
To do this properly, we keep track of the layers of speech and thought in the text.
Consider the first example  READ “The report is full of absurdities” , he continued.
This whole sentence appeared in a newspaper. We know that some writer for the newspaper wrote it but we also know that that person may or may not share the opinion of the person referred to by ‘he’ . So we cannot attribute ‘full of absurdities’ to the writer. We also cannot simply attribute ‘full of absurdities’ directly and only to ‘he’: we have never spoken to that person! So we should record the whole chain of information transmission.
EUROLAN July 30, 2007

13. Nested Sources
(Writer)
“The report is full of absurdities,’’ Xirao-Nima said the next day.
EUROLAN July 30, 2007

14. Nested Sources
(Writer, Xirao-Nima)
“The report is full of absurdities,’’ Xirao-Nima said the next day.
EUROLAN July 30, 2007

15. Nested Sources
(Writer Xirao-Nima)
(Writer Xirao-Nima)
“The report is full of absurdities,’’ Xirao-Nima said the next day.
EUROLAN July 30, 2007

16. Nested Sources
(Writer)
(Writer Xirao-Nima)
(Writer Xirao-Nima)
“The report is full of absurdities,’’ Xirao-Nima said the next day.
EUROLAN July 30, 2007

17. “The report is full of absurdities,” Xirao-Nima said the next day.
Objective speech event
anchor: the entire sentence
source: <writer>
implicit: true
Direct subjective
anchor: said
source: <writer, Xirao-Nima>
intensity: high
expression intensity: neutral
attitude type: negative
target: report
The way we record the private states that are referred to in a text is via annotation frames that have appropriate attributes.
The anchor is the linguistic expression—the stretch of text—that tells us that there is a private state.
The source is the person to whom the private state is attributed. Note that this can be a path.
The target is the content of the private state or what the private state is about.
We also annotate an attitude type. If not specified, it is to be understood as neutral but can be set to positive or negative as required.
Intensity records the intensity of the private state as a whole.
The expression intensity is different, it is just a property of the anchor. Cf. “fumed the next day”
Implicit is used just for the case where e.g. the top-level writer’s speech or writing is not explicitly marked. I.e. where we use the whole sentence as anchor.
Expressive subjective element
anchor: full of absurdities
source: <writer, Xirao-Nima>
intensity: high
attitude type: negative
EUROLAN July 30, 2007

18. “The report is full of absurdities,” Xirao-Nima said the next day.
Objective speech event
anchor: the entire sentence
source: <writer>
implicit: true
Direct subjective
anchor: said
source: <writer, Xirao-Nima>
intensity: high
expression intensity: neutral
attitude type: negative
target: report
The way we record the private states that are referred to in a text is via annotation frames that have appropriate attributes.
The anchor is the linguistic expression—the stretch of text—that tells us that there is a private state.
The source is the person to whom the private state is attributed. Note that this can be a path.
The target is the content of the private state or what the private state is about.
We also annotate an attitude type. If not specified, it is to be understood as neutral but can be set to positive or negative as required.
Intensity records the intensity of the private state as a whole.
The expression intensity is different, it is just a property of the anchor. Cf. “fumed the next day”
Expressive subjective element
anchor: full of absurdities
source: <writer, Xirao-Nima>
intensity: high
attitude type: negative
EUROLAN July 30, 2007

19. “The report is full of absurdities,” Xirao-Nima said the next day.
Objective speech event
anchor: the entire sentence
source: <writer>
implicit: true
Direct subjective
anchor: said
source: <writer, Xirao-Nima>
intensity: high
expression intensity: neutral
attitude type: negative
target: report
The way we record the private states that are referred to in a text is via annotation frames that have appropriate attributes.
The anchor is the linguistic expression—the stretch of text—that tells us that there is a private state.
The source is the person to whom the private state is attributed. Note that this can be a path.
The target is the content of the private state or what the private state is about.
We also annotate an attitude type. If not specified, it is to be understood as neutral but can be set to positive or negative as required.
Intensity records the intensity of the private state as a whole.
The expression intensity is different, it is just a property of the anchor. Cf. “fumed the next day”
Expressive subjective element
anchor: full of absurdities
source: <writer, Xirao-Nima>
intensity: high
attitude type: negative
EUROLAN July 30, 2007

20. “The report is full of absurdities,” Xirao-Nima said the next day.
Objective speech event
anchor: the entire sentence
source: <writer>
implicit: true
Direct subjective
anchor: said
source: <writer, Xirao-Nima>
intensity: high
expression intensity: neutral
attitude type: negative
target: report
The way we record the private states that are referred to in a text is via annotation frames that have appropriate attributes.
The anchor is the linguistic expression—the stretch of text—that tells us that there is a private state.
The source is the person to whom the private state is attributed. Note that this can be a path.
The target is the content of the private state or what the private state is about.
We also annotate an attitude type. If not specified, it is to be understood as neutral but can be set to positive or negative as required.
Intensity records the intensity of the private state as a whole.
The expression intensity is different, it is just a property of the anchor. Cf. “fumed the next day”
Expressive subjective element
anchor: full of absurdities
source: <writer, Xirao-Nima>
intensity: high
attitude type: negative
EUROLAN July 30, 2007

21. “The report is full of absurdities,” Xirao-Nima said the next day.
Objective speech event
anchor: the entire sentence
source: <writer>
implicit: true
Direct subjective
anchor: said
source: <writer, Xirao-Nima>
intensity: high
expression intensity: neutral
attitude type: negative
target: report
The way we record the private states that are referred to in a text is via annotation frames that have appropriate attributes.
The anchor is the linguistic expression—the stretch of text—that tells us that there is a private state.
The source is the person to whom the private state is attributed. Note that this can be a path.
The target is the content of the private state or what the private state is about.
We also annotate an attitude type. If not specified, it is to be understood as neutral but can be set to positive or negative as required.
Intensity records the intensity of the private state as a whole.
The expression intensity is different, it is just a property of the anchor. Cf. “fumed the next day”
Expressive subjective element
anchor: full of absurdities
source: <writer, Xirao-Nima>
intensity: high
attitude type: negative
EUROLAN July 30, 2007

22. “The report is full of absurdities,” Xirao-Nima said the next day.
Objective speech event
anchor: the entire sentence
source: <writer>
implicit: true
Direct subjective
anchor: said
source: <writer, Xirao-Nima>
intensity: high
expression intensity: neutral
attitude type: negative
target: report
The way we record the private states that are referred to in a text is via annotation frames that have appropriate attributes.
The anchor is the linguistic expression—the stretch of text—that tells us that there is a private state.
The source is the person to whom the private state is attributed. Note that this can be a path.
The target is the content of the private state or what the private state is about.
We also annotate an attitude type. If not specified, it is to be understood as neutral but can be set to positive or negative as required.
Intensity records the intensity of the private state as a whole.
The expression intensity is different, it is just a property of the anchor. Cf. “fumed the next day”
Expressive subjective element
anchor: full of absurdities
source: <writer, Xirao-Nima>
intensity: high
attitude type: negative
EUROLAN July 30, 2007

23. “The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities.
Elicit from audience what to mark
EUROLAN July 30, 2007

24. (Writer)
“The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities.
EUROLAN July 30, 2007

25. (writer, Xirao-Nima)
“The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities.
EUROLAN July 30, 2007

26. (writer, Xirao-Nima, US)
“The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities.
EUROLAN July 30, 2007

27. (Writer)
(writer, Xirao-Nima, US)
(writer, Xirao-Nima)
“The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities.
EUROLAN July 30, 2007

28. “The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities.
Objective speech event
anchor: the entire sentence
source: <writer>
implicit: true
Objective speech event
anchor: said
source: <writer, Xirao-Nima>
Direct subjective
anchor: fears
source: <writer, Xirao-Nima, US>
intensity: medium
expression intensity: medium
attitude type: negative
target: spill-over
EUROLAN July 30, 2007

29. The report has been strongly criticized and condemned by
many countries.
EUROLAN July 30, 2007

30. The report has been strongly criticized and condemned by
many countries.
Objective speech event
anchor: the entire sentence
source: <writer>
implicit: true
Direct subjective
anchor: strongly criticized and condemned
source: <writer, many-countries>
intensity: high
expression intensity: high
attitude type: negative
target: report
EUROLAN July 30, 2007

31. As usual, the US state Department published its annual report on human rights practices in world countries last Monday.
And as usual, the portion about China contains little truth and many absurdities, exaggerations and fabrications.
Briefly point out the repetition in “and as usual”
EUROLAN July 30, 2007

32. As usual, the US state Department published its annual report on human rights practices in world countries last Monday.
And as usual, the portion about China contains little truth and many absurdities, exaggerations and fabrications.
Expressive subjective element
anchor : And as usual
source : <writer>
intensity : low
attitude type : negative
Objective speech event
anchor : the entire 1st sentence
source : <writer>
implicit : true
Expressive subjective element
anchor : little truth
source : <writer>
intensity : medium
attitude type : negative
Direct subjective
anchor : the entire 2nd sentence
source : <writer>
implicit : true
intensity : high
expression intensity : medium
attitude type : negative
target : report
Expressive subjective element
anchor : many absurdities, exaggerations,
and fabrications
source : <writer>
intensity : medium
attitude type : negative
EUROLAN July 30, 2007

33. Corpus On my website (in transition due to network migration)
English language versions of articles from the world press (187 news sources)
Themes of the instructions:
No rules about how particular words should be annotated.
Don’t take expressions out of context and think about what they could mean, but judge them as they are used in that sentence.
The annotation scheme that I just described has been used in building a corpus of annotated news articles.
535 documents, 11,114 sentences
The corpus also includes contextual polarity annotations which I haven’t mentioned yet and which we’ll see later.
EUROLAN July 30, 2007

34. Wilson PhD Dissertation 2008
I think people are happy because Chavez has fallen
direct subjective
span: are happy
source: <writer, I, People>
attitude:
inferred attitude
span: are happy because
Chavez has fallen
type: neg sentiment
intensity: medium
target:
target
span: Chavez has fallen
span: Chavez
attitude
type: pos sentiment
span: think
source: <writer, I>
type: positive arguing
span: people are happy because
We have manually annotated data, meaning that humans read and interpreted the texts and then marked it up by hand. Coding; labeling. Our annotated corpus is called the MPQA corpus, and is available here.
The most up to date reference about the corpus is Theresa Wilson’s PhD dissertation, which is available from both of our websites.
MPQA corpus:

35. As usual, the US State Department published its annual report on human rights practices in world countries last Monday.
GATE_objective-speech-event (2, 2) nested-source=w implicit=true []
GATE_agent (46, 108) id=report ['its', 'annual', 'report', 'on', 'human', 'right', 'practice', 'in', 'world', 'country']
And as usual, the portion about China contains little truth and many absurdities, exaggerations and fabrications.
GATE_expressive-subjectivity (128, 140) nested-source=w polarity=neutral intensity=low ['and', 'as', 'usual']
GATE_direct-subjective (128, 128) nested-source=w attitude-link=a100 intensity=high implicit=true []
GATE_target (142, 165) id=t100 ['the', 'portion', 'about', 'china']
GATE_agent (160, 165) id=china ['china']
GATE_attitude (166, 240) intensity=high id=a100 attitude-type=sentiment-neg target-link=t100 ['contain', 'little', 'truth', 'and', 'many', 'absurdity', 'exaggeration', 'and', 'fabrication']
GATE_expressive-subjectivity (175, 187) nested-source=w polarity=negative intensity=medium ['little', 'truth']
GATE_expressive-subjectivity (192, 240) nested-source=w polarity=negative intensity=high ['many', 'absurdity', 'exaggeration', 'and', 'fabrication']

36. Its aim of the 2001 report is to tarnish China's image and exert political pressure on the Chinese Government, human rights experts said at a seminar held by the China Society for Study of Human Rights (CSSHR) on Friday.
GATE_objective-speech-event (248, 248) nested-source=w implicit=true []
GATE_direct-subjective (380, 384) nested-source=w,experts expression-intensity=neutral attitude-link=a110 intensity=medium ['say']
GATE_attitude (248, 357) intensity=medium-high id=a110 attitude-type=sentiment-neg target-link=t2 ['its', 'aim', 'of', 'the', 'report', … 'the', 'chinese', 'government']
GATE_target (259, 274) id=t2 ['the', 'report']
GATE_expressive-subjectivity (281, 288) nested-source=w,experts polarity=negative intensity=medium ['tarnish']
GATE_direct-subjective (252, 255) nested-source=w,experts,report polarity=neutral expression-intensity=medium attitude-link=a120,a130 intensity=medium ['aim']
GATE_attitude (252, 255) intensity=medium id=a120 attitude-type=intention-pos target-link=t3 ['aim']
GATE_target (278, 357) id=t3 ['to', 'tarnish', 'china', "'s", 'image', 'and', 'exert', 'political', 'pressure', 'on', 'the', 'chinese', 'government']
GATE_attitude (252, 288) intensity=medium id=a130 attitude-type=sentiment-neg target-link=t4 ['aim', 'of', 'the', 'report', 'be', 'to', 'tarnish']
GATE_target (289, 294) id=t4 ['china']
GATE_agent (359, 379) nested-source=w,experts id=experts ['human', 'right', 'expert']
GATE_agent (259, 274) nested-source=w,experts,report id=report ['the',’2001’, 'report']

37. Continued on the next slide…
"The United States was slandering China again," said Xirao-Nima, a professor of Tibetan history at the Central University for Nationalities.
GATE_objective-speech-event (475, 475) nested-source=w implicit=true []
GATE_direct-subjective (523, 527) nested-source=w,nima expression-intensity=neutral attitude-link=a140 intensity=high ['say']
GATE_attitude (494, 508) intensity=high id=a140 attitude-type=sentiment-neg target-link=t5 ['be', 'slander']
GATE_target (476, 493) id=t5 ['the', 'unite', 'state']
GATE_expressive-subjectivity (498, 508) nested-source=w,nima polarity=negative intensity=high ['slander']

38. GATE_target (509, 514) id=t6 ['china']
"The United States was slandering China again," said Xirao-Nima, a professor of Tibetan history at the Central University for Nationalities.
GATE_direct-subjective (494, 508) nested-source=w,nima,US polarity=negative expression-intensity=high attitude-link=a150 intensity=high ['be', 'slander']
GATE_attitude (494, 508) intensity=high id=a150 attitude-type=sentiment-neg target-link=t6 ['be', 'slander']
GATE_target (509, 514) id=t6 ['china']
GATE_agent (528, 538) nested-source=w,nima id=nima ['xirao-', 'nima']
GATE_agent (476, 493) nested-source=w,nima,US id=US ['the', 'unite', 'state']
We feel that the “nested source” and “nested target” annotations might be used by a system to recognize nested sources and targets. I feel a project to work on is = recognize when someone mentioning someone else agrees or disagrees with that other person. Pang and Lee have some aspect to that – are these two sentences in agreement with each other? But I want to look at how people talk about other people. Heuristics could be used to create training data – here for example, we have w,nima,US and we have (on previous slide) a negative attitude of nima toward the US. So, this is a mention of someone else where nima does not agree with the US. These are merely heuristic, as you can have both positive and negative attitudes toward the same target in one sentence. But there is something to be done here, I feel. Not sure how much of this to share with the class…

39. These are all the annotations
It shows that these so-called truths are not true at all," said Xirao-Nima
GATE_objective-speech-event (3111, 3111) nested-source=w implicit=true []
GATE_direct-subjective (3170, 3174) attitude-type=negative intensity=high attitude-link=a350,a355 expression-intensity=neutral nested-source=w,nima attitude-toward=report ['say']
GATE_attitude (3111, 3167) intensity=medium-high id=a355 attitude-type=arguing-neg target-link=t101 ['it', 'show', 'that', 'these', 'so-call', 'truth', 'be', 'not', 'true', 'at', 'all']
GATE_attitude (3111, 3167) intensity=high id=a350 attitude-type=sentiment-neg target-link=t101 ['it', 'show', 'that', 'these', 'so-call', 'truth', 'be', 'not', 'true', 'at', 'all']
GATE_target (3125, 3147) id=t101 ['these', 'so-call', 'truth']
GATE_expressive-subjectivity (3131, 3147) nested-source=w,nima polarity=negative intensity=medium ['so-call', 'truth']
GATE_expressive-subjectivity (3152, 3167) nested-source=w,nima polarity=negative intensity=high ['not', 'true', 'at', 'all']
GATE_agent (3175, 3185) nested-source=w,nima ['xirao-', 'nima']

40. Word senses
Senses
EUROLAN July 30, 2007

41. Senses
The challenge of word senses is that they are not always uniformly subjective or objective. Brilliant, for instance, has objective uses related to the physical perception of sight and sound.
EUROLAN July 30, 2007

42. Non-subjective senses of brilliant
Method for identifying brilliant material in paint - US Patent
Halley shines in a brilliant light.
In a classic pasodoble, an opening section in the minor mode features a brilliant trumpet melody, while the second section in the relative major begins with the violins.
Consider these examples. Read 
EUROLAN July 30, 2007