Mining Fine-grained Argument Elements Adam Wyner Department of Computing Science University of Aberdeen 4 November, 2014 CS4025, Computing Science, University of Aberdeen

Outline 2 Problem. Pragmatic goal – extract argument elements. Approach: – flexible, interactive, incremental, modular, semi- automatic information extraction using – semantic annotations that are relative to – argumentation schemes and that support – semantic queries. Examples CS4025, Wyner, Computing Science, University of Aberdeen

What is an Argument? CS4025, Wyner, Computing Science, University of Aberdeen 3 A claim that is justified (premises) and may have exceptions (premises). A 'strict argument'. – Premises: Socrates is a man. Every man is mortal. – Claim: Therefore, Socrates is mortal. A 'defeasible argument'. – Premises: Bill is a doctor. Doctors know about heart disease. Bill told Jill that she has heart disease. – Claim: Therefore, Jill has heart disease.

What is an Argument? CS4025, Wyner, Computing Science, University of Aberdeen 4 A 'strict argument'. – Premises: Socrates is a man. Every man is mortal. – Claim: Therefore, Socrates is mortal. Given the premises, must accept the claim. A 'defeasible argument'. – Premises: Bill is a doctor. Doctors know about heart disease. Bill told Jill that she has heart disease. – Claim: Therefore, Jill has heart disease. Perhaps Bill does not have all the facts, or he is biased, or he has a second rate medical degree....

Corpora - Where Arguments Appear CS4025, Wyner, Computing Science, University of Aberdeen 5 Consumer websites. Law: policy (policy-making, voting, legislation), Supreme Court transcripts, case based reasoning, regulations. BBC's Have Your Say and Moral Maze. Medical diagnosis. Making plans. Debatepedia, Wikipedia, meeting annotations Social media: web-forums, Twitter, Facebook,....

Purposes and Goals CS4025, Wyner, Computing Science, University of Aberdeen 6 Classify/cluster texts as arguments for or against a claim. Search for and extract argument patterns and components, e.g. premises and conclusions. Reconstruct a complex network of arguments dispersed across texts. Support decision-making (e.g. inference and Abstract Argumentation). Relate arguments to provenance, to trust, to rankings/weightings, to values,.... Audience can: view static argument, contribute, interact with analytic tools.

Example: Solve Knowledge Acquisition Bottleneck for Reasoning CS4025, Wyner, Computing Science, University of Aberdeen 7 Identify and extract arguments from natural language, transform them into a knowledge base (KB), then transform the KB into a formal representation such that we can reason with inconsistent KBs. Argument Pipeline (Wyner, van Engers, Hunter 2010)

Three Stages Graph – Structured or Instantiated AFs CS4025, Wyner, Computing Science, University of Aberdeen 8 Three Stages - Caminada and Wu 2011 Knowledge Acquisition Bottleneck: time, labour, expertise to construct a KB at scale.

Logic-based Instantiated Argumentation (Besnard and Hunter (2009)) CS4025, Wyner, Computing Science, University of Aberdeen 9 An argument is an ordered pair ; ψ is a subset of a given KB and α is an atomic proposition from the KB; ψ is a minimal set of formulae such that ψ implies α, and ψ does not imply a contradiction. ψ is said to support the claim α. Where p and q are atoms, and where the KB is comprised of p and p→q, then is an argument. We could have a KB from which we can form an argument which supports ¬q,. In addition and with respect to this argument, suppose we can form an undercutter and a rebuttal. KBs (even relatively small ones) generate lots of arguments and attack relationships which can be structured in a tree.

Abstract Argumentation (Dung 1995) CS4025, Wyner, Computing Science, University of Aberdeen 10 Preferred extension: {a, c, d, h, i, k}

Zeroing In CS4025, Wyner, Computing Science, University of Aberdeen Source text Knowledge base & argumentation schemes Generated arguments (abstract or instantiated). 11

Current Tools to Extract and Structure Arguments from Text CS4025, Wyner, Computing Science, University of Aberdeen 12 Rationale, Araucaria, Carneades, IMPACT Project, Legal Apprentice,.... All manual. No NLP.

Argument Fragment for a Camera CS4025, Wyner, Computing Science, University of Aberdeen 13

Pro and Con CS4025, Wyner, Computing Science, University of Aberdeen 14

Comments on Comments CS4025, Wyner, Computing Science, University of Aberdeen 15

LIBER Response to Copyright Consultation CS4025, Wyner, Computing Science, University of Aberdeen 16 - Question 9. Should the law be clarified with respect to whether the scanning of works held in libraries for the purpose of making their content searchable on the Internet goes beyond the scope of current exceptions to copyright? - Yes. - Not all the material digitised by publishers is scanned with OCR (Optical Character Recognition) with the purpose of making the resulting content searchable. If the rights holders will not do this, libraries should be able to offer this service. It would have a transformative effect on research, learning and teaching by opening up a mass of content to users which can be searched using search engines. The interests of copyright holders will not be harmed, because the resulting output will act as marketing material for their materials.

A Problem with the Analysis of the Data CS4025, Wyner, Computing Science, University of Aberdeen 17 The arguments in these texts are: – fragmented and distributed across texts – conflicting/inconsistent statements How to find, link, and summarise the arguments? Contrast to arguments in a biomedical text or a Moral Maze discussion.

Various Argumentation Schemes 18 Patterns of defeasible reasoning: – Argument from position to know: reasoning about knowledge and individual's relation to it. – Argument from practical reasoning: reasoning about what to do. – Argument from trust from direct experience: reasoning about trusting someone. CS4025, Wyner, Computing Science, University of Aberdeen

Argumentation Schemes (Walton 1996) CS4025, Wyner, Computing Science, University of Aberdeen Patterns of presumptive (defeasible) reasoning Normalise the language. Practical Reasoning with values: – Do action (transition) because: Current circumstances - a list of literals. Consequences – a list of literals. Values (promoted, demoted, neutral wrt actions) – a list of terms. Credible Source: – Z is accepted because: X is an expert in domain Y. X stated literal Z Z is about domain Y. 19

Current Practice and Proposal CS4025, Wyner, Computing Science, University of Aberdeen 20 For consumer websites – none. For policy-making – backroom analysis of Green Paper responses. Method? Transparent? Structure? Proposal: Provide a tool for expert analysts (in context and workflow) to extract relevant material, homogenise statements, and construct/reconstruct arguments, e.g. output structured policy statements derived from source. Better understanding of what to look for and how.

Objectives 21 Identify and extract scheme information in and from across text. Explore how schemes appear in text. Address discontinuity/fragmentary presentation within sentences and across texts. Interconnect relevant, related portions of schemes from extracted information. Construct an argumentation graph for reasoning with inconsistency. CS4025, Wyner, Computing Science, University of Aberdeen

Means: A Language Of Schemes 22 Finger-print the scheme. – characteristic terminology of the scheme. – generalise the terminology to cover variation. – distinguish "domain" from "generic" terminology. – semantic annotations as "conceptual covers" over terminology. A scheme of linked schemes. Complex, flexible queries over the annotations. Use general tools to access many functionalities. CS4025, Wyner, Computing Science, University of Aberdeen

Consumer Argumentation Scheme Typed variables in schemes as targets for extraction. Premises: Camera X has property P. Property P promotes value V for agent A. Conclusion: Agent A should Action Camera X. 23 CS4025, Wyner, Computing Science, University of Aberdeen

Analyst’s Goal: Instantiate Premises: The Canon SX220 has good video quality. Good video quality promotes image quality for casual photographers. Conclusion: Casual photographers should buy the Canon SX CS4025, Wyner, Computing Science, University of Aberdeen

Annotating Text Annotate text: – Scheme's typed variables = semantic annotations. – Atomic (context free) or complex annotations (contextualised). – Colour annotations = XML. – Search for and extract text by annotation. 25 CS4025, Wyner, Computing Science, University of Aberdeen

Find Argument Passages 26 CS4025, Wyner, Computing Science, University of Aberdeen

Rhetorical Terminology 27 CS4025, Wyner, Computing Science, University of Aberdeen

Find What Is Being Discussed 28 CS4025, Wyner, Computing Science, University of Aberdeen

Domain Terminology 29 CS4025, Wyner, Computing Science, University of Aberdeen

Find Attacks Between Arguments 30 CS4025, Wyner, Computing Science, University of Aberdeen

Sentiment Terminology 31 CS4025, Wyner, Computing Science, University of Aberdeen

32 CS4025, Wyner, Computing Science, University of Aberdeen

Query For Patterns 33 Create queries on the fly from the "language" of the annotations. Extract strings that match the query. Use the results to identify useful, complex queries. Discontinuities. CS4025, Wyner, Computing Science, University of Aberdeen

An Argument To Buy The Camera Premises: The pictures are perfectly exposed. The pictures are well-focused. No camera shake. Good video quality. Each of these properties promotes image quality. Conclusion: (You, the reader,) should buy the CanonSX CS4025, Wyner, Computing Science, University of Aberdeen

An Argument NOT To Buy The Camera Premises: The colour is poor when using the flash. The images are not crisp when using the flash. The flash causes a shadow. Each of these properties demotes image quality. Conclusion: (You, the reader,) should not buy the CanonSX CS4025, Wyner, Computing Science, University of Aberdeen

Contrary Propositions The colour is poor when using the flash. For good colour, use the colour setting, not the flash. The images are not crisp when using the flash. No need to use flash even in low light. The flash causes a shadow. There is a corrective video about the flash shadow. 36 CS4025, Wyner, Computing Science, University of Aberdeen

Input to Argument Evaluators CS4025, Wyner, Computing Science, University of Aberdeen 37 Preferred extension: {a, c, d, h, i, k}

Additions 38 Verb classes, e.g. propositional attitudes/opinion terms. Fine-grained domain terminology. Adjectival and noun classes for sentiment/opinion. Ontologies. Terminology for contexts, for user classes.... CS4025, Wyner, Computing Science, University of Aberdeen

Argument Exploration 39 Flexible, interactive, incremental, modular, semi- automatic information extraction. Identify and extract scheme information in and from across text. Explore how schemes appear in text. Discontinuity/fragmentation by semantic search within sentences and across texts. Interconnect relevant, related portions of schemes from extracted information. Construct an argumentation graph for reasoning with inconsistency. CS4025, Wyner, Computing Science, University of Aberdeen

Different From 40 CS4025, Wyner, Computing Science, University of Aberdeen Not statistical NLP/Machine learning, though could use some aspects. Want to explain results, (manually) learn from error, and modify to suit (easy with lists and rules). Can be used to develop a gold standard corpora. No fixed syntactic or semantic structures.

Relates to CS4025, Wyner, Computing Science, University of Aberdeen Scheme/template filling for opinion and event mining. Issues of discontinuity and coherence. Structured argumentation. Abstract argument evaluation. Entailment graphs.

Thanks For Your Attention! Questions? Contacts: –Adam 42 CS4025, Wyner, Computing Science, University of Aberdeen