Digging by Debating (DbyD): linking massive datasets to specific arguments UK: Prof Andrew Ravenscroft (University of East London) Dr David Bourget (University of London) Prof Chris Reed (University of Dundee) US: Prof Colin Allen & Prof Katy Borner (Indiana University) …all together

DbyD Overview 1. The Problem 2. InterDebates: Making a shovel 3. Work and progress 4. Challenges, issues and solutions

Map argument dialogue and argumentative search semantics against argument structures implicit within and across texts (Philosophy domain) …harmonise human and machine semantics …extract and interrogate argument relations …deeper dialectic understanding of subjects …find new relations within and across texts (e.g. penetration of science and philosophy)

InterDebates: Making a shovel Make a shovel through integrating, adapting and extending existing technologies 1. Visualisation/Science Mapping: SCiVis (Indiana) 2. Data: Hathi Trust, PhilPapers, SEP, InPho (Indiana & London) 3. Interfaces: PhilPapers, Dialogue Games/InterLoc (UK) 4. Argument structure and analysis: Dialogue Games, Araucaria, AIF (London, Dundee) …how marshall these to interrogate massive document collections in argument-driven ways? …quite ambitious!

,

Citation networks: InPho, SEP

Digital Dialogue Games and InterLoc

OVA: Online Argument Analysis

Work and progress Small-scale studies using argument analysis and representation techniques applied to exemplar texts (in Animal Intelligence) AI simulation of argument mining process Argument mining literature review  …Key Insights and Implications for ongoing work

Marking up and mapping example texts Small-scale studies using OVA to map arguments in example texts Animal Psychology from Hathi Trust Collection Philosophers: Perform their own markup of important arguments in key texts Externalise the ‘algorithm’ they used to perform the markup Can OVA represent the arguments in a way that Philosophers want? Can we automate their algorithms or approximate them using combination of statistical, syntactic and semantic means?

AI Simulation of Argument Mining A hand simulation of a feasible machine analysis of an example text (a 10 page extract from Comparative Psychology) Uses syntactic parsing and rewrite rules into pre-formed argument patterns - results were mixed and fragmentary In example text obvious premises to the argument often missing - intelligent reader assumed by author Argument results are fragmentary - needs linking up - but require semantics of the argument to do this What arguments are important? What can be ignored? Cannot proceed further without sophisticated semantic layer to interpret results Would need to create 'ontology of argument' plus domain knowledge (to decide importance) - a big task

Argument Mining Literature Review Extracting meaningful argument information from natural text remains a major challenge facing computer science and AI Key works on argument mining are: Moens et al., 2007; Palau and Moens, 2009; Mochales and Moens, 2011; Feng and Hirst, 2011; Mainly classification using maximum entropy model and support vector machine % accuracy in legal cases Also considered works on opinion mining (e.g. Pang et al., 2002), citation mining (e.g. Teufel et al., 2006), and argumentative zoning (e.g. Teufel et al., 2009) Extra information, for example on argument scheme features, can improve accuracy

Implications for the Workplan May be possible to auto-identify argumentative blocks of text, using statistical / training techniques e.g Argumentative Zoning Also possible to auto-select only those blocks dealing with pre-specified topics thought to be important No way to reliably auto-extract 'core' arguments from texts without a sophisticated semantic layer (a large task!) Therefore use human experts to extract 'core' information - provide a tool that helps them be very efficient at it A tool could help with -  locating key blocks of argument in a text  indicating author's 'position' with regard to cited texts  associating 'core' argument(s) with the text We will need volunteers to process lots of texts to provide training sets for machine learning

Challenges, issues and solutions Ambitious problem + complicated international collaboration = hard work! Managing the collaboration How perform a joint project with independent ‘halves ‘ that have different funding and project management models? How hold regular working meetings to maintain national and international co- operation and joint working with limited cross-over in typical working hours? Consortium agreement How procure a consortium agreement that covers a joint project but is only ‘legally binding’ on one half? How articulate data access issues that are evolving as the project progresses? How to specify the joint working in a way that captures joint responsibility and also ‘legal’ independence? Data Access and Use How dig into data that is freely available to one partner (the US) but restricted to the other (UK)? e.g. Hathi Trust Research Collection (in Google Books)

Challenges, issues and solutions Managing the collaboration How perform a joint project with independent ‘halves ‘ that have different funding and project management models? How hold regular working meetings to maintain national and international co- operation and joint working with limited cross-over in typical working hours? Extended project planning phase (WP1 ) Continued problematisation and scoping phase, the outcome of which is a revised joint workplan (WP2) Had two ‘opportunistic’ f2f meetings in Europe, during the first we agreed a skype meeting regime for the life of the project: - UK meetings every second and last Tuesday at UK/US meetings every first and third Tuesday 1400/0900

Challenges, issues and solutions Consortium agreement How procure a consortium agreement that covers a joint project but is only ‘legally binding’ on one half? How articulate data access issues that are evolving as the project progresses? How to specify the joint working in a way that captures joint responsibility and also ‘legal’ independence? UK only consortium agreement, following JISC prototype, plus link to related documents covering: - UK/US joint working - Data access protocols …UK finance etc., covered by consortium agreement with joint working and data access covered by separate but related documents

Challenges, issues and solutions Data Access and Use How dig into data that is freely available to one partner (the US) but restricted to the other (UK)? e.g. Hathi Trust Research Collection (in Google Books)? 2 phase solution: 1. Immediate work-around – limited access to Data at Indiana via specific computers/ip addresses at UK site 2. UK partners negotiate equivalent data access directly with HTRC with help from US partners

Contacts Summary of progress: " Though this be madness, yet there is method in it“ Hamlet