From logic and probability to argument and evidence: a cognitive perspective Mathematical logic and probability theory are usually taken to provide the standards for rational reasoning and decision making and, hence, the basis for understanding argument and evidence in law, science, medicine etc. In 1958 the philosopher Stephen Toulmin questioned this assumption. His position was that natural argument is a dialectical not a deductive process, a debate between proponents who exchange reasons for their competing positions, not a function which is simply true or false. He also takes a swipe at “scientific” probability on the related grounds that the rationale for a probability judgement is lost in the formalism. Since his complaints were largely intuitive and his solution, the famous “Toulmin schema”, only informal his criticisms were seen as interesting by some but of limited practical significance by the mainstream. In recent years, however, Toulmin’s schema has been developed and applied in various areas of computer science and cognitive science, and given formal interpretations and semantics. This is providing new ways of analysing arguments and may offer a useful perspective on how we should understand evidence. It may even show us how to design automatic procedures for constructing and appraising lines of argument and evidence which are mathematically sound but non- mathematicians find natural. S Toulmin The uses of argument. Cambridge University Press, 1958

From logic and probability to argument and evidence: a cognitive perspective John Fox London Research Institute Cancer Research UK

Argument and evidence: ancient forms of thought Gods Oracles Meaning 1 Meaning 2 Observations & reports Dreams May, Can Will … Diagnoses prophecies Humours Thanks to Jason Davies

The heart of it “Some philosophers … have an ineradicable suspicion of our everyday forms of thought … In their view, the development of science, and the displacement of all our ordinary, pre-scientific ideas by the more refined notions of the theoretical sciences, hold out the only hope of salvation for incoherence, fallacy and intellectual confusion.” S E Toulmin, 1958 p92 “… there is essentially only one way to reach a decision sensibly. First, the uncertainties present in the situation must be quantified in terms of values called probabilities. Second, the consequences of the courses of actions must be similarly described in terms of utilities. Third, that decision must be taken which is expected on the basis of the calculated probabilities to give the greatest utility. The force of ‘must’ used in three places there is simply that any deviation from the precepts is liable to lead the decision maker in procedures which are demonstrably absurd” D Lindley, 1983 “As living and moving beings, we are forced to act … [even when] our existing knowledge does not provide a sufficient basis for a calculated mathematical expectation.” J M Keynes, 1936

Overview of talk What is an argument? Toulmin’s alternative to “scientific” views of logic and probability Qualitative and quantitative aspects of argument and evidence Dung’s calculus of opposition; LA a logic of argument Talking about argument and evidence – the place of ordinary language Argument as a “meta-language”

What is an argument? Classical view P, (P  Q) Q Since a man born in Bermuda will generally Be a British citizen Harry was born in Bermuda So, presumably, Harry is a british citizen On account of the following Statutes and other legal Provisions … Unless Both his parents were aliens He has become a naturalised American Non-traditional view (Toulmin) Data Qualifier, Claim since Warrant unless Rebuttal On account of Backing

Argument and evidence: Toulmin form Background knowledge Field 2 Field n Claim 1 Claim 2 Qualifiers (possibly, probably, presumably …) Data Field 1 Qualified claims

Toulmin in practice Steele et al, Proc. European Conf. AI in Medicine

Argument and evidence General knowledge, expertise Area 2 Hypothesis Known situation Area 1 Hypothesis 2 Area n

Argument and evidence: modern form (medicine) Knowledge base Epidemiology Genetics Hypoth- esis Argument aggregation Strength of evidence Argument pattern Modalities (possibly, probably, presumably …) Records, symptoms, test results Immunology Qualified hypotheses Anatomy Metabolism Pathology Hypoth- esis 2

Medical arguments “Fields” of reasoning Immunology Physiology Anatomy Biochemistry Genetics Morphology Epidemiology Mental health Social dysfunction Modes of reasoning Causal Statistical Functional Structural Spatial Temporal Modal Deontic } anatomy

Argument and evidence: Bayesian form Knowledge of variables CPs 2 CPs n Hypoth- esis 1 Hypoth- esis Probability revision Belief revision Data and priors CPs 1 Posterior probabilities

Protein structure prediction

Argument and evidence: protein structure prediction Knowledge base Predict Argument pattern Modalities (possibly, probably, presumably …) Amino acid sequence Predict 2 Physical folding Structural arguments e.g. packing constraints Physical folding Structural arguments e.g. packing constraints Causal arguments e.g. charged residues on outside of molecule Physical folding Structural arguments e.g. packing constraints Causal arguments e.g. charged residues on outside of molecule Energetic arguments e.g. folding minimises free energy Tentative models Other constraints Functional arguments e.g. reducing osmotic pressure Other constraints Functional arguments e.g. reducing osmotic pressure Evolutionary arguments e.g. conservation of structures over generations Other constraints Functional arguments e.g. reducing osmotic pressure Evolutionary arguments e.g. conservation of structures over generations Similarity arguments e.g. homologies between related proteins Argument aggregation Probabilities

Formalising argument: Dung’s “calculus of opposition” (1995) The basic intuition is that a statement is “believable” if we can identify an acceptable argument for it An acceptable argument is –one that is not attacked, or –if it is attacked the attacking argument can be defeated Attack Defeat

The calculus of opposition Dung (1995) et seq Support can include ordinary logical proofs with negation as failure, and Defeat is a contradiction of this proof (rebuttal of the conclusion or undercutting of a premise) Dung provides formal criteria for –Admissibility of arguments –Stable extensions, Preferred extensions –Fixpoint semantics, Grounded semantics All argument games, debates, disputes, etc. are to be analysed solely in terms of these concepts

Araucaria argument modelling tool: Chris Reed and Glenn Rowe, U Dundee

Is the Dung interpretation sufficient? Genetic risk assessment - +c+c Coulson et al Methods of Information in Medicine Emery et al British Medical Journal 2000, 2001,

LA: A logic of argument Syntax –Propositional symbols: , , … –Connectives: , , , ,  –Dictionary of qualifiers: +, -, ++, -- Variables (Q, Q’, Q’’) –Auxiliary symbols: (, ), : Data  Theory LA (Claim : Qualifier)

Introduction rules (  I)  :Q  :Q`  : min(Q,Q`)  :++ (  I)  :Q  :Q (  I)  :Q  :Q  :Q  :++ (  I)  :Q  :Q Elimination rules (  E)  :Q  :Q  :Q  :Q (  E)  :Q  :Q`  : min(Q,Q’)  :++  :++ (  E)  :Q  :Q`  :Q``  :min(Q,Q `,Q``) (  E)  :Q  :Q`  :min(Q,Q`) Weakening  :++  : Fox et al Proc. Eur. Conf. AI 1992; Fox et al Proc. Uncertainty and AI 1993; Krause et al Comp. Intelligence, 1994 LA: Logic of argument

Formal semantics Proof theoretic (category theory) –Ambler “ A categorial approach to the semantics of argumentation ” Math. Structures in Comp. Sci Model-theoretic (possible worlds) –Das "How much does an agent believe: An extension of modal epistemic logic", Hunter and Parsons (Editors): Applications of Uncertainty Formalisms, Lecture Notes in AI 1455, Springer, Berlin 1998, –Also Ch12 in Safe and Sound, Fox and Das, MIT Press, 2000)

Modelling uncertainty in LA: qualifiers and dictionaries Symbolic dictionaries {+,-}delta dictionary (“pros and cons”) {++,--, +, -}bounded delta Quantitative dictionaries [0,1]probability, possibility [-1,+1]certainty factors {1,2,3,…n}integer weights Linguistic dictionaries P-modals (Possible, probable, plausible …) - +

Argument and evidence: LA form Deductive database Theory 2 Theory n World 1 World Argument aggregation Measures of strength e.g. from [0,1] Argument pattern Modalities (e.g. possibly, probably, presumably …) Situation beliefs, goals, plans Theory 1 Defeasible claims Fox et al Proc. Eur. Conf. AI 1992; Fox et al Proc. Uncertainty and AI 1993; Krause et al Comp. Intelligence, 1994

Talking about evidence and risk the place of natural language J Fox “Will it happen? can it happen? a new approach to formal risk analysis” Risk and public policy (1999).

Informal semantics: talking about uncertainty, risk and evidence in everyday language confirmed Carcinogen epidemiological data and/or established causal relationship possible Carcinogen potential hazard recognised probable Carcinogen better evidence than merely recognition of possible carcinogenic activity improbable Carcinogen possible carcinogenic activity, but strong evidence against action equivocal hazard recognised and both evidence for and evidence against not Carcinogen test case data or direct chemical analysis disconfirms carcinogenic activity open no information regarding potential hazard available Guidelines for the evaluation of chemicals for carcinogenicity International Agency for Research on Cancer

Informal semantics: talking about uncertainty, risk and evidence in everyday language Guidelines for the evaluation of chemicals for carcinogenicity International Agency for Research on Cancer confirmed Carcinogen epidemiological data and/or established causal relationship possible Carcinogen potential hazard recognised probable Carcinogen better evidence than merely recognition of possible carcinogenic activity improbable Carcinogen possible carcinogenic activity, but strong evidence against action equivocal hazard recognised and both evidence for and evidence against not Carcinogen test case data or direct chemical analysis disconfirms carcinogenic activity open no information regarding potential hazard available

From arguments to modalities Given a set of arguments for and against a claim we can map the set into many different qualifiers or modalities { (Claim : Warrant : Qualifier) }  (Claim : Modality) Data  Theory LA (Claim : Warrant : Qualifier)

P is  if it is any well-formed formula in the language of the logic P is  if an argument, possibly using inconsistent data, can be constructed P is  if a consistent argument can be constructed (we may also be able to construct a consistent argument against) P is  if a consistent argument can be constructed for it, and no consistent argument can be constructed against it. P is  if it satisfies the conditions of being probable and, in addition, no consistent arguments can be constructed against any of the premises used in its supporting argument P is  if it is a tautology of the logic (meaning that its validity is not contingent on any data in the knowledge base). Logical acceptability classes Elvang-Gorannson, Krause and Fox “Logic and linguistic uncertainty terms” Proc. UAI (1993)

Logical acceptability classes : “Linguistic annotations” P is open if it is any well-formed formula in the language of the logic P is supported if an argument, possibly using inconsistent data, can be constructed P is plausible if a consistent argument can be constructed (we may also be able to construct a consistent argument against) P is probable if a consistent argument can be constructed for it, and no consistent argument can be constructed against it. P is confirmed if it satisfies the conditions of being probable and, in addition, no consistent arguments can be constructed against any of the premises used in its supporting argument P is certain if it is a tautology of the logic (meaning that its validity is not contingent on any data in the knowledge base). Elvang-Gorannson, Krause and Fox “Logic and linguistic uncertainty terms” Proc. UAI (1993)

Qualifiers, modalities and commitments “Human practice introduces new modalities on an ad hoc basis. … Introducing new modalities should involve no more fuss than introducing a new predicate. In particular, human-level AI requires that programs be able to introduce modalities when this is appropriate” John McCarthy “Modality, Si! Modal logic, No!” Possible, probable, plausible, conceivable, certain, equivocal, dubious, likely …may be, could be, might be … believed, suspected, known … lexical negation, affixal negation…

Combining quantitative and qualitative methods (REACT: Risk, Events, Actions and their Consequences over Time) David Glasspool, Tito Castillo, Vicky Monaghan, Ayelet Oettinger

Argument and evidence: legal form Case law Expert opinion Guilty Argument aggregation Quantitative talk Argument pattern Language based talk Claimed facts Witness testimony Judgement (subject to appeal) Not guilty

Charting the evidence (The Umilian case, Wigmore 1931)

Argument as meta-language CLAIM  Uncertainty must be represented with a well-behaved measure ([0,1] probability)  We often cannot measure uncertainty, but must still act (authority: Keynes)  Medical and countless everyday examples  Measures that do not satisfy the [0,1] probability axioms will be “incoherent”  Dutch book and similar arguments (e.g. Lindley)  This does not necessitate that all formalisms are incoherent in all circs.  the Delta argument  There is a vast body of well-understood, classical techniques available  Classical methods don’t address all the problems  Example, the “ill-formed problem” problem  Example, the human communication problem  Example, the knowledge representation problem  Example, the meta-representation problem  Non-probabilistic methods lack precision and cannot work well  There is a significant body of empirical evidence that they do  Theory is a stronger guide than mere evidence  That depends upon your point of view! CLAIM “[0,1] probability is the only correct representation for reasoning under uncertainty”

 There is a theoretical case against  There are sound alternatives to probability  Example, possibility theory (Quantitative)  Example, modal logic (Qualitative)  There are formalisms that are more expressive  Example, first-order logic (description logics?)  There is a practical case against  We need more general methods in medicine than probability  We commonly lack a basis for estimating probabilities  We can manage this, e.g. by upper-lower bounds  This compounds the problem technically  Example, estimation difficulties  Example, computational costs  We may not know whether other assumptions are satisfied  Example, conditional independence  Example, probability estimates are valid  The logical structure (topology) is what matters not the weightings  A substantial body of empirical information supports this Argument as meta-language CLAIM “[0,1] probability is the only correct representation for reasoning under uncertainty” (the case against)

 Psychological arguments  In the end uncertainty is an entirely polysemous and subjective notion  Example, linguistic terms (belief-doubt, possible-probable)  Uncertainty can be treated as a uni-dimensional quantity measure  Objective (frequentistic)  Subjective (Bayesian)  People find mathematical probability difficult to understand  We need to develop rational theory, not make people happy  That depends on your point of view  Representational arguments  We need to use the representation with the highest practical benefit and power  No, we must use a standard, normal form i.e. probability  The normal form should be the most general form (logic of argument!)  There are arguments for and against probability as an objective mathematical idea  Scientific truth excludes inconsistency, you can’t “cherry pick” theories  Even scientific theories must live with ambiguity  Example: Waves and particles  Theories are always incomplete and inconsistent Lakatos, Kuhn, Popper, Feyerabend … Argument as meta-language CLAIM “[0,1] probability is the only correct representation for reasoning under uncertainty” (the case against, continued)

Argument and evidence: generic form “Background” knowledge Area 2 Area n Claim Argument aggregation Quantitative talk Argument pattern Language based talk Known situation Area 1 Conceivable situation Claim 2

Conclusions We currently view much “common sense” reasoning as degenerate forms of reasoning (e.g. talk about evidence or risk) Toulmin criticised the scientific view, claiming that natural argument implements a different but valid reasoning model Recent work in AI and non-classical logic provides a clearer interpretation and sound formalisation for such ideas LA provides a way of reconciling natural patterns of reasoning with scientific and mathematical forms, possibly grounded in Dung semantics (in part) Studies of risk assessment, decision-making and planning in medicine suggests that the approach is practically effective Reactions suggest that the approach is also intuitive for those who are formally trained and those who are not. The challenge is to develop our understanding of the relationship between quantitative and qualitative perspectives on uncertainty, risk and evidence rather than seeing them as competitors.

Some references Fox J et al “Argumentation as a general framework for uncertain reasoning” Proc. Uncertainty in AI, , 1993 Dung P M “On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games” AI Journal 77, , 1995 Krause P et al “A logic of argumentation for reasoning under under uncertainty” Computational Intelligence, 11, , 1995 Fox J “Logic, probability and the cognitive foundations of rational belief” J Applied Logic, 1, , 2003

Aleatory matrix Aleatory: depending on an uncertain event or contingency as to both profit and loss SubjectiveObjective Situation Believe, think, assume Expect, anticipate Equivocal Probable, plausible, possible, necessary, will be, may be Ambiguous Action Prefer Intend, plan Permitted, obligatory Aleatory Expected value, supported, opposed Cost-benefit, advantages- disadvantages, benefits- harms

Aleatory semantics Aleatory: depending on an uncertain event or contingency as to both profit and loss SubjectiveObjective Situation Epistemic arguments Possible worlds, frequentistic probability Action Deontic argumentsPossible worlds, Aleatory GoalsValue functions

Semantics of medical knowledge

Spoken/typed input Applying the knowledge

Spoken/typed input Speech recogniser (commercial) Finite state machine for low level response selection High level dialogue manager Applying the knowledge

Argument and decision General knowledge, expertise Area 2 Area n Option 1 Option Argument aggregation Strength of evidence e.g. [0,1] Argument pattern Status of hypothesis (possibly, probably, presumably true …) Known situation Area 1 Conceivable situation