From natural language to Bayesian Networks (and back) Floris Bex Utrecht University

Introduction Analysts and decision-makers work with natural language text (or semi-structured arguments, scenarios) Particularly in domains where numerical judgments are less common Witness testimonies, expert reports, complaints, social media Outlines, reports, timelines, mind-maps Court arguments, rulings, reports

Introduction Formal and structured methods can be powerful tools for decision support Critical thinking & manual analysis Dialectical analysis Sensitivity analyses (Semi-)automation of decision making Formal ontologies Argumentation and non-monotonic logics Bayesian networks

Introduction Decision support systems for reasoning with evidence Principled ways of going from text to formal models Linguistic aspects How do people express and interpret scenarios, probabilities, arguments? Formal aspects What are the relations between formalisms? Design aspects What are the goals of the system? Psychological aspects How do/should people reason?

From text to formalisms Formal (math) models Semi-structured or unstructured text Structured arguments and scenarios

From text to formalisms argumentation Wigmore graphs Ontologies Bayesian networks Timelines Reports, transcripts, etc Formal (math) models Semi-structured or unstructured text Structured arguments and scenarios

Criminal complaint handling for police Online criminal complaints (politie.nl) How can we get the most/best information from the complainant? How do we build a case based on multiple complaints? Current intake is with a static form Fields for names, addresses, bank accounts Free text field “What happened?” Examples of typically missing information: How did the counterparty convince the victim to do business? What other evidence of fraud is there?

Criminal complaint handling for police Get structured information (scenario, argument) from form & text Apply formal semantics for scenarios & arguments What are the gaps in the story? Which evidence is missing? Ask for missing information Give feedback about scenarios & arguments

Criminal complaint handling for police Structure and formalize information Argument mining scenario mining Provide formal semantics Formal (math) models Semi-structured or unstructured text Structured arguments and scenarios Text generation Natural dialogues Include or ask for new information Feedback of findings to user

Demo

Understanding stories I bought a Samsung S3 from Wesley. I paid him 45 euros

Understanding stories Syntactic parsing I bought a Samsung S3 from Wesley. I paid him 45 euros S v Od PP S v Od Oi

Understanding stories Syntactic parsing Named Entity Recognition Floris Samsung S3 Wesley 45 euros I bought a Samsung S3 from Wesley. I paid him 45 euros S v Od PP S v Od Oi

Understanding stories Domain ontology sent to sent by Complainant Product Counterparty paid to Payment paid by Floris Samsung S3 Wesley 45 euros I bought a Samsung S3 from Wesley. I paid him 45 euros S v Od PP S v Od Oi

Understanding stories Connect domain ontology to story (machine learning) sent to sent by Complainant Product Counterparty paid to Payment paid by Floris Samsung S3 Wesley 45 euros I bought a Samsung S3 from Wesley. I paid him 45 euros S v Od PP S v Od Oi

Scenario reasoning Scenario from text Floris bought a Samsung S3 from Wesley Floris paid Wesley 45 euros

Scenario reasoning ? Is the scenario complete (ontology)? The complainant bought product from the counterparty The complainant made a payment to the counterparty The counterparty sent the product to the complainant Floris bought a Samsung S3 from Wesley Floris paid Wesley 45 euros ?

Argumentative reasoning Is there evidence for the scenario? Floris bought a Samsung S3 from Wesley Floris paid Wesley 45 euros ? ?

Argumentative reasoning Is there evidence for the scenario? Floris bought a Samsung S3 from Wesley Floris paid Wesley 45 euros Emails, Whatsapp Bank statements

Criminal complaint handling for police Ontological reasoning, reasoning with arguments & scenario’s Complaint form(s) Structured scenarios Formal models Semi-structured or unstructured text Structured arguments and scenarios Questions to complainant Structured argumentation dialogue

From arguments to Bayesian networks Translate structured arguments based on evidence to Bayesian Networks A formal account of constraints on a BN imposed by structured arguments Semi-structured or unstructured text Structured arguments and scenarios Bayesian networks

Structured argumentation: ASPIC+ Arguments are Directed Acyclic graphs Nodes are statements in a logical language with neg. Links are applications of inference rules (strict or defeasible) Arguments constructed from knowledge base Ke (evidence, certain premises), Kp (assumptions, uncertain premises) Attack On uncertain premises, on defeasible inferences, on conclusions

Structured arguments The burglary (Bur) was committed by the suspect, because there is a footprint match (Ftpr) and a motive (Mot) backed by a report (For) and a testimony (Tes1), and the suspect has no alibi, so Opp. Bur Ftpr Mot Opp For Tes1

Structured arguments However, there is evidence of a mixup in the lab (Mix), which means the footprint match is not really backed by evidence. Furthermore, the suspect later gave a testimony (Tes2) with an alibi, so −Opp. Bur Ftpr Mot Opp −Opp Mix For Tes1 Tes2

Bayesian Networks Represent joint probability distribution as DAG + CPT Directed Acyclic Graph Nodes are variables Bur = [Bur, −Bur] Arcs represent probabilistic dependencies between nodes (Mot, Bur) Tes1 Bur For Rel Mot Ftpr Mix

Bayesian Networks Observations E Tes1 Bur For Rel Mot Ftpr Mix

Bayesian Networks Chains can be blocked, or inactive, given E if s contains node with two incoming arcs which is not in E and has no descendants in E; or s contains node in E that has at most one incoming arc on the chain. Tes1 Bur For Rel Mot Ftpr Mix

Bayesian Networks Active chains are not blocked Tes1 Bur For Rel Mot Ftpr Mix

Bayesian Networks Sets of variables X and Y are independent given E iff there is no active chain from X to Y Tes1 Bur For Rel Mot Ftpr Mix

From arguments to constraints on BN Nodes Every proposition v or −v in the argument is a node representing variable v in the BN Every proposition v in Ke is the observed value of v Ftpr Ftpr For For

From arguments to constraints on BN Inference chains For every rule v1,…, vn => vc / v1,…, vn −> vc used in an argument there is an active chain between nodes v1,…, vn and vc Ftpr Ftpr Ftpr Ftpr v1…vn For For For For

From arguments to constraints on BN Attack chains For contradictory (i.e. Opp and −Opp) propositions, this is captured by inference chains Opp Opp −Opp v1…vn Tes2 Tes2

From arguments to constraints on BN Attack chains If vu undercuts the application of rule vp => vc, then there are active chains from vp, vc to vu Mix Ftpr For Ftpr Ftpr Mix For Mix For

From arguments to constraints on BN Attack chains If vu undercuts the application of rule vp => vc, then there are active chains from vp, vc to vu Mix Ftpr Ftpr Mix vi…vj vk…vn For For

Using arguments to check BNs Differences in modelling between judge & BN expert Bur Ftpr Mot Opp −Opp Mix For Tes1 Tes2 Tes1 Bur For Rel Mot Ftpr Mix

Using arguments to check BNs Missing variables Bur Ftpr Mot Opp −Opp Mix For Tes1 Tes2 Tes1 Bur For ?? Rel Mot Ftpr Mix

Using arguments to check BNs Active inference chains Bur Ftpr Mot Opp −Opp Mix For Tes1 Tes2 Tes1 Bur For Rel Mot Ftpr Mix

Using arguments to check BNs Active inference chains Bur Ftpr Mot Opp −Opp Mix For Tes1 Tes2 Tes1 Bur For Rel Mot Ftpr Mix

Using arguments to check BNs Active attack chains If Mix undercuts the application of rule For => Ftpr, then there are active chains from For, Ftpr to Mix Ftpr Mot Opp −Opp Mix For Tes1 Tes2 Tes1 Bur For Rel Mot Ftpr Mix

Using arguments to check BNs Active attack chains If Mix undercuts the application of rule For => Ftpr, then there are active chains from For, Ftpr to Mix Ftpr Mot Opp −Opp Mix For Tes1 Tes2 Tes1 Bur For Rel Mot Ftpr Mix

Using arguments to build BNs Bur Ftpr Mot Opp −Opp Mix For Tes1 Tes2 For Tes1 Tes2 For Mix Ftpr Mot Opp Mix Bur

Using arguments to build BNs Bur Ftpr Mot Opp −Opp Mix For Tes1 Tes2 For Tes1 Tes2 For Mix Ftpr Mot Opp Mix Bur

From arguments to constraints on BN Probability constraints Defeasible rule v1,…, vn => vc v1,…, vn is evidence for vc Different interpretations of “is evidence for” Pr(vc | v1,…, vn) > 0 (Verheij 2014) Pr(vc | v1,…, vn) > 0.5 (Pollock 1995) Pr(vc | v1,…, vn) > Pr(vc) (Hahn & Hornikx 2015) …

Constraints and multiple BNs Constraints on structure and probabilities lead to multiple different BNs Semi-structured or unstructured text Structured arguments and scenarios Bayesian networks

Constraints and multiple BNs Constraints on structure and probabilities lead to multiple different BNs Compare BNs – where are the real differences? Ask decision maker about these differences. Questions to decision maker Structured arguments and scenarios Bayesian networks

Conclusions Principled and automated ways to go from text to formal models Implement algorithms & translations into working systems! Vlek et al. Fenton, Neil, Lagnado Timmer et al. Formal (math) models Semi-structured or unstructured text Structured arguments and scenarios Vlek et al. Yet et al. Timmer et al.