A. Darwiche Bayesian Networks. A. Darwiche Reasoning Systems Diagnostics: Which component failed? Information retrieval: What document to retrieve? On-line.

Slides:

Advertisements

Similar presentations

Gated Graphs and Causal Inference

Advertisements

Bayesian networks Chapter 14 Section 1 – 2. Outline Syntax Semantics Exact computation.

BAYESIAN NETWORKS Ivan Bratko Faculty of Computer and Information Sc. University of Ljubljana.

Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing Conditional Distributions Summary.

Bayesian Networks. Quiz: Probabilistic Reasoning 1.What is P(F), the probability that some creature can fly? 2.Creature b is a bumble bee. What’s P(F|B),

Bayesian Network and Influence Diagram A Guide to Construction And Analysis.

Bayesian Networks CSE 473. © Daniel S. Weld 2 Last Time Basic notions Atomic events Probabilities Joint distribution Inference by enumeration Independence.

Artificial Intelligence Universitatea Politehnica Bucuresti Adina Magda Florea

BAYESIAN NETWORKS. Bayesian Network Motivation  We want a representation and reasoning system that is based on conditional independence  Compact yet.

1 Bayes nets Computing conditional probability Polytrees Probability Inferences Bayes nets Computing conditional probability Polytrees Probability Inferences.

Probability: Review The state of the world is described using random variables Probabilities are defined over events –Sets of world states characterized.

PROBABILITY. Uncertainty  Let action A t = leave for airport t minutes before flight from Logan Airport  Will A t get me there on time ? Problems :

BAYESIAN NETWORKS CHAPTER#4 Book: Modeling and Reasoning with Bayesian Networks Author : Adnan Darwiche Publisher: CambridgeUniversity Press 2009.

Identifying Conditional Independencies in Bayes Nets Lecture 4.

For Monday Read chapter 18, sections 1-2 Homework: –Chapter 14, exercise 8 a-d.

For Monday Finish chapter 14 Homework: –Chapter 13, exercises 8, 15.

From Rumelhart (1977) "Toward Interactive Model of Reading"

A. Darwiche Inference in Bayesian Networks. A. Darwiche Query Types Pr: –Evidence: Pr(e) –Posterior marginals: Pr(x|e) for every X MPE: Most probable.

Bayesian Networks VISA Hyoungjune Yi. BN – Intro. Introduced by Pearl (1986 ) Resembles human reasoning Causal relationship Decision support system/ Expert.

Causal and Bayesian Network (Chapter 2) Book: Bayesian Networks and Decision Graphs Author: Finn V. Jensen, Thomas D. Nielsen CSE 655 Probabilistic Reasoning.

Review: Bayesian learning and inference

A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graph.

A. Darwiche Bayesian Networks. A. Darwiche Bayesian Network Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine.

Bayesian networks Chapter 14 Section 1 – 2.

Bayesian Belief Networks

. Bayesian Networks Lecture 9 Edited from Nir Friedman’s slides by Dan Geiger from Nir Friedman’s slides.

Bayesian Networks. Graphical Models Bayesian networks Conditional random fields etc.

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering CSCE 580 Artificial Intelligence Ch.6 [P]: Reasoning Under Uncertainty Sections.

1 Bayesian Reasoning Chapter 13 CMSC 471 Adapted from slides by Tim Finin and Marie desJardins.

Bayesian Networks Alan Ritter.

A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graphs.

Bayesian networks More commonly called graphical models A way to depict conditional independence relationships between random variables A compact specification.

Quiz 4: Mean: 7.0/8.0 (= 88%) Median: 7.5/8.0 (= 94%)

Made by: Maor Levy, Temple University  Probability expresses uncertainty.  Pervasive in all of Artificial Intelligence  Machine learning 

A Brief Introduction to Graphical Models

Context-specific independence Graphical Models – Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4,

Soft Computing Lecture 17 Introduction to probabilistic reasoning. Bayesian nets. Markov models.

Bayesian networks Chapter 14 Section 1 – 2. Bayesian networks A simple, graphical notation for conditional independence assertions and hence for compact.

An Introduction to Artificial Intelligence Chapter 13 & : Uncertainty & Bayesian Networks Ramin Halavati

Bayesian networks. Motivation We saw that the full joint probability can be used to answer any question about the domain, but can become intractable as.

1 Chapter 14 Probabilistic Reasoning. 2 Outline Syntax of Bayesian networks Semantics of Bayesian networks Efficient representation of conditional distributions.

For Wednesday Read Chapter 11, sections 1-2 Program 2 due.

1 Monte Carlo Artificial Intelligence: Bayesian Networks.

Introduction to Bayesian Networks

An Introduction to Artificial Intelligence Chapter 13 & : Uncertainty & Bayesian Networks Ramin Halavati

Belief Propagation Revisited Adnan Darwiche. Graphical Models Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine.

Reasoning Under Uncertainty: Conditioning, Bayes Rule & the Chain Rule Jim Little Uncertainty 2 Nov 3, 2014 Textbook §6.1.3.

Bayesian Nets and Applications. Naïve Bayes 2  What happens if we have more than one piece of evidence?  If we can assume conditional independence 

INTERVENTIONS AND INFERENCE / REASONING. Causal models  Recall from yesterday:  Represent relevance using graphs  Causal relevance ⇒ DAGs  Quantitative.

Computer Science CPSC 322 Lecture 27 Conditioning Ch Slide 1.

Review: Bayesian inference  A general scenario:  Query variables: X  Evidence (observed) variables and their values: E = e  Unobserved variables: Y.

1 CMSC 671 Fall 2001 Class #20 – Thursday, November 8.

Pattern Recognition and Machine Learning

Conditional Independence As with absolute independence, the equivalent forms of X and Y being conditionally independent given Z can also be used: P(X|Y,

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Classification COMP Seminar BCB 713 Module Spring 2011.

PROBABILISTIC REASONING Heng Ji 04/05, 04/08, 2016.

Chapter 12. Probability Reasoning Fall 2013 Comp3710 Artificial Intelligence Computing Science Thompson Rivers University.

CS 2750: Machine Learning Bayesian Networks Prof. Adriana Kovashka University of Pittsburgh March 14, 2016.

Compiling Bayesian Networks Using Variable Elimination

Another look at Bayesian inference

CS 2750: Machine Learning Directed Graphical Models

Reasoning Under Uncertainty: Conditioning, Bayes Rule & Chain Rule

Probabilistic Reasoning; Network-based reasoning

Structure and Semantics of BN

CS 188: Artificial Intelligence Fall 2007

Class #21 – Monday, November 10

Structure and Semantics of BN

Context-specific independence

Chapter 14 February 26, 2004.

Presentation transcript:

A. Darwiche Bayesian Networks

A. Darwiche Reasoning Systems Diagnostics: Which component failed? Information retrieval: What document to retrieve? On-line help: What is he trying to do? E-commerce: Is he interested in books? CAD: Does the design meet the specification? Decoders: What was the original message? Predictive maintenance: Will the device break?

A. Darwiche Model-Based Reasoning Formal Model (KB) Reasoning Engine Query Conclusions What modeling language? How to get the model? How to reason efficiently?

A. Darwiche Bayesian Network Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start

A. Darwiche Bayesian Network Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Pr(Lights=ON | Battery-Power=OK) =.99 ON OFF OK WEAK DEAD Lights Battery Power θ 1 + θ 2 = 1

A. Darwiche Model-Based Reasoning Efficiency: Time and space resources (algorithms) Formalization –What modeling language? (symbolic, quantitative) –What query semantics? (diagnosis, recommender, belief revision, causality) Synthesis of models –Knowledge engineering (sensitivity analysis) –Synthesize from design information (model checking, verfication) –Synthesize from online data (learning) Embeddability (compiled reasoning, anyspace reasoning)

A. Darwiche

Diagnosis Scenario Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Query Behavior

A. Darwiche Diagnosis Scenario Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start okonyesno.001 okoffyesno.090

A. Darwiche Probabilistic Reasoning Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Posterior Marginals

A. Darwiche Probabilistic Reasoning Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Maximum a Posteriori (MAP)

A. Darwiche Probabilistic Reasoning Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Maximum a Posteriori (MAP)

A. Darwiche Probabilistic Reasoning Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Maximum a Posteriori (MAP)

A. Darwiche Probabilistic Reasoning Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Maximum a Posteriori (MAP)

A. Darwiche Diagnostic System Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Bayesian Network Inference Engine okonyesno okoffyesno

A. Darwiche Agenda Propositional (Boolean) Logic Probability Calculus Independence & Causality Bayesian networks: –Markovian Assumption –Chain Rule for Bayesian Networks –d-separation

A. Darwiche Propositional Logic

A. Darwiche Probablity Calculus

A. Darwiche Bayesian Networks

A. Darwiche A Bayesian Network Compact representation of a probability distribution: –Complete model –Consistent model Embeds many independence assumptions: –Faithful model

A. Darwiche

A Bayesian Network Compact representation of a probability distribution: –Complete model –Consistent model Embeds many independence assumptions: –Faithful model

A. Darwiche Bayesian Network Earthquake (E) Burglary (B) Alarm (A) Pr(E=true)Pr(E=false).1.9 Pr(B=true)Pr(B=false).2.8 Pr(A=true)Pr(A=false) E=true, B=true E=false, B=true.9.1 E=true, B=false.7.3 E=false, B=false.01.99

A. Darwiche Joint Probability Distribution EBAPr(.) True.019 True False.001 TrueFalseTrue.056 TrueFalse.024 FalseTrue.162 FalseTrueFalse.018 False True.0072 False.7128

A. Darwiche Independence Assumptions of a Bayesian Network

A. Darwiche Chol Test1Test2 Causal Structure I(Test1,Test2 | Chol)

A. Darwiche Chol Test1Test2 Causal Structure Nurse I(Test1,Test2 | Chol, Nurse) I(Test1,Test2 | Chol)

A. Darwiche H O1On Naïve Bayes O2 H: Disease O1, …, On: Findings (symptoms, lab tests, …) …

A. Darwiche Genetic Tracking G1G2 G3G4G5 G6 G7G8P4 Each node is independent of its non-descendants given its parents

A. Darwiche Genetic Tracking G1G2 G3G4G5 G6 G7G8P4 Each node is independent of its non-descendants given its parents

A. Darwiche Genetic Tracking G1G2 G3G4G5 G6 G7G8P4 Each node is independent of its non-descendants given its parents

A. Darwiche Dynamic Systems S1 O1 S2 O2 S3 O3 S4 O4 S5 O5 Each node is independent of its non-descendants given its parents

A. Darwiche Dynamic Systems S1 O1 S2 O2 S3 O3 S4 O4 S5 O5 Each node is independent of its non-descendants given its parents

A. Darwiche The chain rule for Bayesian Networks

A. Darwiche Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) Pr(c|a) Pr(craeb)=Pr(c|raeb)Pr(r|aeb)Pr(a|eb)Pr(e|b)Pr(b) Pr(r|e)Pr(a|eb)Pr(e)Pr(b) Pr(e)Pr(b) Pr(a|eb) Pr(r|e) Pr(c|a)

A. Darwiche Example: Build Joint Probability Table Earthquake (E) Burglary (B) Alarm (A) Pr(E=true)Pr(E=false).1.9 Pr(B=true)Pr(B=false).2.8 Pr(A=true)Pr(A=false) E=true, B=true E=false, B=true.9.1 E=true, B=false.7.3 E=false, B=false.01.99

A. Darwiche Temperature/Sensors Temperature: high (20%), low (10%), nominal (70%) 3 Sensors (true, false): true (90%) given high temperature true (1%) given low temperature true (5%) given nominal temperature

A. Darwiche

Queries Pr(Sensor1=true)? Pr(Temperature=high | Sensor1=true)? Pr(Temperature=high | Sensor1=true, Sensor2=true, Sensor3=true)?

A. Darwiche d-separation

Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) … (F) Is A Independent of R given E?

A. Darwiche Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) Chain Link E & C not d-separated …Active!

A. Darwiche Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) Chain Link E & C are d-separated by A …Blocked!

A. Darwiche Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) Divergent Link R & A not d-seperated …Active!

A. Darwiche Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) Divergent Link R & A d-separated by E …Blocked!

A. Darwiche Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) Convergent Link E & B d-seperated …Blocked!

A. Darwiche Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) Convergent Link E & B not d-separated by A …Active!

A. Darwiche Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) Convergent Link E & B not d-separated by C …Active!

A. Darwiche Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) Are B & R d-separated by E & C ? Active Blocked

A. Darwiche Earthquake (E) Burglary (B) Alarm (A) Call (C) Radio (R) Active Are C & R d-separated ?

A. Darwiche blocked active

A. Darwiche d-separation Nodes X are d-separated from nodes Y by nodes Z iff every path from X to Y is blocked by Z. A path is blocked by Z if some link on the path is blocked: –For some →X→ or ←X→, X in Z –For some →X←, neither X nor one of its descendents in Z

A. Darwiche d-separation in Asia Network Visit to Asia / Smoker: –No evidence: No –Given TB-or-Cancer: Yes –Given +ve X-Ray: Yes Visit to Asia / +ve X-ray: –No evidence: Yes –Given TB: No –Given TB-or-Cancer: No Bronchitis / Lung Cancer: –No evidence: Yes –Given Smoker: No –Given Smoker and Dysnpnoea: Yes

A. Darwiche Building Bayesian Networks

A. Darwiche Three Steps Identifying variables Catching the structure Defining the CPTs

A. Darwiche Identifying Variables Hypothesis variables Information variables (observables) Others (mediate relationships)

A. Darwiche Car Start Problem “In the morning, my car will not start. I can hear the starter turn, but nothing happens. There May be several reasons for my problem. I can hear the starter roll, so there must be power in the battery. Therefore, the most probable causes are that the fuel has been stolen overnight or that the spark plugs are dirty. It may also be due to dirt in the carburetor, a leak in the ignition system, or something more serious. To find out, I first look at the fuel meter. It shows ½ full, so I decide to clean the spark plugs”

A. Darwiche Car Start Problem “In the morning, my car will not start. I can hear the starter turn, but nothing happens. There May be several reasons for my problem. I can hear the starter roll, so there must be power in the battery. Therefore, the most probable causes are that the fuel has been stolen overnight or that the spark plugs are dirty. It may also be due to dirt in the carburetor, a leak in the ignition system, or something more serious. To find out, I first look at the fuel meter. It shows ½ full, so I decide to clean the spark plugs” Fuel Meter Standing Fuel? Start? Clean Spark Plugs

A. Darwiche Fuel Meter Standing “FM” Fuel? “Fu” Start? “St” Clean Spark Plugs “Sp” Fu = yes Fu = no FM = full FM = ½ FM = empty Fu = yes Fu = no Sp = yes (0.99, 0.01) (0,1) Sp = no (0.01, 0.99) (0,1) P(FM | Fu) P(St | Fu,Sp)

A. Darwiche Milk test “Milk from a cow may be infected. To detect whether the milk is infected, you have a test, which may give either a positive or a negative test result. The test is not perfect. It may give a positive result on a clean milk as well as a negative result on infected milk” Infected?Test

A. Darwiche Digital Systems...

A. Darwiche Digital Systems Build network structure Specificy CPTs Fault modes What if we have two test vectors Single vs multiple faults How to pose queries Synthesis automatically

A. Darwiche Channel Coding Information bits: U 1 … U k Redundant bits: X 1 …X m Code word: U 1 …U k X 1 … X m (Channel input) Channel output: Y 1 …Y k+m Given channel output Y, restore the channel input

A. Darwiche Channel Coding U1 U2 U4 Y1 Y2 Y3 Y4 Y5 Y6Y7 X1 X2 X3 U3 (7,4) Hamming code

A. Darwiche Noisy-Or

E C1C1 C2C2 CnCn... LNoisy-OR CnCn C2C2 C1C1 L E Global enabler (leak) 1

A. Darwiche Let v be an assignment of truth values to C1…Cn Let S contain all indices i such that Ci=true E... CnCn C2C2 C1C1 L Global enabler (leak) 1

A. Darwiche O(n) parameters instead of O(2 n ) CPCS network : 448 nodes 906 links 8254 parameters; instead of 134 million! Noisy-OR

A. DarwicheNoisy-Or 5% of the mornings yields a sore throat (l =.95) Cold causes a sore throat with probability 0.4; (q=.6) Angina causes a sore throat with probability 0.7 (q=.3) Cold? SoreThroat? Angina?

A. DarwicheNoisy-Or Cold? SoreThroat? Angina? Angina? = noAngina? = yes Cold? = no * 0.3 Cold? = yes0.95 * * 0.3 * 0.6 P(SoreThroat? = no | Cold?, Angina? )

A. DarwicheNoisy-Or Angina? = noAngina? = yes Cold? = no * 0.3 Cold? = yes * * 0.3 * 0.6 P(SoreThroat? = yes | Cold?, Angina? ) Cold? SoreThroat? Angina?