Bayesian Networks COSC 4426 Brad Legault. Bayes’ Theorem “Bayes’ theorem is to probability theory what Pythagora’s theorem is to geometry.”  Sir Harold.

Slides:



Advertisements
Similar presentations
Artificial Intelligence 12. Two Layer ANNs
Advertisements

Heuristic Search techniques
BAYESIAN NETWORKS Ivan Bratko Faculty of Computer and Information Sc. University of Ljubljana.
BAYESIAN NETWORKS. Bayesian Network Motivation  We want a representation and reasoning system that is based on conditional independence  Compact yet.
For Monday Read chapter 18, sections 1-2 Homework: –Chapter 14, exercise 8 a-d.
For Monday Finish chapter 14 Homework: –Chapter 13, exercises 8, 15.
Karnaugh Maps. What are karnaugh maps? Boolean algebra can be represented in a variety of ways. These include: Boolean expressions Truth tables Circuit.
STAT Section 5 Lecture 23 Professor Hao Wang University of South Carolina Spring 2012 TexPoint fonts used in EMF. Read the TexPoint manual before.
Copyright © 2005 Brooks/Cole, a division of Thomson Learning, Inc. 6.1 Chapter Six Probability.
Chapter 19 Confidence Intervals for Proportions.
From: Probabilistic Methods for Bioinformatics - With an Introduction to Bayesian Networks By: Rich Neapolitan.
Planning under Uncertainty
Integrating Bayesian Networks and Simpson’s Paradox in Data Mining Alex Freitas University of Kent Ken McGarry University of Sunderland.
Rotem Golan Department of Computer Science Ben-Gurion University of the Negev, Israel.
Mr Barton’s Maths Notes
CS 589 Information Risk Management 6 February 2007.
CS 589 Information Risk Management 23 January 2007.
CS 188: Artificial Intelligence Spring 2007 Lecture 14: Bayes Nets III 3/1/2007 Srini Narayanan – ICSI and UC Berkeley.
Sampling Distributions
CS 188: Artificial Intelligence Fall 2006 Lecture 17: Bayes Nets III 10/26/2006 Dan Klein – UC Berkeley.
Bayesian Networks Alan Ritter.
A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graphs.
Lesson 4: Percentage of Amounts.
Artificial Intelligence CS 165A Tuesday, November 27, 2007  Probabilistic Reasoning (Ch 14)
Psy B07 Chapter 1Slide 1 ANALYSIS OF VARIANCE. Psy B07 Chapter 1Slide 2 t-test refresher  In chapter 7 we talked about analyses that could be conducted.
Intermediate Statistical Analysis Professor K. Leppel.
Chapter 1 Basics of Probability.
1 9/8/2015 MATH 224 – Discrete Mathematics Basic finite probability is given by the formula, where |E| is the number of events and |S| is the total number.
1 Psych 5500/6500 Chi-Square (Part Two) Test for Association Fall, 2008.
7 Graph 7.1 Even and Odd Degrees.
Mathematical Methods Wallands Community Primary School
1 9/23/2015 MATH 224 – Discrete Mathematics Basic finite probability is given by the formula, where |E| is the number of events and |S| is the total number.
Slides are based on Negnevitsky, Pearson Education, Lecture 12 Hybrid intelligent systems: Evolutionary neural networks and fuzzy evolutionary systems.
Copyright © Cengage Learning. All rights reserved. Elementary Probability Theory 5.
Tonga Institute of Higher Education Design and Analysis of Algorithms IT 254 Lecture 8: Complexity Theory.
Chapter 6 Lecture 3 Sections: 6.4 – 6.5.
Bayesian networks. Motivation We saw that the full joint probability can be used to answer any question about the domain, but can become intractable as.
For Wednesday Read Chapter 11, sections 1-2 Program 2 due.
1 Monte Carlo Artificial Intelligence: Bayesian Networks.
Introduction to Bayesian Networks
Playing the Lottery 6/4/2016Copyright © 2010… REMTECH, inc … All Rights Reserved1 Probability Theory and the Lottery ● The Lottery picks numbers at random.
Instructor: Prof. Pushpak Bhattacharyya 13/08/2004 CS-621/CS-449 Lecture Notes CS621/CS449 Artificial Intelligence Lecture Notes Set 7: 29/10/2004.
METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Lecture notes 9 Bayesian Belief Networks.
CS 416 Artificial Intelligence Lecture 14 Uncertainty Chapters 13 and 14 Lecture 14 Uncertainty Chapters 13 and 14.
Slides for “Data Mining” by I. H. Witten and E. Frank.
POSC 202A: Lecture 4 Probability. We begin with the basics of probability and then move on to expected value. Understanding probability is important because.
Graphs A ‘Graph’ is a diagram that shows how things are connected together. It makes no attempt to draw actual paths or routes and scale is generally inconsequential.
Bayesian networks and their application in circuit reliability estimation Erin Taylor.
Weight & Normal Force Weight  The force of gravity on an object.
Chapter 6 Lecture 3 Sections: 6.4 – 6.5. Sampling Distributions and Estimators What we want to do is find out the sampling distribution of a statistic.
Learning and Acting with Bayes Nets Chapter 20.. Page 2 === A Network and a Training Data.
1 Bayesian Networks: A Tutorial. 2 Introduction Suppose you are trying to determine if a patient has tuberculosis. You observe the following symptoms:
Probability. What is probability? Probability discusses the likelihood or chance of something happening. For instance, -- the probability of it raining.
Today Graphical Models Representing conditional dependence graphically
Brad Legault Soft Computing CONDITIONAL DEPENDENCE & INDEPENDENCE.
Graphs. Graph Definitions A graph G is denoted by G = (V, E) where  V is the set of vertices or nodes of the graph  E is the set of edges or arcs connecting.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Classification COMP Seminar BCB 713 Module Spring 2011.
Slide 1 Directed Graphical Probabilistic Models: inference William W. Cohen Machine Learning Feb 2008.
The Law of Averages. What does the law of average say? We know that, from the definition of probability, in the long run the frequency of some event will.
Weng-Keen Wong, Oregon State University © Bayesian Networks: A Tutorial Weng-Keen Wong School of Electrical Engineering and Computer Science Oregon.
Chapter 12. Probability Reasoning Fall 2013 Comp3710 Artificial Intelligence Computing Science Thompson Rivers University.
By: Megan Funk. I will: 1. Explain the binary number system How to: -Generate binary from a number -Add binary 2. Explain the base-b number system 3.
Final Exam Information These slides and more detailed information will be posted on the webpage later…
 Negnevitsky, Pearson Education, Lecture 12 Hybrid intelligent systems: Evolutionary neural networks and fuzzy evolutionary systems n Introduction.
Discrete Event Simulation - 4
Propagation Algorithm in Bayesian Networks
Fundamentals of Data Representation
Hankz Hankui Zhuo Bayesian Networks Hankz Hankui Zhuo
Probabilistic Reasoning
Presentation transcript:

Bayesian Networks COSC 4426 Brad Legault

Bayes’ Theorem “Bayes’ theorem is to probability theory what Pythagora’s theorem is to geometry.”  Sir Harold Jeffreys Bayes’ Theorem concerns how to “update” the probability of a particular event given certain related events. This is a fancy way of saying, “What is the probability of event A happening given that event B has happened?” It is the fundamental theorem behind Bayesian Networks

Example There is a disease that is currently hitting the population for the first time. There is a 10% chance that any given person will have the disease. The medical community has developed a test which can detect the presence of the disease with 92% accuracy. Unfortunately, the test has a 5% chance of giving a false positive result to people who do not have the disease. Question: You just took the test and it returned as positive. What are the chances you actually have the disease?

Tree Time

The Math

So what?  Given partial knowledge, one can make a “best guess” for a course of action or determine if a particular risk is worth the reward  Could apply to pretty much any field where decision making and risk exists  It can also be used for comparing relative strengths of probability, which is fundamental to the project I’m going to be doing

Bayesian Networks  A Bayesian Network is a graphical probabilistic model used to show cause and effect of certain events in a closed system  It requires two things: 1. Directed Acyclic Graph (DAG) 2. Conditional Probability Table (CPT) (There are usually multiple CPT’s for a single Network)

DAG  Directed Acyclic Graphs are something we’ve been using in a few of our courses here (Discrete II, Data Structures, etc.)  It’s just a graph where the edges have directionality (arrows rather than lines) and there are no cycles (ie: you can’t go around in circles).  A binary tree is a DAG, for example

Nodes & Edges  A node in a Bayesian Network concerns an event  If one event affects another event (ie: helps determine another event’s outcome), the “affector” node will have a directed edge pointing to the “affectee”  If you find a cycle in your graph (not a DAG), you need to introduce more nodes to explain the relationship

Example You’re away on vacation, preferably somewhere with no snow. You get a call from your neighbour saying that your house alarm is going off. You immediately fear a burglar is stealing your prized Lionel Ritchie music collection, and wonder if you should call the police. Here’s what you know: Your neighbour is getting old and has been known to hear things Every once and awhile, a big truck driving by will rattle your windows and set the alarm off If there is a burglar, they probably would set off the alarm and definitely steal your music collection (it’s Lionel Ritchie, after all)

Example DAG This is a fairly simple example of how the DAG could be drawn Events T and B directly affect event A, so A is said to be a “child” of T and B. T and B are parents of A. Event N is a “descendent” of T and B. T and B are “ancestors” of N.

CPT  Conditional Probability Tables give a probability for each event depending on the outcome of the parent(s) event  If an event has no parents, it is simply a P(true) and P(false) for the event  Bayes Theorem is made easier when calculating the probability of an outcome over multiple events using these CPTs.

Parentless CPT’s  These are very simple, just a straight probability since they are not dependent on anything Event T Probability True0.008 False0.992 Event B Probability True False0.9999

Normal CPT  Most events in a Bayesian Network have dependencies (parents), which means their probabilities are based on the occurrence or lack thereof of other events.  For example, the chances the neighbour reporting the alarm vary greatly on the event of the alarm actually going off. Event A Event N Pr ( N | A ) True 0.72 TrueFalse0.28 FalseTrue0.01 False 0.99

Abbreviated CPT CPT’s do not need all their rows. The 1 st and 2 nd rows add to 1, as do the 3 rd and 4 th. As such, the table on the previous page can actually leave those out, and the person reading the table can deduce those values with subtraction, in order to make the network less cluttered. Event A Event N Pr ( N | A ) True 0.72 FalseTrue0.01

Multiple Parents  Gets tricky when an event has multiple parents, since there is now more than one variable on which it is dependent. This tree below represents the possibilities of the dependent variables:

Create CPT TBAPr(A|T,B) True True False TrueFalseTrue0.75 TrueFalse 0.25 FalseTrue 0.85 FalseTrueFalse0.15 False True0 False 1 With the additional dependency, the table gets much larger very quickly. This example uses the probability that a truck has a 0.75 chance of setting off the alarm, and a burglar has a 0.85 chance of setting off the alarm. Notice every pair of rows sums to 1. This means it can be abbreviated.

Create CPT TBAPr(A|T,B) True TrueFalseTrue0.75 FalseTrue 0.85 False True0 That’s much easier to fit on a diagram, and contains the same information (besides a couple subtractions that are easily done).

Now it’s ready!  Once each node in the DAG has a conditional probability table assigned to it, the diagram is now a Bayesian Network  Notice the 2 nd last column is always all “true”? It can also be condensed for the purpose of space

My Project  The project I plan to create is an offshoot of work done by an engineering PhD student from the University of Toronto, named Samah El-Tantawy  She designed and implemented a system using Computational Intelligence (hybrid) techniques which reduced congestion at certain busy intersections by up to 40%

My Project  My system will be far less involved than Ms. El- Tantawy’s (hers was a much lauded masters’ thesis, not just an undergrad-level project)  Mine will be much simpler, and will concern itself with a 3 x 3 grid of lights on a street  Will count volume of cars and/or wait time of cars (still figuring out which is better) going through the intersections  The intersections will then be modelled using Bayesian Networks

My Project  Will likely use multiple networks which will need to be weighted against each other using another model (hopefully, nothing too complicated)  The goal will be to try and reduce wait times for all vehicles  This is an optimisation process with an element of probability involved (exact volume), so not easy to solve

My Project The project will have 3 phases: 1. Mapping intersections and creating Bayesian Networks 2. Observation of vehicles passing through and/or wait times at lights 3. Analysis & adjustment of stop lights based on calculations to reduce overall wait times Phases 2 and 3 will repeat several times to generate more optimal solution

Resources I’m using two books primarily for this project: 1. Probabilistic Graphical Models (Daphne Koller, Stanford) 2. Modeling and Reasoning with Bayesian Networks (Adnan Darwiche, UCLA) Adnan Darwiche is also the Editor-in-Chief for the Journal of Artificial Intelligence Research ( which contains a lot of articles related to AI / CI and is free to downloadwww.jair.org

Thank you This concludes my presentation. Thanks for listening.