Comp 538 Course Presentation Discrete Factor Analysis Learning Hidden Variables in Bayesian Network Calvin Hua & Lily Tian Computer Science Dep, HKUST.

Slides:



Advertisements
Similar presentations
Bayesian network for gene regulatory network construction
Advertisements

Autonomic Scaling of Cloud Computing Resources
Big Ideas in Cmput366. Search Blind Search State space representation Iterative deepening Heuristic Search A*, f(n)=g(n)+h(n), admissible heuristics Local.
CSE 473/573 Computer Vision and Image Processing (CVIP) Ifeoma Nwogu Lecture 27 – Overview of probability concepts 1.
A Tutorial on Learning with Bayesian Networks
Naïve Bayes. Bayesian Reasoning Bayesian reasoning provides a probabilistic approach to inference. It is based on the assumption that the quantities of.
Variational Methods for Graphical Models Micheal I. Jordan Zoubin Ghahramani Tommi S. Jaakkola Lawrence K. Saul Presented by: Afsaneh Shirazi.
Lauritzen-Spiegelhalter Algorithm
BAYESIAN NETWORKS. Bayesian Network Motivation  We want a representation and reasoning system that is based on conditional independence  Compact yet.
Dynamic Bayesian Networks (DBNs)
EE462 MLCV Lecture Introduction of Graphical Models Markov Random Fields Segmentation Tae-Kyun Kim 1.
GS 540 week 6. HMM basics Given a sequence, and state parameters: – Each possible path through the states has a certain probability of emitting the sequence.
Progress Report Reihaneh Rabbany Presented for NLP Group Computing Science Department University of Alberta April 2009.
Overview Full Bayesian Learning MAP learning
Bayesian Learning Rong Jin. Outline MAP learning vs. ML learning Minimum description length principle Bayes optimal classifier Bagging.
Review: Bayesian learning and inference
Date:2011/06/08 吳昕澧 BOA: The Bayesian Optimization Algorithm.
December Marginal and Joint Beliefs in BN1 A Hybrid Algorithm to Compute Marginal and Joint Beliefs in Bayesian Networks and its complexity Mark.
Visual Recognition Tutorial
Computer vision: models, learning and inference Chapter 10 Graphical Models.
EM Algorithm Likelihood, Mixture Models and Clustering.
Computer Science 1 Web as a graph Anna Karpovsky.
1 Bayesian Networks Chapter ; 14.4 CS 63 Adapted from slides by Tim Finin and Marie desJardins. Some material borrowed from Lise Getoor.
Cristina Manfredotti D.I.S.Co. Università di Milano - Bicocca An Introduction to the Use of Bayesian Network to Analyze Gene Expression Data Cristina Manfredotti.
Ant Colony Optimization: an introduction
1 Naïve Bayes A probabilistic ML algorithm. 2 Axioms of Probability Theory All probabilities between 0 and 1 True proposition has probability 1, false.
Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields Yong-Joong Kim Dept. of Computer Science Yonsei.
A Brief Introduction to Graphical Models
Learning Structure in Bayes Nets (Typically also learn CPTs here) Given the set of random variables (features), the space of all possible networks.
Crowdsourcing with Multi- Dimensional Trust Xiangyang Liu 1, He He 2, and John S. Baras 1 1 Institute for Systems Research and Department of Electrical.
Visibility Graph. Voronoi Diagram Control is easy: stay equidistant away from closest obstacles.
Bayesian networks. Motivation We saw that the full joint probability can be used to answer any question about the domain, but can become intractable as.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 28 of 41 Friday, 22 October.
Graph-based Text Classification: Learn from Your Neighbors Ralitsa Angelova , Gerhard Weikum : Max Planck Institute for Informatics Stuhlsatzenhausweg.
Ch 8. Graphical Models Pattern Recognition and Machine Learning, C. M. Bishop, Revised by M.-O. Heo Summarized by J.W. Nam Biointelligence Laboratory,
Course files
Announcements Project 4: Ghostbusters Homework 7
LAC group, 16/06/2011. So far...  Directed graphical models  Bayesian Networks Useful because both the structure and the parameters provide a natural.
Marginalization & Conditioning Marginalization (summing out): for any sets of variables Y and Z: Conditioning(variant of marginalization):
K2 Algorithm Presentation KDD Lab, CIS Department, KSU
Lecture 2: Statistical learning primer for biologists
DeepDive Model Dongfang Xu Ph.D student, School of Information, University of Arizona Dec 13, 2015.
Learning and Acting with Bayes Nets Chapter 20.. Page 2 === A Network and a Training Data.
1 Param. Learning (MLE) Structure Learning The Good Graphical Models – Carlos Guestrin Carnegie Mellon University October 1 st, 2008 Readings: K&F:
1 CMSC 671 Fall 2001 Class #20 – Thursday, November 8.
Computer vision: models, learning and inference Chapter 2 Introduction to probability.
Bayesian Optimization Algorithm, Decision Graphs, and Occam’s Razor Martin Pelikan, David E. Goldberg, and Kumara Sastry IlliGAL Report No May.
Introduction on Graphic Models
Today Graphical Models Representing conditional dependence graphically
A Cooperative Coevolutionary Genetic Algorithm for Learning Bayesian Network Structures Arthur Carvalho
1 Relational Factor Graphs Lin Liao Joint work with Dieter Fox.
1 Structure Learning (The Good), The Bad, The Ugly Inference Graphical Models – Carlos Guestrin Carnegie Mellon University October 13 th, 2008 Readings:
Bayesian Brain Probabilistic Approaches to Neural Coding 1.1 A Probability Primer Bayesian Brain Probabilistic Approaches to Neural Coding 1.1 A Probability.
Ch 6. Markov Random Fields 6.1 ~ 6.3 Adaptive Cooperative Systems, Martin Beckerman, Summarized by H.-W. Lim Biointelligence Laboratory, Seoul National.
Graphical Models for Segmenting and Labeling Sequence Data Manoj Kumar Chinnakotla NLP-AI Seminar.
An Algorithm to Learn the Structure of a Bayesian Network Çiğdem Gündüz Olcay Taner Yıldız Ethem Alpaydın Computer Engineering Taner Bilgiç Industrial.
On the Ability of Graph Coloring Heuristics to Find Substructures in Social Networks David Chalupa By, Tejaswini Nallagatla.
Exam Preparation Class
Markov ó Kalman Filter Localization
Data Mining Lecture 11.
Hidden Markov Models Part 2: Algorithms
Jianping Fan Dept of Computer Science UNC-Charlotte
An Algorithm for Bayesian Network Construction from Data
Markov Random Fields Presented by: Vladan Radosavljevic.
Class #19 – Tuesday, November 3
Learning Probabilistic Graphical Models Overview Learning Problems.
Authors: Wai Lam and Kon Fan Low Announcer: Kyu-Baek Hwang
Class #16 – Tuesday, October 26
Chapter 14 February 26, 2004.
A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models Jeff A. Bilmes International.
Presentation transcript:

Comp 538 Course Presentation Discrete Factor Analysis Learning Hidden Variables in Bayesian Network Calvin Hua & Lily Tian Computer Science Dep, HKUST

Objectives and Outlines Objective: - Present a space of BN topologies with hidden variables(or factors) and a method for rapidly learning an appropriate topology from data. Outline: -Motivating example -Methodology * Finding the topology * Constructing the factors - Some Results & Evaluation Q&A

Motivating Example N H S C Observable Variables: H- Hives N-Nasal Congestion C-Cough S-Sore Throat Question: 1.What independencies are encoded? 2.Is the direction of each edge unique?

Motivating Example (Cond.) VA H NC S More compact Inference easier Hidden variables used to explain dependencies and independencies

Our Work DATA SET MODEL DO INFERENCE OUR WORK Task: 1.How to find the topology given data(structure)? 2.How to construct the factors(parameters)?

Learning Factor Structure Finding the topology –Decide which observable variables each factor should cover –Decide what factors to use Constructing the factors –Determine highly probable number of values per factor –Determine highly probable conditional dependencies between factors and observable variables

Algorithm - Finding the topology Step1: –Introduce a link between two variables when they are dependent –Label each link with the probability that those two variables are dependent Step2: –Extract cliques in the graph Step3: –Perform a greedy search for the cliques

Algorithm - Step1 (Cond.) How to test whether two variables are dependent or not? –Using Chi-Squared Test

Algorithm - Step2(Cond.) Principles of extracting cliques Iterating through the variables, in each iteration we do the following : –Adding a variable to an existing clique if the variable is dependent on all other variables in that clique.

Algorithm- Step3(Cond.) Perform a greedy search for cliques –By maximizing the sum of the labels represented in the set of cliques. –Labels: the probability that those variables are dependent.

Algorithm- Constructing the factors Initialization Calculate the most probable assignment of the nth instance, I, to the values of each factor given the first n-1 instances: (1)Choose a random order of factors (2)Iterate over the factors (details later)

Algorithm- Constructing the factors (Cond.) Task: –Choose the number of values for each factor –Choose the conditional probabilities Note : –FL(Factor Learning) can do so rapidly by approximating the normative Bayesian method for learning hidden variables –The normative way should consider all possible numbers of values and all possible assignments of hidden variable values to the instances in the data set(Cooper & Herskovits, 1992; Cooper, 1994)

Algorithm- Step 2 (Cond.) 1.Compute for each value,, of the ith factor 2.Calculate the probability of a new value for the ith factor, 3.- Label the instance with the factor value with the maximum probability - Update the estimated prior probabilities of the ith factor’s values and the estimated conditional probabilities of the observable values given the factor’s value Note: In all cases where probabilities must be estimated from frequencies we use the following formula:

Some Results and Evaluation M P G C Association tested by FL M-Math P-Physics C-Chemistry G-Geology Note: In this figure, the arc’s denote the dependencies between any pair of variables

Some Results and Evaluation AM2 MP CG A-Analytic ability M2-Memory M-Math P-Physics C-Chemistry G-Geology

Some Results and Evaluation Characteristics of factor structure 1.There are hidden variables, called factors. 2.Hidden variables can interact to influence observable variables. 3.It can support polynomial time probabilistic inference. 4.The resulting network captures conditional independencies among the observable variables.