Introduction on Graphic Models Jin Mao Postdoc, School of Information, University of Arizona Jan 20, 2015

Outline Probabilistic Graphical Models Bayesian Network Markov Network Factor & Factor Graph

Probabilistic Graphical Models Definition Probabilistic graphical models use a graph-based representation as the basis for compactly encoding a complex distribution over a high-dimensional space. Nodes: The Variables Edges: Direct probabilistic interactions between the variables

Probabilistic Graphical Models Features Representationtransparent, in that a human expert can understand and evaluate its semantics and properties. Inferenceposterior probability based on the structure. Learningconstruct the model from observed data.

Probabilistic Graphical Models Types Bayesian Networks, directed Markov Networks , indirected

Bayesian Networks Definition Bayesian networks (BNs) are graphical models for reasoning under uncertainty. The nodes in a Bayesian network represent a set of random variables, X = X1,.. Xi,.. Xn, from the domain.

Bayesian Networks Definition Bayesian networks (BNs) are graphical models for reasoning under uncertainty. The nodes in a Bayesian network represent a set of random variables, X = X1,.. Xi,.. Xn, from the domain. Types of nodes: Boolean: true/false Ordered: low/medium/high Integral: 1- 20 …

directed acyclic graphs, or simply dags. Bayesian Networks Definition A set of directed arcs (or links) connects pairs of nodes, Xi Xj, representing the direct dependencies between variables. Assuming discrete variables, the strength of the relationship between variables is quantified by conditional probability distributions associated with each node: P(Xj|Xi). directed acyclic graphs, or simply dags.

Model by Knowledge Engineer Bayesian Networks Model by Knowledge Engineer Identify variables of interests; Define the states of variables: both mutually exclusive and exhaustive. To choose values that represent the domain efficiently, but with enough detail to perform the reasoning required; Design the structure/topology. Determine conditional probabilities. Use it for prediction

Bayesian Networks An example

Bayesian Networks An example

Structure Terminology Bayesian Networks Structure Terminology a parent of a child an ancestor of a descendant Root node, leaf node, intermediate node Markov blanket the node’s parents, its children, and its children’s parents.

Conditional Probabilities Bayesian Networks Conditional Probabilities A conditional probability table (CPT), discrete variables. First, look at all the possible combinations of values of those parent nodes. (instantiation) For each distinct instantiation of parent node values, we need to specify the probability that the child will take each of its values by some means. Pollution and Smoking and take the possible joint values < H;T >;< H;F >;< L;T >; < L;F > < 0:05;0:02;0:03;0:001 > Root nodes---prior probablities

Independence-maps (or, I-maps for short) Bayesian Networks The Markov property there are no direct dependencies in the system being modeled which are not already explicitly shown via arcs. Independence-maps (or, I-maps for short) it is not generally required that the arcs in a BN correspond to real dependencies in the system. Redundant edges, minimal I-maps Every arc happens to correspond to a real direct dependence---Dependence-Map Both D-map and I-map, A perfect map. no hidden “backdoor”

diagnostic reasoning: Bayesian Networks Types of reasoning diagnostic reasoning: reasoning from symptoms to cause, the opposite direction to the network arcs.

Predictive reasoning: Bayesian Networks Types of reasoning Predictive reasoning: reasoning from new information about causes to new beliefs about effects, following the directions of the network arcs.

intercausal reasoning: the mutual causes of a common effect Bayesian Networks Types of reasoning intercausal reasoning: the mutual causes of a common effect

Specific Evidence: observe a specific value Bayesian Networks Types of Evidence Specific Evidence: observe a specific value Negative Evidence: not a value Virtual evidence, likelihood evidence, not so sure. Suppose, for example, that the radiologist who has taken and analyzed the Xray in our cancer example is uncertain. He thinks that the X-ray looks positive, but is only 80% sure.

Bayesian Networks Learning

AKA Markov random field (MRF) Markov Networks Definition Undirected graphs can also be used to represent dependency relationships. Useful in modeling domains where the interactions between the variables seem symmetrical and one cannot naturally ascribe a directionality to the interaction between variables AKA Markov random field (MRF)

AKA Markov random field (MRF) Markov Networks Definition Undirected graphs can also be used to represent dependency relationships. Useful in modeling domains where the interactions between the variables seem symmetrical and one cannot naturally ascribe a directionality to the interaction between variables AKA Markov random field (MRF)

Markov Networks Moralization: Moral graph of G From BN to MN Moralization: Start with the DAG G, add an edge between every variable and each of its spouses, and finally drop the directionality of all edges in DAG G. Moral graph of G The moralization process ensures that every variable in DAG G will be connected to each variable in its Markov Blanket

Factor & Factor Graph A Factor(potential) Factors A Factor(potential) A factor over a set of variables X is a function which maps each instantiation x of variables X to a non-negative real number, denoted. The variables in a factor often have dependency relationships. The result of multiplying factors is another factor.

Factor & Factor Graph Factorization

Factor & Factor Graph Factor Graph A factor graph is an undirected graph containing two types of nodes: variable nodes and factor nodes. The graph only contains edges between variable nodes and factor nodes. Each factor node Vφ is associated with precisely one factor φ, whose scope is the set of neighbors of Vφ.

Factor & Factor Graph Why Factor Graph They are used extensively for breaking down a problem into pieces. “We can simplify computations based on how variables are related to these factors. We’ll break up the joint distribution into a bunch of factors on a graph. ” (read: http://www.moserware.com/2010/03/computing-your-skill.html)

Thank you!