1. SampleSearch: Importance Sampling in the presence of Determinism
Rina Dechter
Bren school of Information and Computer Sciences
University of California, Irvine
Graphical models are the most popular knowledge representation schemes in AI and computer sciences for representing and reasoning with probabilistic and deterministic relationship.
Exact reasoning is hard when the graph is dense (treewidth is high) so approximation schemes must be used at some point. Approximation with guarantees are hard also, but there are not as difficult when the information is just probabilistic. No strict constraints. However when some of the knowledge involves constraints approximatioin is hard as well. Iin particular sampling schemes becomes hard.
In this talk we will address this issue of sampling in the presence of hard constraints.
I will present SampleSearch which sample and search and analyze the implication of the ideas for importance sampling. We will identify the rejection problem, show how it can be hanbdled and when it is cost-effective.
Empirical evaluation will demonstrate how cost-effective the scheme is.
I will then move to the orthogonal approach of exploiting problem structure in sampling
And show our strategy.
Joint work with Vibhav Gogate

2. Overview Introduction: Mixed graphical models
SampleSearch: Sampling with Searching
Exploiting structure in samplinig: AND/OR Importance sampling
Here is a brief outline of my talk.
I’ll start by describing some preliminaries and motivating applications.
Then I’ll describe the main problem which we investigated in this thesis which is that of poor performance of sampling based algorithms in presence of determinism.
Then I’ll describe the four main contributions of this thesis.

3. Overview Introduction: Mixed graphical models
SampleSearch: Sampling with Searching
Exploiting structure in samplinig: AND/OR Importance sampling
Here is a brief outline of my talk.
I’ll start by describing some preliminaries and motivating applications.
Then I’ll describe the main problem which we investigated in this thesis which is that of poor performance of sampling based algorithms in presence of determinism.
Then I’ll describe the four main contributions of this thesis.

4. Bayesian Networks: Representation (Pearl, 1988)
P(D|C,B)
P(B|S)
P(S)
P(X|C,S)
P(C|S)
Smoking
lung Cancer
Bronchitis
X-ray
Dyspnoea
Bayesian networks are a compact representation of a joint distribution. Here we assume only discrete distributions. Given a directed acyclic graph over a set of random variables, the joint distribution over all the random variables can be expressed as a product of various conditional distributions or CPTs defined as P(X) given its parents in the graph.
Two queries are of interest. Belief Updating
P(S, C, B, X, D) = P(S) P(C|S) P(B|S) P(X|C,S) P(D|C,B)
Belief Updating:
P (lung cancer=yes | smoking=no, dyspnoea=yes ) = ?

5. Mixed Networks: Mixing Belief and Constraints
Belief or Bayesian Networks
Constraint Networks A A D B C E F F B C B C
D=0
D=1 1 .1 .9 .3 .7 E D B= R=
The focus is on graphical models having determinism and mixed networks is a unifying framework for such graphical models.
A graphical model captures knowledge using a collection of functions whose interaction can be captured by a graph representation.
We have Bayesian networks to allow reasoning under conditions of uncertainty. A Bayesian network is a compact representation of a joint probability distribution. Given a directed acyclic graph over random variables, the joint distribution can be expressed as product of conditional probability tables (CPTS) defined over each variable. Each CPT is specified using the variable given its parents in the network.
Constraint networks are deterministic graphical models which contains a set of variables, a set of values for each variable and constraints which restrict the assignments that can be made to the variables. A solution is an assignment to all variables such that all constraints are satisfied.
Mixed networks are graphical models which augment a probabilistic graphical model having determinism with a constraint network. The constraint network is basically used to model zero probabilities as constraints. The constraints can also be specified externally.
For example, the zero probabilities in the Bayesian network can be modeled as constraints as shown.
The central aim is to allow exploiting the power of constraint processing techniques for efficient probabilistic inference.
Constraints could be specified externally or may occur as zeros in the Belief network

6. Mixed networks: Distribution and Queries
The distribution represented by a mixed network T=(B,R):
Queries:
Weighted Counting (Equivalent to P(e), partition function, solution counting)
Marginal distribution:
The distribution defined by a mixed network is the distribution represented by a Bayesian network projected onto the solutions of the constraint portion. The distribution can be normalized using a constant M.
Evidence is a special type of unary constraint.
Common queries over the mixed network are computing the normalization constant M and the marginal distribution over a variable.
M is same as P(e), partition function and solution counting.
Two queries: Marginals and Counting
The process of answering these queries is called as inference and “skip to the next slide”

7. Applications
Determinism: More Ubiquitous than you may think!
Transportation Planning (Liao et al. 2004, Gogate et al. 2005)
Predicting and Inferring Car Travel Activity of individuals
Genetic Linkage Analysis (Fischelson and Geiger, 2002)
associate functionality of genes to their location on chromosomes.
Functional/Software Verification (Bergeron, 2000)
Generating random test programs to check validity of hardware
First Order Probabilistic models (Domingos et al. 2006, Milch et al. 2005)
Citation matching
Applications of mixed network.
Graphical models which have determinism occur quite frequently in many real world applications.
Examples are transportation planning in which given GPS data about a person, the problem is to infer and predict Travel activity routines of individuals.
Another application is in computational Biology, specifically genetic linkage analysis in which given a chromosome and some marker locations on the chromosome and the task is figure out if any DNA on the chromosome of the marker affects a particular disease.
A third application is functional or software verification in which you are given a circuit or a piece of code and the task is to generate test programs which test whether the circuit or the software conforms to the specification.
A final application is first order probabilistic models which have lot of determinism and are used for things like Citation matching or figuring out Author Author relationships.

8. Functions on Orange nodes are deterministic
L11m
L11f
L12m
L12f
S13m
X11
X12
S13f
L13m
L13f
X13
Model for locus 1
L21m
L21f
L22m
L22f
S23m
X21
X22
S23f
Functions on Orange nodes are deterministic
L23m
L23f
X23
Model for locus 2

9. Bayesian Network for Recombination
L11m
L11f
L12m
L12f
Locus 1
S13m
X11
X12
S13f y1 y2
L13m
L13f
Deterministic relationships
X13 y3
Probabilistic relationships
L21m
A Bayesian network for 3 people and two loci is given here. Theta is the parameter controling the distance between the two markers.
The linkage instances are interesting as graphical models because they involve both probabilistic and deterministic relationships,
L21f
L22m
L22f
Locus 2
S23m
X21
X22
S23f
L23m
L23f
P(e|Θ) ?
X23 9
UNL, April 2009 9

10. Linkage analysis: 6 people, 3 markers
L11m
L11f
L12m
L12f
X11
X12
S15m
S13m
L13m
L13f
L14m
L14f
X13
X14
S15m
S15m
L15m
L15f
L16m
L16f
S15m
S16m
X15
X16
L21m
L21f
L22m
L22f
X21
X22
S25m
S23m
L23m
L23f
L24m
L24f
X23
X24
S25m
S25m
L25m
L25f
L26m
L26f
Here is an example of using Bayesian networks for linkage analysis.
It models the genetic inheritance in a family of 6 individuals relative to some genes of interest.
The task is to find a desease gene on a chromosome
This domain yields very hard probabilistic networks that contain both probabilistic information and deterministic relationship
And it drives many of the methods that we currently develop. .
S25m
S26m
X25
X26
L31m
L31f
L32m
L32f
X31
X32
S35m
S33m
L33m
L33f
L34m
L34f
X33
X34
S35m
S35m
L35m
L35f
L36m
L36f
S35m
S36m
UNL, April 2009
X35
X36

11. Approximate Inference
Approximations are hard with determinism
Randomized Polynomial ε-approximation possible when no zeros are present (Karp 1993, Cheng 2001)
ε-approximation NP-hard in the presence of zeros
Gibbs sampling is problematic when MCMC is not ergodic.
Current remedies
Replace zeros with very small values (Laplace correction: Naive Bayes, NLP)
bad performance when zeros or determinism is real!
The goal is to perform effective sampling based approximate inference in presence of determinism.
As compared with exact inference which in fact becomes easier when determinism is present in the graphical model, approximate inference rather counter-intuitively becomes harder!
In particular, there exists a randomized polynomial time algorithm that bounds the marginal probabilities with relative error epsilon when there is no determinism. However, when determinism is present the problem of bounding marginal probabilities with relative error epsilon suddenly becomes NP-hard
Other approaches in Machine learning community and NLP for tackling zeros use something called the Laplace correction. Namely replace all zeros by very small values and continue. This works quite well when determinism is not real in machine learning for instance. Example: missing data. However, when determinism is real, laplace correction can yield very bad performance.

12. Overview Introduction: Mixed graphical models
SampleSearch: Sampling with Searching
Rejection problem
Recovery, and analysis
Empirical evaluation
Exploiting structure in samplinig: AND/OR Importance sampling
Here is a brief outline of my talk.
I’ll start by describing some preliminaries and motivating applications.
Then I’ll describe the main problem which we investigated in this thesis which is that of poor performance of sampling based algorithms in presence of determinism.
Then I’ll describe the four main contributions of this thesis.

13. Importance Sampling: Overview
Given a proposal or importance distribution Q(z) such that f(z)>0 implies Q(z)>0, rewrite
Given i.i.d. samples z1,..,zN from Q(z),
Importance sampling is a general technique which can be used to approximate any sum of the form M where f is a function over a set of variables.
The idea is to multiply and divide by a distribution Q(z) and transform the sum into an expectation w.r.t. the distribution Q. Q is often called the proposal distribution or the importance distribution and that is why this scheme is called as importance sampling. Q could be any distribution as long as it satisfies the condition shown. However, for effective performance Q should be as close as possible to the posterior distribution of the mixed network.
Then one generates samples z1 to zn from Q and estimates the sum by an average over the samples.

14. How to generate i.i.d. samples from Q(X)
We have n variables

15. Generating i.i.d. samples from Q
Root
0.8
0.2
A=1
A=0
0.8
We assume that Q is discrete and can be expressed in a product form along an ordering of variables.
Also, we assume that Q can be specified in polynomial space. Namely each component of Q involves only polynomial number of variables.
Once, Q is expressed in product form, we can express it using a probability tree as shown. The probability tree here is over three variables with a variable at each level. Each branch in the tree is labeled with a probability of reaching that branch given the parent. To sample from this probability tree, one just selects each branch with the appropriate probability until a leaf node is reached. The probability of reaching the leaf node is the product of the probabilities on the path from the root to that leaf. For example, the probability of sampling the leaf C=1 is 0.8 times 0.6 times 0.8.
0.2
0.4
0.6
B=0
B=1
B=0
B=1
0.8
0.8
0.2
0.2
0.8
0.2
0.8
0.2
C=0
C=1
C=0
C=1
C=0
C=1
C=0
C=1

16. Rejection Problem Toss a biased coin P(H)=0.8, P(T)=0.2 Say we get H
Root
0.8
0.2
A=1
A=0
0.8
0.2
0.4
0.6
B=0
B=1
B=0
B=1
0.8
0.8
0.2
0.2
0.8
0.2
0.8
0.2
C=0
C=1
C=0
C=1
C=0
C=1
C=0
C=1

17. Rejection Problem Root Toss a biased coin P(H)=0.4, P(T)=0.6
Say, We get a Head
0.8
A=0
0.4
0.6
B=0
B=1
0.8
0.2
0.2
0.8
C=0
C=1
C=0
C=1

18. Rejection Problem Root Toss a biased coin P(H)=0.4, P(T)=0.6
Say, We get a Head
0.8
A=0
0.4
0.6
B=0
B=1
0.8
0.2
0.2
0.8
C=0
C=1
C=0
C=1

19. Rejection Problem Root Toss a biased coin P(H)=0.4, P(T)=0.6
Say, We get a Head
0.8
A=0
0.4
B=0
0.8
0.2
C=0
C=1
A large number of assignments generated will be rejected, thrown away

20. Rejection Problem Constraints: A≠B A≠C
Root
0.8
0.2
0.4
0.6
Unfortunately, in presence of constraints or determinism, the proposal distribution may suffer from the rejection problem in that it may generate samples which are inconsistent i.e. they are not solutions to the constraint portion.
In the tree, all blue leaves are solutions while all red leaves are non-solutions and as we can see there is a positive probability of generating a non-solution.
To understand why this is a problem, let us look at the expression shown above. f is the true function while Q is the proposal distribution. If a sample xi is not a solution its f-value is zero and the ratio will be zero. We call such samples whose weight is zero as rejected samples. If we only generate rejected sample our estimate M-hat will be zero.
All Blue leaves correspond to solutions i.e. f(x) >0
All Red leaves correspond to non-solutions i.e. f(x)=0

21. Revising Q to backtrack-free distribution:
QF(branch)=0 if no solutions under it
QF(branch)=1 if no solutions under the other branch
QF(branch) =Q(branch) otherwise
Constraints: A≠B A≠C
A=0
B=0
C=0
B=1
A=1
C=1
Root
0.8
0.2
0.40
0.61 1
0.21
0.80
To reemedy this situation we can try to modify Q so that it will be easy to sample from and it will have the same set of solutions (same non-zero tuples) as f, the true function
It is easy to see that if we modify Q so that it respects all constraints in the graphical model, we would never generate inconsistent samples or non-solutions. In the constraint satisfaction jargon, this process of generating a solution without backtracking is called as making the constraint network backtrack-free. So we call our modification of the proposal which always generates samples which are solutions as the backtrack-free distribution.
It is defined as shown. At each branch, we check to see if there are solutions under that branch, if there are then we keep the branch as it is.
All Blue leaves correspond to solutions i.e. f(x) >0
All Red leaves correspond to non-solutions i.e. f(x)=0

22. Generating samples from QF
QF(branch)=0 if no solutions under it
QF(branch)=1 if no solutions under the other branch
QF(branch) =Q(branch) otherwise
Constraints: A≠B A≠C
Root
0.8
0.2
A=0
00.4
0.6  1
?? Solution
Invoke an oracle at each branch.
Oracle returns True if there is a solution under a branch
False, otherwise
?? Solution
B=1
00.2
0.81
?? Solution
C=1

23. Generating samples from QF
Gogate et al., UAI 2005, Gogate and Dechter, UAI 2005
Generating samples from QF
Constraints: A≠B A≠C
Oracles: In practice
Adaptive consistency as pre-processing step
A complete CSP solver
Too costly
O(exp(treewidth))
Invoked O(nd) for each sample
Root
0.2
0.8
A=0 1
B=1 1
C=1

24. Gogate et al., UAI 2005, Gogate and Dechter, UAI 2005
Approximations of QF
Use i-consistency instead of adaptive consistency
O(ni) time and space complexity
identify some zeros so that they are never sampled
Cons: Too weak when constraint portion is hard.
No processing
Percentage of Zeros identified
Polynomial consistency enforcement
Rejection Rate
NP-hard exponential time and space

25. Algorithm SampleSearch
Gogate and Dechter, AISTATS 2007, AAAI 2007
Algorithm SampleSearch
Constraints: A≠B A≠C
A=0
B=0
C=0
B=1
A=1
C=1
Root
0.8
0.2
0.4
0.6

26. Algorithm SampleSearch
Gogate and Dechter, AISTATS 2007, AAAI 2007
Algorithm SampleSearch
Constraints: A≠B A≠C
Root
0.8
0.2
A=1
A=0
0.8
0.2
0.4
0.6 1
B=0
B=1
B=0
B=1
0.8
0.8
0.2
0.2
0.8
0.2
0.8
0.2
C=0
C=1
C=0
C=1
C=0
C=1
C=0
C=1

27. Algorithm SampleSearch
Gogate and Dechter, AISTATS 2007, AAAI 2007
Algorithm SampleSearch
Constraints: A≠B A≠C
Root
0.8
0.2
A=1
A=0
0.8
0.2 1
B=1
B=0
B=1
0.8
0.2
0.8
0.2
0.8
0.2
C=0
C=1
C=0
C=1
C=0
C=1
Resume Sampling

28. Algorithm SampleSearch
Gogate and Dechter, AISTATS 2007, AAAI 2007
Algorithm SampleSearch
Constraints: A≠B A≠C
Root
0.8
0.2
A=1
A=0
0.8
0.2 1
B=1
B=0
B=1
0.8
0.2
0.8 1
0.2
0.8
0.2
C=0
C=1
C=0
C=1
C=0
C=1
Until f(sample)>0
Constraint Violated

29. Gogate and Dechter, AISTATS 2007, AAAI 2007
Generate more Samples
Constraints: A≠B A≠C
Root
0.8
0.2
A=1
A=0
0.8
0.2
0.4
0.6
B=0
B=1
B=0
B=1
0.8
0.8
0.2
0.2
0.8
0.2
0.8
0.2
C=0
C=1
C=0
C=1
C=0
C=1
C=0
C=1

30. Gogate and Dechter, AISTATS 2007, AAAI 2007
Generate more Samples
Constraints: A≠B A≠C
Root
0.8
0.2
A=1
A=0
0.8
0.2
0.4
0.6
B=0
B=1
B=0
B=1
0.8 1
0.8
0.2
0.2
0.8
0.2
0.8
0.2
C=0
C=1
C=0
C=1
C=0
C=1
C=0
C=1

31. Traces of SampleSearch
Gogate and Dechter, AISTATS 2007, AAAI 2007
Traces of SampleSearch
Constraints: A≠B A≠C
A=0
B=0
B=1
C=1
Root
A=0
B=1
C=1
Root
C=0
A=0
B=0
B=1
C=1
Root
C=0
Root
A=0
B=1
C=1

32. SampleSearch: Sampling Distribution
Gogate and Dechter, AISTATS 2007, AAAI 2007
SampleSearch: Sampling Distribution
Problem: Due to Search, the samples are no longer i.i.d. from Q
Theorem: SampleSearch generates i.i.d. samples from the backtrack-free distribution
Now that we know how to always generate consistent samples, we are not done yet because our task is to estimate the probability of evidence or the normalization constant M?
If we use Q to estimate M, then it would be incorrect because SampleSearch does not generate samples from Q due to its backtracking search component.
Ironically, we can prove that SampleSearch generates samples from the backtrack-free distribution and as we already saw it is quite expensive to compute.

33. The Sampling distribution QF of SampleSearch
Gogate and Dechter, AISTATS 2007, AAAI 2007
The Sampling distribution QF of SampleSearch
Constraints: A≠B A≠C
Root
What is probability of generating A=0?
QF(A=0)=0.8
Why? SampleSearch is systematic
0.8
What is probability of generating (A=0,B=1)?
QF(B=1|A=0)=1
Why? SampleSearch is systematic
A=0 1
Skip this slide.
B=0
B=1
What is probability of generating (A=0,B=0)?
Simple: QF(B=0|A=0)=0
All samples generated by SampleSearch are solutions 1
C=0
C=1
C=0
C=1
Backtrack-free distribution

34. Asymptotic approximations of QF
Gogate and Dechter, AISTATS 2007, AAAI 2007
Asymptotic approximations of QF
Root
IF Hole THEN
UF=Q (i.e. there is a solution at the other branch)
LF=0 (i.e. no solution at the other branch)
0.8
Hole ?
Don’t know
A=0 1
No solutions here
B=0
B=1
QF(branch)=0 if no solutions under it
QF(branch) =Q(branch) otherwise
Let us assume that SampleSearch traversed the trace shown in the Figure to sample the Blue solution. To know the exact Backtrack-free distribution at each branch point, we have to figure out whether there is a solution on the other branch point.
If there is no solution on the other branch point according to the definition of the backtrack-free distribution, we set the probability of the branch to zero. Otherwise, we keep it the same.
In its trace, SampleSearch has figured out that some branches are solutions or non solutions but it has not figured out the consistency of some other branches which are the holes or question marks in the figure.
A simple approximation suggests itself. We can either set the hole to Q or to 0 corresponding to the definition of the backtrack-free distribution. 1
No solutions here
C=0
C=1
C=0
C=1

35. Approximations: Convergence in the limit
Gogate and Dechter, AISTATS 2007, AAAI 2007
Approximations: Convergence in the limit
Store all possible traces
A=0
B=1
C=1
Root
0.8 1 ?
A=0
B=0
B=1
C=1
Root
0.8 ? 1
A=0
B=1
C=1
Root
C=0
0.8 ?
0.6 1

36. Approximations: Convergence in the limit
Gogate and Dechter, AISTATS 2007, AAAI 2007
Approximations: Convergence in the limit
From the combined sample tree, update U and L.
IF Hole THEN UFN=Q and LFN=0
Root
0.8 ?
A=0 1
B=1 1
C=1

37. Improving Naive SampleSeach: The IJGP-wc-SS algorithm
Gogate and Dechter, AISTATS 2007, AAAI 2007
Improving Naive SampleSeach: The IJGP-wc-SS algorithm
Better Search Strategy
Can use any state-of-the-art CSP/SAT solver e.g. minisat (Een and Sorrenson 2006)
Better Proposal distribution
Use output of IJGP- a generalized belief propagation to compute the initial importance function
w-cutset importance sampling (Bidyuk and Dechter, 2007)
Reduce variance by sampling from a subspace

38. Experiments Tasks Benchmarks Algorithms Weighted Counting Marginals
Satisfiability problems (counting solutions)
Linkage networks
Relational instances (First order probabilistic networks)
Grid networks
Logistics planning instances
Algorithms
IJGP-wc-SS/LB and IJGP-wc-SS/UB
IJGP-wc-IS (Vanilla algorithm that does not perform search)
SampleCount (Gomes et al. 2007, SAT)
ApproxCount (Wei and Selman, 2007, SAT)
EPIS (Changhe and Druzdzel, 2006)
RELSAT (Bayardo and Peshoueshk, 2000, SAT)
Edge Deletion Belief Propagation (Choi and Darwiche, 2006)
Iterative Join Graph Propagation (Dechter et al., 2002)
Variable Elimination and Conditioning (VEC)

39. Results: Probability of Evidence Linkage instances (UAI 2006 evaluation)
Time Bound: 3 hrs
M: number of samples generated in 10 hrs
Z: Probability of Evidence

40. Results: Probability of Evidence Relational instances (UAI 2008 evaluation)
Time Bound: 10 hrs
M: number of samples generated in 10 hrs
Z: Probability of Evidence

42. Results: Solution Counts Latin Square instances (size 8 to 16)
Here, I’ll explain the methodology that each algorithm was run for the same amount of time (10hrs).
Also explain what n, k, c and w stand for.
If exact results are not known, we report the lower bound. Higher the lower bound, better the particular scheme.
Also, ApproxCount does not output a lower bound.
Time Bound: 10 hrs
M: number of samples generated in 10 hrs
Z: Solution Counts

43. Results: Solution Counts Langford instances
Time Bound: 10 hrs
M: number of samples generated in 10 hrs
Z: Solution Counts

45. Results on Marginals Evaluation Criteria
Always bounded between 0 and 1
Lower Bounds the KL distance
When probabilities close to zero are present KL distance may tend to infinity.

46. Results: Posterior Marginals Linkage instances (UAI 2006 evaluation)
Time Bound: 3 hrs
Table shows the Hellinger distance (∆) and Number of samples: M

48. Summary: SampleSearch
Manages rejection problem while sampling
Sampling Distribution is the backtrack-free distribution QF
Approximation of QF by storing all traces yielding an asymptotically unbiased estimator
Linear time and space overhead
Bound the weighted counts from above and below
Empirically, when a substantial number of zero probabilities are present, SampleSearch dominate.

49. AND/OR Importance sampling
Using structure within importance sampling
Our final contribution is AND/OR Importance sampling which utilizes structure within importance sampling.
AND/OR Importance sampling

50. Overview Introduction: Mixed graphical models
SampleSearch: Sampling with Searching
Exploiting structure in sampling: AND/OR Importance sampling
Here is a brief outline of my talk.
I’ll start by describing some preliminaries and motivating applications.
Then I’ll describe the main problem which we investigated in this thesis which is that of poor performance of sampling based algorithms in presence of determinism.
Then I’ll describe the four main contributions of this thesis.

51. Motivation
Given Q(z), Importance sampling totally disregards the structure of f(z) while approximating it.
Consider the two sums. Both are sums over four variables {1,2,3,4} but both have different structure. The second sum is sparser than the first sum.
Given a proposal distribution Q(z), Importance sampling totally disregards the structure of f(z) while approximating M.
Intuitively, we can expect that the second sum which has a much sparser graph structure is easier to approximate than the first sum and we will build on this intuition to generalize importance sampling.

52. OR Search Tree Constraint Satisfaction – Counting Solutions A B C D E
RABC 1 B C D
RBCD 1 A B E
RABE 1 A E F
RAEF 1 A E C B F D C D F E B A 1 1 1
Defined by a variable ordering,
Fixed but may be dynamic 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

53. AND/OR Search Tree Constraint Satisfaction – Counting Solutions A=aB C
RABC 1 B C D
RBCD 1 A B E
RABE 1 A E F
RAEF 1 A E C B F D A
A=a B A E C B F D
DFS tree C A F
B=b C E
C=c
E=e D B E D F
D=d
F=f

54. AND/OR Search Tree Constraint Satisfaction – Counting SolutionsB C
RABC 1 B C D
RBCD 1 A B E
RABE 1 A E F
RAEF 1 A D B E C F A E C B F D
pseudo tree OR A
AND 1 OR B B
Defined by a variable ordering,
Fixed but may be dynamic
AND 1 1 OR C E C E C E C E
AND 1 1 1 1 1 1 1 1 OR D D F F D D F F D D F F D D F F
AND 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

55. AND/OR Tree OR – casing upon variable valuesF
OR – casing upon variable values
AND – decomposition into independent subproblems
pseudo tree OR A
AND 1 OR B B
Defined by a variable ordering,
Fixed but may be dynamic
AND 1 1 OR C E C E C E C E
AND 1 1 1 1 1 1 1 1 OR D D F D F F D D F D F
AND 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

56. Complexity of AND/OR Tree Search
Space
O(n)
Time
O(n km)
O(n kw* log n)
[Freuder & Quinn85], [Collin, Dechter & Katz91],
[Bayardo & Miranker95], [Darwiche01]
O(kn)
k = domain size
m = depth of pseudo-tree
n = number of variables
w*= treewidth

57. Background: AND/OR search spaceZ 1 2 X Y OR
AND
AND/OR Tree X
{0,1,2} Z X Y
Pseudo-tree ≠ Z
{0,1,2} ≠ Y
{0,1,2}
Problem X 1 2 Z Y
OR Tree X Z Y
Chain
Pseudo-tree
To take advantage of the structure of the graphical model within importance sampling, we will use the notion of AND/OR search spaces for graphical models.
Given a graphical model, the AND/OR search space is driven by a pseudo tree which captures the problem decomposition. For example, given Z, X and Y are independent of each other. Given the pseudo-tree, the associated AND/OR search tree, has alternating levels of OR nodes and AND nodes. The OR nodes are circles and are labeled with variables while the AND nodes are squares and labeled with values and correspond to value assignments in the domains of the variables. The root of the AND/OR search tree is an OR node, labeled with the root of the pseudo-tree T.
Semantically, the OR states represent alternative solutions, whereas the AND states represent problem decomposition into independent sub-problems, all of which need be solved. When the pseudo-tree is a chain, it does not take advantage of decomposition and the AND/OR search tree coincides with the regular OR search tree.

58. Example Bayesian network
P(Z)
Z=0
Z=1
0.8
0.2
f(Z) Y B Z X A
P(X|Z)
X=0
X=1
X=2
Z=0
0.3
0.4
Z=1
0.2
0.7
0.1
P(Y|Z)
Y=0
Y=1
Y=2
Z=0
0.5
0.1
0.4
Z=1
0.2
0.6
f(XZ)
f(YZ)
P(X)
X=0
0.1
X=1
0.2
X=2
0.6
P(Y)
Y=0
0.2
Y=1
0.7
Y=2
0.1
We will use the given Bayesian network to explain the main idea in AND/OR importance sampling. Here we are interested in computing probability of evidence or weighted counting.
The evidence here is A=0, B=0. When we instantiate the variable A to 0, the CPT on A is a function over X. Similarly, when we instantiate B, the CPT on B is a function over Y.
For simplicity and to save space on the slides, we will combine the CPTs mentioning X into a single function f(XZ) and the CPTs involving Y into a single function f(YZ).
In this case the probability of evidence P(a,b) can be computed by summing out X,Y,Z from the product of the three functions.
Evidence A=0, B=0

59. Recap: Conventional Importance Sampling
Gogate and Dechter, UAI 2008, CP2008
Recap: Conventional Importance Sampling Y B Z X A
AND/OR idea!
Decompose this expectation
Let us quickly recap how conventional importance sampling converts the summation into an expectation.
Given a proposal distribution Q, we multiply and divide by Q yielding the given expectation.
Now the main idea in AND/OR importance sampling is to decompose the given expectation based on the structure of the Bayesian network.

60. AND/OR Importance Sampling (General Idea)Y B Z X A
AND/OR Importance Sampling (General Idea) Z X Y
Pseudo-tree
Decompose Expectation
Consider the expression for M.
By applying simple symbolic manipulations, we can rewrite the expression as shown.
The terms in brackets are nothing but conditional expectations. First bracket gives conditional expectation of X given Z.
Second bracket gives conditional expectation of Y given Z.
The conditional expectations can also be read by following the pseudo-tree structure.

61. AND/OR Importance Sampling (General Idea)Y B Z X A
AND/OR Importance Sampling (General Idea) Z X Y
Pseudo-tree
Estimate all conditional expectations separately
How?
Record all samples
For each sample that has Z=j
Estimate the conditional expectations X|Z and Y|Z using samples corresponding to X|Z=j and Y|Z=j respectively.
Combine the results
So how to use this decomposition for sampling.
The idea is to compute the expectations separately.
How to do that
Well, First, we record all the samples.
Next, for each Z=j, we estimate the conditional expectation at X and at Y separately using the generated samples.
Combine the results.

62. AND/OR Importance Sampling
Gogate and Dechter, UAI 2008, CP2008 Y B Z X A
Sample # Z X Y 1 2 3 4 Z
<1.6,2>
<0.4,2> 1 X X Y Z X Y
Pseudo-tree Y
<0.12,1>
<0.2,1>
<0.16,1>
<0.36,1>
<0.08,1>
<0.14,1>
<0.28,1>
<0.84,1> 1 2 1 1 2 1
The operations can be intuitively expressed on an AND/OR search tree. Let us assume that we have generated four samples. We simply arrange the samples on an AND/OR tree. Note that I have labeled the arcs using some weird number pairs. Where do they come from.
The first quantity is simply the ratio of f over Q. The second quantity keeps track of how many times the assignment is sampled. Now coming back to the computation part.
At each OR node, we estimate conditional expectation of the variable at that node given the value of the variable ordered above it. Fancy word conditional expectation means average of the children.
For example, the conditional expectation of X given Z=0 is simply the average the quantities below it.
At each AND node, we combine the results, namely take the product based on the expression shown above.

63. Example Bayesian network
P(Z)
Z=0
Z=1
0.8
0.2 Y B Z X A
P(X|Z)
X=0
X=1
X=2
Z=0
0.3
0.4
Z=1
0.2
0.7
0.1
P(Y|Z)
Y=0
Y=1
Y=2
Z=0
0.5
0.1
0.4
Z=1
0.2
0.6
P(X)
X=0
0.1
X=1
0.2
X=2
0.6
P(Y)
Y=0
0.2
Y=1
0.7
Y=2
0.1
We will use the given Bayesian network to explain the main idea in AND/OR importance sampling. Here we are interested in computing probability of evidence or weighted counting.
The evidence here is A=0, B=0. When we instantiate the variable A to 0, the CPT on A is a function over X. Similarly, when we instantiate B, the CPT on B is a function over Y.
For simplicity and to save space on the slides, we will combine the CPTs mentioning X into a single function f(XZ) and the CPTs involving Y into a single function f(YZ).
In this case the probability of evidence P(a,b) can be computed by summing out X,Y,Z from the product of the three functions.
Evidence A=0, B=0

64. Gogate and Dechter, UAI 2008, CP2008
AND/OR Importance Sampling
Sample # Z X Y 1 2 3 4 Z
<1.6,2>
<0.4,2> 1 X X Y Y
<0.12,1>
<0.2,1>
<0.16,1>
<0.36,1>
<0.08,1>
<0.14,1>
<0.28,1>
<0.84,1> 1 2 1 1 2 1
Putting these ideas together, we can compute a new estimate for M as follows.
We arrange the samples on an AND/OR search tree.
At each AND node, we compute a product and at each OR node we compute a weighted average.
The value of the root node now expresses an estimate of the weighted counts.
All AND nodes: Separate Components. Take Product
Operator: Product
All OR nodes: Conditional Expectations given the assignment above it
Operator: Average

65. Algorithm AND/OR Importance Sampling
Gogate and Dechter, UAI 2008, CP2008
Algorithm AND/OR Importance Sampling
Generate samples x1, ,xN from Q along O.
Build a AND/OR sample tree for the samples x1, ,xN along the ordering O.
FOR all leaf nodes i of AND-OR tree do
IF AND-node v(i)= 1 ELSE v(i)=0
FOR every node n from leaves to the root do
IF AND-node v(n)=product of children
IF OR-node v(n) = Average of children
Return v(root-node)

66. # samples in AND/OR vs Conventional
Gogate and Dechter, UAI 2008, CP2008
# samples in AND/OR vs Conventional
Sample # Z X Y 1 2 3 4 Z
<1,1>
<1,1> 1 X Y X Y
<1,1>
<1,1>
<1,1>
<1,1>
<1,1>
<1,1>
<1,1>
<1,1> 1 2 1 1 2 1
8 Samples in AND/OR space versus 4 samples in importance sampling
Example: Z=0, X=2, Y=0 is not generated but still considered in AND/OR space
AND/OR sampling has lower error because it computes an average by considering more samples.
For example, Z=0, X=2, Y=0 is not generated but still considered in AND/OR space (see the red branches in the AND/OR tree)
Let us count the number of samples. Remember that OR node represents alternatives while AND node stands for decomposition or independent sub-problems each of which must be solved. So we need to sum over all alternatives or OR nodes and take a product over all AND nodes.

67. Properties of AND/OR Importance Sampling
Gogate and Dechter, UAI 2008, CP2008
Properties of AND/OR Importance Sampling
Unbiased estimate of weighted counts.
AND/OR estimate has lower Variance than conventional importance sampling estimate.
Variance Reduction
Easy to Prove for case of complete independence (Goodman, 1960)
Complicated to prove for general conditional independence case (Gogate thesis, papers!)

68. AND/OR w-cutset (Rao-Blackwellised) sampling
Gogate and Dechter, Under review, CP 2009
AND/OR w-cutset (Rao-Blackwellised) sampling
Rao-Blackwellisation (Rao, 1963)
Partition X into K and R, such that we can compute P(R|k) efficiently.
Sample from K and sum out R
Estimate:
w-cutset sampling (Bidyuk and Dechter, 2003): Select K such that the treewidth of R after removing K is bounded by “w”.
Weighted Counts conditioned on K=ki
We can even combine AND/OR sampling with Rao-Blackwellised w-cutset sampling yielding further variance reduction.
The idea in Rao-Blackwellisation is to combine exact inference with sampling which reduces variance based on the Rao-Blackwell theorem.
Here the set of variables is partitioned into K and R such that it is easy to do exact inference on R given K and the Rao-Blackwell estimate is shown below.
w-cutset sampling is an elegant implementation of Rao-Blackwellisation in which the set K is selected in such a way that the treewidth of R given K is bounded by w. Thus the exact inference step can be carried out in polynomial time yielding a very efficient practical scheme.

69. AND/OR w-cutset sampling
Gogate and Dechter, Under review, CP 2009
AND/OR w-cutset sampling
Perform AND/OR tree or graph sampling on K
Exact inference on R
Orthogonal approaches:
Theorem: Combining AND/OR sampling and w-cutset sampling yields further variance reduction. A B C D E G F A B C D E G F A A B C B
The main idea in AND/OR w-cutset sampling is to perform AND/OR sampling on the w-cutset variables K and exact inference on R.
The main point here is that AND/OR sampling is orthogonal to w-cutset Rao-Blackwellised sampling and the two could be combined to yield further variance reduction.
Start pseudo tree
on the cutset variables C
OR pseudo tree on the cutset variables
Graphical model
Full pseudo tree

70. From Search Trees to Search Graphs
Any two nodes that root identical subtrees (subgraphs) can be merged

71. From Search Trees to Search Graphs
Any two nodes that root identical subtrees (subgraphs) can be merged

72. Merging Based on Context
One way of recognizing nodes that can be merged:
context (X) = ancestors of X in pseudo tree that are connected to X, or to descendants of X A D B E C F
pseudo tree
[ ]
[A] A E C B F D A E C B F D
[AB]
[AB]
We may have two nodes that are identical, then we can merge.
Highlight: rooting the same tree.
Maybe an example of and/or tree, show animation that we merge.
Minimal is when we remove all redundancy
[BC]
[AE]

73. AND/OR Search Graph Constraint Satisfaction – Counting Solutions
Context A B C
RABC 1 B C D
RBCD 1 A B E
RABE 1 A E F
RAEF 1 A D B E C F
[ ]
[A] A E C B F D
[AB]
[AB]
[BC]
[AE] OR
pseudo tree A
AND 1 OR B B
AND 1 1
Defined by a variable ordering,
Fixed but may be dynamic OR C E C E C E C E
AND 1 1 1 1 1 1 1 1 OR D D D D F F F F B C
Value 1
AND 1 1 1 1 1 1 1 1
context minimal graph
Cache table for D

74. How Big Is the Context?
Theorem: The maximum context size for a pseudo tree is equal to the treewidth of the graph along the pseudo tree. C H K D M F G A B E J O L N P
[AB]
[AF]
[CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
(C K H A B E J L N O D P M F G)
max context size = treewidth

75. Complexity of AND/OR Graph Search
Space
O(n kw*)
O(n kpw*)
Time
k = domain size
n = number of variables
w*= treewidth
pw*= pathwidth
w* ≤ pw* ≤ w* log n

76. AND/OR Graphs ≠ C B D A {0,1,2} 2 B D 1 C A AND/OR tree 2 B D 1 C A C1 C A
AND/OR tree 2 B D 1 C A C B D A
AND/OR graph
The figure shows a full AND/OR tree corresponding to the pseudo tree. We can see that node A does not depend on C and therefore the subtree corresponding to C=0 and C=1 is identical for A.
Therefore, the two could be merged yielding an AND/OR graph which is even more compact and uses more conditional indenpendencies that an AND/OR tree.

77. AND/OR graph sampling ≠ C B D A {0,1,2} OR Sample Tree C B D AB 1 D A 2 C B D A
AND/OR Sample Tree C 1 B D 2 A
AND/OR Sample Graph C 1 D 2
We can implement the same idea and convert an AND/OR sample tree into an AND/OR sample graph by merging all identical subtrees. This will yield a new structure on which we can perform similar computations, namely average at OR and product at AND to yield a new estimate. This new estimate will have lower variance. B 1 A B A 2 2 A 1 2

78. Variance Hierarchy and Complexity
O(nN)
O(1)
Variance Hierarchy and Complexity IS
O(cN+(n-c)Nexp(w))
O(1)
O(nN)
O(h)
w-cutset IS
AND/OR Tree IS
O(nNw*)
O(nN)
O(cN+(n-c)Nexp(w))
O(h+(n-c)exp(w))
AND/OR Graph IS
AND/OR w-cutset Tree IS
O(cNw*+(n-c)Nexp(w))
O(cN+(n-c)exp(w))
AND/OR w-cutset Graph IS

79. Experiments Benchmarks Algorithms Linkage analysis Graph coloring
Grids
Algorithms
OR tree sampling
AND/OR tree sampling
AND/OR graph sampling
w-cutset versions of the three schemes above

80. Results: Probability of Evidence Linkage instances (UAI 2006 evaluation)
Time Bound: 1hr

81. Here we see that the scheme that employs the most decomposition AND/OR w-cutset graph sampling yields the best performance.

82. Summary: AND/OR Importance sampling
AND/OR sampling: A general scheme to exploit conditional independence in sampling
Theoretical guarantees: lower sampling error than conventional sampling
Variance reduction orthogonal to Rao-Blackwellised sampling.
Better empirical performance than conventional sampling.

83. Conclusion
Effective sampling in presence of determinism to address the rejection and non-convergence problems
SampleSearch
manages rejection while sampling
SampleSearch-SIR and SampleSearch-MH
Convergent MCMC sampling schemes
Lower Bounding schemes
Extending Markov inequality to multiple samples
AND/OR Importance sampling
Post-processing, use conditional independence for reducing the variance of estimates

84. Mixed Networks (Mateescu and Dechter, 2004)
Constraint Network
Belief Network A A F F B C B C
Moral mixed graph E D A E D B C D 1 F B C
D=0
D=1 .2 .8 1 .1 .9 .3 .7 .5 B C
We often have both probabilistic information and constraints. Our approach is to allow the two representation to co-exist,
explicitlly. It has virtues for user interface and for computation.
Complex cnf queries:
P((A or B) and (~CVD)) E D 84
UNL, April 2009 84

85. Transportation Planning: Graphical modeldt wt
dt-1
wt-1
D: Time-of-day (discrete)
W: Day of week (discrete)
G: collection of locations where the person spends significant amount of time. (discrete)
F: Counter
Route: A hidden variable that just predicts what path the person takes (discrete)
Location: A pair (e,d) e is the edge on which the person is and d is the distance of the person from one of the end-points of the edge (continuous)
Velocity: Continuous
GPS reading: (lat,lon,spd,utc).
gt-1 gt
Ft-1 Ft
rt-1 rt
vt-1 vt
Here is an example dynamic model which models car travel activity routines of individuals
lt-1 lt
yt-1 yt