1. Simple Sampling Sampling Methods Inference Probabilistic Graphical
Models
Sampling Methods
Simple Sampling

2. Sampling-Based Estimation
sampled IID from P
If P(X=1) = p
Estimator for p:
More generally, for any distribution P, function f:

3. Sampling from Discrete Distribution
Val(X) = {x1, …, xk}
P(xi) = i 1

4. Sampling-Based Estimation
Hoeffding Bound:
Chernoff Bound:

5. Sampling-Based Estimation
Hoeffding Bound:
For additive bound  on error with probability > 1-:
Chernoff Bound:
For multiplicative bound  on error with probability > 1-:

6. Forward sampling from a BN
0.6
0.4 i0 i1
0.7
0.3
Difficulty
Intelligence g1 g2 g3
i0,d0
0.3
0.4
i0,d1
0.05
0.25
0.7
i1,d0
0.9
0.08
0.02
i1,d1
0.5
0.2
Grade
SAT s0 s1 i0
0.95
0.05 i1
0.2
0.8
Letter l0 l1 g1
0.1
0.9 g2
0.4
0.6 g3
0.99
0.01

7. Forward Sampling for Querying
Goal: Estimate P(Y=y)
Generate samples from BN
Compute fraction where Y=y
For additive bound  on error with probability > 1-:
For multiplicative bound  on error with probability > 1-:

8. Queries with Evidence Goal: Estimate P(Y=y | E=e)
Rejection sampling algorithm
Generate samples from BN
Throw away all those where Ee
Compute fraction where Y=y
Expected fraction of samples kept ~ P(e)
# samples needed rows exponentially
with # of observed variables

9. Summary Generating samples from a BN is easy
(,)-bounds exist, but usefulness is limited:
Additive bounds: useless for low probability events
Multiplicative bounds: # samples grows as 1/P(y)
With evidence, # of required samples grows exponentially with # of observed variables
Forward sampling generally infeasible for MNs

10. END END END