Simple Sampling Sampling Methods Inference Probabilistic Graphical Models Sampling Methods Simple Sampling

Sampling-Based Estimation sampled IID from P If P(X=1) = p Estimator for p: More generally, for any distribution P, function f:

Sampling from Discrete Distribution Val(X) = {x1, …, xk} P(xi) = i 1

Sampling-Based Estimation Hoeffding Bound: Chernoff Bound:

Sampling-Based Estimation Hoeffding Bound: For additive bound  on error with probability > 1-: Chernoff Bound: For multiplicative bound  on error with probability > 1-:

Forward sampling from a BN 0.6 0.4 i0 i1 0.7 0.3 Difficulty Intelligence g1 g2 g3 i0,d0 0.3 0.4 i0,d1 0.05 0.25 0.7 i1,d0 0.9 0.08 0.02 i1,d1 0.5 0.2 Grade SAT s0 s1 i0 0.95 0.05 i1 0.2 0.8 Letter l0 l1 g1 0.1 0.9 g2 0.4 0.6 g3 0.99 0.01

Forward Sampling for Querying Goal: Estimate P(Y=y) Generate samples from BN Compute fraction where Y=y For additive bound  on error with probability > 1-: For multiplicative bound  on error with probability > 1-:

Queries with Evidence Goal: Estimate P(Y=y | E=e) Rejection sampling algorithm Generate samples from BN Throw away all those where Ee Compute fraction where Y=y Expected fraction of samples kept ~ P(e) # samples needed rows exponentially with # of observed variables

Summary Generating samples from a BN is easy (,)-bounds exist, but usefulness is limited: Additive bounds: useless for low probability events Multiplicative bounds: # samples grows as 1/P(y) With evidence, # of required samples grows exponentially with # of observed variables Forward sampling generally infeasible for MNs

END END END