1. Adaptive Importance Sampling for Estimation in Structured Domains L.E. Ortiz and L.P. Kaelbling

2. 2 Contents t Notations t Importance Sampling t Adaptive Importance Sampling t Empirical Results

3. 3 Notations t Bayesian network (BN) and influence diagram (ID) (A: decision node, U: utitity node)

4. 4 t Probabilities of interest (O: variables of interest, Z: remaining ones) t Best strategy: The strategy with the highest expected utility. The action ‘a’ maximizing the value associated with the evidence ‘o’ (i.e. the parents of ‘a’). t Importance sampling is needed to calculate the above summations

5. 5 Importance Sampling t quantity of interest: t Z ~ important sampling distribution f(z): t estimation of G : (sampling of w from f) t Cf. Estimation of

6. 6 t BN: likelihood weighting (prior) (likelihood) t ID:

7. 7 t Eg. t G can be calculated by sampling of w’s. t Cf.

8. 8 t Variance of the weights: t Minimum variance importance sampling distributions: (taking a derivitive from above) t The weights have 0 variance in this case(w=G) t f (z) must have “ Fat Tail ”: as for at least one value of Z.

9. 9 Adaptive Importance Sampling t Parameterizing the importance sampling distribution (tabularizing) t Update rules based on gradient descent

10. 10 t Three different forms of gradient  minimize variance directly  minimize distance between the current sampling distribution and approximate optimal sampling distribution  minimize distance between the current sampling distribution and empirical optimal distribution

11. 11 t Minimizing variance: t via approximate optimal distribution:

12. 12 t via parameterized empirical distribution: (, if RHS=0)

13. 13 Remarks t  ’s are proportional to square, linear, logarithmic of the weights. t  L2 is positive if w/G > 1 (under estimation of g) t The size and sign of  are related to under or over estimation of g.

14. 14

15. 15 Empirical Results t Problem: Calculate V MP(t) (A) for A=2, MP(t)=1 in the computer mouse problem. t Evaluation: by MSE between the true value and the estimation from sampling method. t Var and L2 are better than LW(traditional method) t L2 is more stable than other methods

16. 16