1. Generative Models
for probabilistic inference
Michael Stewart

2. Remember the Joint Distribution?

3. What about very large/complex models?

4. "Generative" Modeling
Implement probability theory in computer science to infer a joint distribution
     Bayesian prior -> posterior provides learning opportunity
     Sampling methods are their own field of study
Applications in neuroscience, machine learning, biology [1]
Requires a model and sampling method...

5. "Generate" examples

6. Define a Model This is a broad procedure; what is the goal? Bio:
observe fMRI data, infer latent locality function in brain
observe genome data, infer latent gene relationships
AI:
observe words, infer latent topics or semantic information

7. How to sample
Rejection sampling: no Usually a kind of Gibbs/MCMC

8. Some implementations:
[2] BLOG
[3] Church
[4] Python modules provided by Tom Haines
     

9. Recommended reading  
A. Daud, J. Li, L. Zhou, and F. Muhammad, "Knowledge discovery through directed probabilistic topic models: a survey," Frontiers of Computer Science in China, vol. 4, no. 2, pp , Jun [Online]. Available:
M. Steyvers and T. Griffiths, Probabilistic Topic Models. Lawrence Erlbaum Associates, [Online]. Available:
N. Goodman, J. Tenenbaum, T. O'Donnell, and the Church Working Group. Probabilistic Models of Cognition 
Coin examples: Church Learning as Conditional Inference

10. References
[1] A. Venkataraman, Y. Rathi, M. Kubicki, C.-F. Westin, and P. Golland, "Joint generative model for fMRI/DWI and its application to population studies." Medical Image Computing and Computer-Assisted Intervention, vol. 13, no. Pt 1, pp , [Online]. Available:
[2] B. Milch, B. Marthi, S. Russell, D. Sontag, D. L. Ong, and A. Kolobov, "Blog: Probabilistic models with unknown objects," in In IJCAI, 2005, pp [Online]. Available:
[3] N. D. Goodman, V. K. Mansinghka, D. Roy, K. Bonawitz, and J. B. Tenenbaum, "Church: a language for generative models," in Uncertainty in Artificial Intelligence, 2008.
[Online]. Available:
[4]
plate notation example: M. Steyvers and T. Griffiths, Probabilistic Topic Models. Lawrence Erlbaum Associates, [Online]. Available: