Generative Models for probabilistic inference Michael Stewart.

Slides:

Advertisements

Similar presentations

Week 11 Review: Statistical Model A statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution.

Advertisements

Probabilistic Models in Human and Machine Intelligence.

Title: The Author-Topic Model for Authors and Documents

HOW TO MAKE A MACHINE THINK? CONTACT DETAILS Our team constists of 7 undergraduate students fascinated by consciousness and AI. We are aiming to simulate.

Bart Selman CS CS 475: Uncertainty and Multi-Agent Systems Prof. Bart Selman Introduction.

Probabilistic Models of Cognition Conceptual Foundations Chater, Tenenbaum, & Yuille TICS, 10(7), (2006)

Probabilistic inference in human semantic memory Mark Steyvers, Tomas L. Griffiths, and Simon Dennis 소프트컴퓨팅연구실오근현 TRENDS in Cognitive Sciences vol. 10,

Causes and coincidences Tom Griffiths Cognitive and Linguistic Sciences Brown University.

Introduction  Bayesian methods are becoming very important in the cognitive sciences  Bayesian statistics is a framework for doing inference, in a principled.

Exploring subjective probability distributions using Bayesian statistics Tom Griffiths Department of Psychology Cognitive Science Program University of.

LARGE SAMPLE TESTS ON PROPORTIONS

CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference (Sec. )

© 2002 Franz J. Kurfess Introduction 1 CPE/CSC 481: Knowledge-Based Systems Dr. Franz J. Kurfess Computer Science Department Cal Poly.

CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference.

A probabilistic approach to semantic representation Paper by Thomas L. Griffiths and Mark Steyvers.

Intelligent Systems Group Emmanuel Fernandez Larry Mazlack Ali Minai (coordinator) Carla Purdy William Wee.

Baysian Approaches Kun Guo, PhD Reader in Cognitive Neuroscience School of Psychology University of Lincoln Quantitative Methods 2011.

By Neng-Fa Zhou1 PRISM: A Probabilistic Language for Modeling and Learning Joint work with Taisuke Sato (Tokyo Institute of Technology)

CPSC 322 Introduction to Artificial Intelligence September 10, 2004.

CS 561, Sessions 28 1 Uncertainty Probability Syntax Semantics Inference rules.

1Ort ML A Figures and References to Topic Models, with Applications to Document Classification Wolfgang Maass Institut für Grundlagen der Informationsverarbeitung.

CSE 515 Statistical Methods in Computer Science Instructor: Pedro Domingos.

Vedrana Vidulin Jožef Stefan Institute, Ljubljana, Slovenia

NA-MIC National Alliance for Medical Image Computing Algorithms MIT PI: Polina Golland.

Learning causal theories Josh Tenenbaum MIT Department of Brain and Cognitive Sciences Computer Science and AI Lab (CSAIL)

Chapter 4 by Kathleen Mastrian and Dee McGonigle

Relational Probability Models Brian Milch MIT 9.66 November 27, 2007.

Sampletalk Technology Presentation Andrew Gleibman

IJCAI 2003 Workshop on Learning Statistical Models from Relational Data First-Order Probabilistic Models for Information Extraction Advisor: Hsin-His Chen.

第十讲概率图模型导论 Chapter 10 Introduction to Probabilistic Graphical Models

AP STATISTICS LESSON COMPARING TWO PROPORTIONS.

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 25 Wednesday, 20 October.

Bayesian Networks for Data Mining David Heckerman Microsoft Research (Data Mining and Knowledge Discovery 1, (1997))

MIT Computer Science and Artificial Intelligence Laboratory

Latent Dirichlet Allocation D. Blei, A. Ng, and M. Jordan. Journal of Machine Learning Research, 3: , January Jonathan Huang

Probabilistic Models for Discovering E-Communities Ding Zhou, Eren Manavoglu, Jia Li, C. Lee Giles, Hongyuan Zha The Pennsylvania State University WWW.

Computational Modeling of Anatomical and Functional Variability in Populations Polina Golland Computer Science and Artificial Intelligence Laboratory Massachusetts.

BLOG: Probabilistic Models with Unknown Objects Brian Milch, Bhaskara Marthi, Stuart Russell, David Sontag, Daniel L. Ong, Andrey Kolobov University of.

Probabilistic Models in Human and Machine Intelligence.

Cognitive Psychology What is cognitive psychology?

Vedrana Vidulin Jožef Stefan Institute, Ljubljana, Slovenia

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 24 of 41 Monday, 18 October.

Observation vs. Inferences The Local Environment.

COMPUTER SYSTEM FUNDAMENTAL Genetic Computer School INTRODUCTION TO ARTIFICIAL INTELLIGENCE LESSON 11.

Outline Historical note about Bayes’ rule Bayesian updating for probability density functions –Salary offer estimate Coin trials example Reading material:

Pedro Domingos University of Washington. Evolution Experience Culture.

Introduction to Artificial Intelligence Heshaam Faili University of Tehran.

Introduction to Artificial Intelligence Prof. Kathleen McKeown 722 CEPSR Tas: Andrew Rosenberg Speech Lab, 7 th Floor CEPSR Sowmya Vishwanath TA Room.

Integrative Genomics I BME 230. Probabilistic Networks Incorporate uncertainty explicitly Capture sparseness of wiring Incorporate multiple kinds of data.

Bayesian Estimation and Confidence Intervals Lecture XXII.

Learning Bayesian Networks for Complex Relational Data

Brief Intro to Machine Learning CS539

CS440/ECE448: Artificial Intelligence Lecture 1: What is AI?

A probabilistic approach to cognition

Eick: Introduction Machine Learning

School of Computer Science & Engineering

Artificial Intelligence and Lisp Lecture 13 Additional Topics in Artificial Intelligence LiU Course TDDC65 Autumn Semester,

Topic and Role Discovery In Social Networks

What is Pattern Recognition?

INFORMATION SOCIETY E-SOCIETY and INTELLIGENCE

Basic Intro Tutorial on Machine Learning and Data Mining

CSE 515 Statistical Methods in Computer Science

Probabilistic Models in Human and Machine Intelligence

CSCI 5822 Probabilistic Models of Human and Machine Learning

CS621/CS449 Artificial Intelligence Lecture Notes

The causal matrix: Learning the background knowledge that makes causal learning possible Josh Tenenbaum MIT Department of Brain and Cognitive Sciences.

Generative Models for probabilistic inference Michael Stewart.

Christoph F. Eick: A Gentle Introduction to Machine Learning

CS639: Data Management for Data Science

Machine learning CS 229 / stats 229

Presentation transcript:

Generative Models for probabilistic inference Michael Stewart

Remember the Joint Distribution?

What about very large/complex models?

"Generative" Modeling Implement probability theory in computer science to infer a joint distribution Bayesian prior -> posterior provides learning opportunity Sampling methods are their own field of study Applications in neuroscience, machine learning, biology [1] Requires a model and sampling method...

"Generate" examples

Define a Model This is a broad procedure; what is the goal? Bio: observe fMRI data, infer latent locality function in brain observe genome data, infer latent gene relationships AI: observe words, infer latent topics or semantic information

How to sample Rejection sampling: no Usually a kind of Gibbs/MCMC

Some implementations: [2] BLOG [3] Church [4] Python modules provided by Tom Haines

Recommended reading A. Daud, J. Li, L. Zhou, and F. Muhammad, "Knowledge discovery through directed probabilistic topic models: a survey," Frontiers of Computer Science in China, vol. 4, no. 2, pp. 280-301, Jun. 2010. [Online]. Available: http://dx.doi.org/10.1007/s11704-009-0062-y M. Steyvers and T. Griffiths, Probabilistic Topic Models. Lawrence Erlbaum Associates, 2007. [Online]. Available: http://www.worldcat.org/isbn/1410615340 N. Goodman, J. Tenenbaum, T. O'Donnell, and the Church Working Group. Probabilistic Models of Cognition http://projects.csail.mit.edu/church/wiki/Probabilistic_Models_of_Cognition Coin examples: Church Learning as Conditional Inference

References [1] A. Venkataraman, Y. Rathi, M. Kubicki, C.-F. Westin, and P. Golland, "Joint generative model for fMRI/DWI and its application to population studies." Medical Image Computing and Computer-Assisted Intervention, vol. 13, no. Pt 1, pp. 191-199, 2010. [Online]. Available: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3056120&tool=pmcentrez&rendertype=abstract [2] B. Milch, B. Marthi, S. Russell, D. Sontag, D. L. Ong, and A. Kolobov, "Blog: Probabilistic models with unknown objects," in In IJCAI, 2005, pp. 1352-1359. [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.116.2131 [3] N. D. Goodman, V. K. Mansinghka, D. Roy, K. Bonawitz, and J. B. Tenenbaum, "Church: a language for generative models," in Uncertainty in Artificial Intelligence, 2008. [Online]. Available: http://web.mit.edu/droy/www/papers/GooManRoyBonTenUAI2008.pdf [4] http://code.google.com/p/haines/ plate notation example: M. Steyvers and T. Griffiths, Probabilistic Topic Models. Lawrence Erlbaum Associates, 2007. [Online]. Available: http://www.worldcat.org/isbn/1410615340