ICS 280 Learning in Graphical Models Time: Tu-Th, 5-6.20 pm Instructor: Max Welling

Further Information URL: http://www.ics.uci.edu/~welling/teaching/GraphicalModels.html check for readings, homework and other updates. Prerequisites: ICS 274 Probabilistic Learning: Theory and Algorithms, or with consent of instructor.

Overview review of statistical concepts introduction to graphical models hidden variables and exact inference learning in graphical models unsupervised learning supervised learning graphical models of time series approximate inference Bayesian learning and structure learning

Review of Statistical Concepts

Basic Definitions definition probability distribution, discrete/continuous joint, conditional & marginal distributions independence Bayes rule moments: mean & covariance multivariate Gaussian distribution learning, supervised & unsupervised observed and unobserved random variables

Bayesian Estimation parameters are random variables “learning” = computing the posterior distribution of the parameters given the data. priors and hyper-parameters the marginal likelihood or evidence the predictive distribution example & demo_Bayes

MAP Estimation MAP = maximum a posteriori value of the parameter much easier to compute throw away uncertainty in estimate of the parameter prior distribution still has an impact example & demo_MAP

Maximum Likelihood MAP with uniform prior (misleading) frequentist versus Baysian ML is an estimator which has “nice” properties. Bias-variance trade-off ML is asymptotic unbiased and efficient (minimal variance) Cramer-Rao bound Gaussian example & demo_ML

Generalization & Overfitting What is overfitting ? ML is not protected against overfitting MAP is partially protected against overfitting Baysian estimation is completely protected against overfitting Adding regularization terms to ML Minimum description length: the coding perspective