1. ICS 280 Learning in Graphical Models
Time: Tu-Th, pm
Instructor: Max Welling

2. Further Information
URL:
check for readings, homework and other updates.
Prerequisites: ICS 274 Probabilistic Learning: Theory and Algorithms, or with consent of instructor.

3. Overview review of statistical concepts
introduction to graphical models
hidden variables and exact inference
learning in graphical models
unsupervised learning
supervised learning
graphical models of time series
approximate inference
Bayesian learning and structure learning

4. Review of Statistical Concepts

5. Basic Definitions
definition probability distribution, discrete/continuous
joint, conditional & marginal distributions
independence
Bayes rule
moments: mean & covariance
multivariate Gaussian distribution
learning, supervised & unsupervised
observed and unobserved random variables

6. Bayesian Estimation parameters are random variables
“learning” = computing the posterior distribution of the parameters given the data.
priors and hyper-parameters
the marginal likelihood or evidence
the predictive distribution
example & demo_Bayes

7. MAP Estimation MAP = maximum a posteriori value of the parameter
much easier to compute
throw away uncertainty in estimate of the parameter
prior distribution still has an impact
example & demo_MAP

8. Maximum Likelihood MAP with uniform prior (misleading)
frequentist versus Baysian
ML is an estimator which has “nice” properties.
Bias-variance trade-off
ML is asymptotic unbiased and efficient (minimal variance)
Cramer-Rao bound
Gaussian example & demo_ML

9. Generalization & Overfitting
What is overfitting ?
ML is not protected against overfitting
MAP is partially protected against overfitting
Baysian estimation is completely protected against overfitting
Adding regularization terms to ML
Minimum description length: the coding perspective