1. Lecture 8 Generalized Linear Models &
Iterated Reweighted Least Squares (IRLS) Algorithm

2. Exponential Family
Moments & canonical parameters representation for EFD.
Sufficiency: T(x) is all there is to know about parameters.
ML estimation: moments are simply average SS.
Generalized Linear Models for discriminative Supervised L.
- p(Y|X) = expFamDistr.
- conditional mean = f(z) f = link func. or response func.
- z = a’*x (linear)
Canonical link function gives simple MLE problem (linear in x)
Online gradient descent algorithm.

3. IRLS Do Newton-Ralphson iterations.
Updates become like solving a weighted least squares problem, with weights changing at each iteration.
example: logistic regression
demo_LogReg