1. Homework 1 (parts 1 and 2)
For the general system described by the following dynamics:
We have the following algorithm for generating the Kalman gain and var-cov matrix:
Hidden variable
Measured variable
Part 1. Using the matrix inversion lemma, first show that:
Part 2: And then show that:
So we see that if the measurement noise is large (e.g., resulting in an R with large eigen values), then the Kalman gain will become small. The system will not “trust” the measurements. On the other hand, if the confidence in the current estimates of w* is small, i.e., P is large, then the system will tend to “trust” the measurements and try to learn.

2. Homework 1 (part 3)
Part 3: We are interested in the value of the Kalman gain as the system converges. Suppose that we have a scalar measurement y and scalar hidden parameter w:
Hidden variable
Measured variable
Using the results that you got for parts 1 and 2, find an expression for the Kalman gain as n goes to infinity. Hint: using result of part 1, find the steady state value of P by setting P(n|n) = P(n-1|n-1).
Check your results to see if the following statements are true:
If the state update noise q is zero, then K converges to zero (the system stops learning).
The larger the state update noise, the larger the value that K converges to.
The larger the measurement noise, the smaller the value that K converges to.