Lecture 11: Recursive Parameter Estimation u(t) y(t) ^ Model q(t-1) ^ - + u(t) y(t) Plant q + + Dr Martin Brown Room: E1k Email: martin.brown@manchester.ac.uk Telephone: 0161 306 4672 http://www.eee.manchester.ac.uk/intranet/pg/coursematerial/ e(t) EE-M110 2006/7: EF L11

Lecture 11: Outline In this lecture, we’re going to look at how to perform on-line, recursive parameter estimation The recursive parameter estimation means that training data (x(t), y(t)) is received sequentially. The aim is to use the best estimate of the parameter vector q(t) at each prediction step. The data (x(t), y(t)) will be used to “improve” the parameter vector estimate q(t-1)→q(t) after the prediction is made. The parameters trace out trajectories (with respect to time) and converge to their least squares estimates as time increases, although care needs to be taken to ensure the inputs are persistently exciting (see next lecture). ^ ^ ^ EE-M110 2006/7: EF L11

Resources Core text Ljung, Chapter 11 On-line notes, Chapter 5 Norton, Chapter 7.3 Introduction In this lecture, we’re looking at how the model’s parameters can be sequentially updated, based on information about the model’s performance. This could be because the model was only partially identified prior to operation (unlikely) or the plant is slowly time-varying or weakly non-linear. EE-M110 2006/7: EF L11

Introduction to Recursive Parameter Estimation Recursive parameter estimation is a technique that aims to alter the model’s parameters as data is sequentially received from The Hessian matrix, H(t) = X(t)TX(t), can be calculated at each stage and directly inverted, but this is computationally costly. Recursive parameter estimation is an iterative technique that aims to iteratively, efficiently calculate the new value, based on the previous one x(t) ^ Model q(t-1) y(t) ^ - + x(t) y(t) Plant q + + e(t) EE-M110 2006/7: EF L11

Rational for Recursive Parameter Estimation There are many reasons for considering on-line adaptive modelling: The real plant parameters switch or drift over time, possibly due to mechanical wearing or changing environmental conditions The plant is non-linear and a local linear model is used and adaptation is used to account for changing operating point In both cases, however, the basic theory we’re looking at must be adapted as we’re assuming that the optimal model is time invariant and is described by q. EE-M110 2006/7: EF L11

Preliminary Notation The parameter vector where At time t, before the parameter vector is updated The information used to update t-1→ t is contained in: EE-M110 2006/7: EF L11

Next Instant Data The scaled parameter covariance matrices at successive time instants: The cross correlation vectors at successive time instants are related by: Therefore: EE-M110 2006/7: EF L11

Small Matrix Inversion Lemma The small matrix inversion lemma states that: The small here does not refer to the fact that the result is not true if the matrix BCD is not small in rank, rather to the fact that this works efficiently when BCD has a small rank. This can be verified by pre/post multiplication by (A+BCD) Now let P(t) = A+BCD, A = P-1(t-1), B = x(t), C = 1, D = xT(t): While this looks complex, it means that to update P(t-1) to P(t), we do not need to invert a matrix (O(n3)), all we need to perform are matrix multiplications and division by the scalar term (O(n2)): EE-M110 2006/7: EF L11

Iterative, Error-based Parameter Update There are many ways to combine the update for P and b. The most common way is to define the error variable Then substituting for y(t) in b(t) expression gives: Multiplying by P(t) and substituting gives: This can be interpreted as adding a correction to the current least squares parameter estimate, that depends on: The size of the (scalar) output error The inner product (vector) between the input and covariance matrix EE-M110 2006/7: EF L11

Recursive Parameter Estimation Algorithm Form the new input vector x(t) using the new data Form e(t) from the model using Form P(t) using Update the least squares estimate Proceed with next time step EE-M110 2006/7: EF L11

Initialising Iterative Algorithm To start the algorithm, we need to know This is typically achieved in two ways: Wait for t-1 time steps and use the data to directly calculate P(t-1), . Then use recursive algorithm from step t. Guess the values of and set P(0) = p0I, where p0=1000 is usual. p0 determines the speed of initial adaptation as it represents the initial covariance matrix. A large value means the initial estimates are uncertain. EE-M110 2006/7: EF L11

Example: Electrical Circuit Initial RLS parameters: There is a step at zero and 1000 learning iterations where u=1 Note that noise is: randn(‘state’, 123456); y = y+0.1*randn(1,1); Reasonably fast initial convergence to q  [0.5 0.5]T, and good setting down to the final values, filtering out the effect of the additive measurement noise EE-M110 2006/7: EF L11

Summary 11: Recursive Least Squares Recursive Least Squares is an on-line parameter estimation algorithm that directly calculates the optimal, least squares parameter estimates at each time step. It uses the small matrix lemma to efficiently calculate the new inverse Hessian/parameter covariance matrix P(t), given the previous matrix P(t-1) The iterative parameter estimates has a familiar form To start the algorithm, we need to initialise In the next lecture we’ll look at the concept of persistent excitation and forgetting RLS algorithms An important part of Kalman filtering, where we simultaneously estimate the states EE-M110 2006/7: EF L11

Laboratory 11: Recursive Least Squares Theory Verify the small matrix inversion lemma S8 Verify the derivation of the Recursive Least Squares algorithm, S7-10 Matlab 1. Implement the RLS algorithm on the electrical circuit with: with 10 time steps. Plot the parameter trajectories (S12). Try changing p0 and q(0) 2. Increase the number of time steps to 1000, what are the optimal estimated parameter values (S12). EE-M110 2006/7: EF L11