1. CY3A2 System identification1 Maximum Likelihood Estimation: Maximum Likelihood is an ancient concept in estimation theory. Suppose that e is a discrete random process, and we know its probability density function as a functional of a parameter θ, such that we know P(e; θ). Now we have n data samples, given just as before ( y, u ), how do we estimate θ ? The idea of Maximum Likelihood Estimation is to maximize a Likelihood function which is often defined as the joint probability of e i.

2. CY3A2 System identification2 Suppose e i is uncorrelated, the Likelihood function L can be written as (the joint probability of e i ) This means that the Likelihood function is the product of data each sample’s pdf. Consider using log Likelihood function Log L. Log function is a monotonous function. This means when L is maximum, so is Log L.

3. CY3A2 System identification3 Instead of looking for, that maximizes L, We now look for, that maximizes log L, the result will be the same, but computation is simpler!

4. CY3A2 System identification4 If is Gaussian with zero mean, and variance Also consider the link between and data observations is

5. CY3A2 System identification5

6. 6 By setting We get Which is simply equivalent to LS estimate. A common fact: Under Gaussian assumption, the Least Squares estimates is equivalent to Maximum Likelihood estimate.

7. CY3A2 System identification7 Modelling Nonlinear AutoRegressive (NAR) Model by Radial Basis Function (RBF) neural networks e.g Gaussian Radial basis function:

8. CY3A2 System identification8 Radial Basis Function Neural Networks

9. CY3A2 System identification9 Least squares (LS) can be readily used to identify RBF networks. 1.Some method to determine the centres (k-means clustering, or random selection from the data set), and given width σ. 2. You know how to estimate θ. is filled by