Download presentation
Presentation is loading. Please wait.
Published byRichard Holt Modified over 8 years ago
1
Computacion Inteligente Least-Square Methods for System Identification
2
Content System Identification: an Introduction Least-Squares Estimators Statistical Properties & the Maximum Likelihood Estimator LSE for Nonlinear Models
3
3 System Identification: Introduction Goal –Determine a mathematical model for an unknown system (or target system) by observing its input-output data pairs
4
4 System Identification: Introduction Purposes –To predict a system’s behavior, –As in time series prediction & weather forecasting –To explain the interactions & relationships between inputs & outputs of a system
5
5 System Identification: Introduction Context –To design a controller based on the model of a system, –as an aircraft or ship control –Simulate the system under control once the model is known
6
6 System Identification: Introduction There are 2 main steps that are involved –Structure identification –Parameter identification
7
7 System Identification: Introduction Structure identification Apply a-priori knowledge about the target system to determine a class of models within which the search for the most suitable model is to be conducted This class of model is denoted by a function y = f(u, ) where: y is the model output u is the input vector Is the parameter vector
8
8 System Identification: Introduction Structure identification f(u, ) depends on –the problem at hand –the designer’s experience –the laws of nature governing the target system
9
9 System Identification: Introduction Parameter identification –The structure of the model is known, however we need to apply optimization techniques –In order to determine the parameter vector such that the resulting model describes the system appropriately:
10
10 Block diagram for parameter identification System Identification: Introduction
11
11 System Identification: Introduction The data set composed of m desired input-output pairs –(u i, y i ) (i = 1,…,m) is called the training data System identification needs to do both structure & parameter identification repeatedly until satisfactory model is found
12
12 System Identification: Steps –Specify & parameterize a class of mathematical models representing the system to be identified –Perform parameter identification to choose the parameters that best fit the training data set –Conduct validation set to see if the model identified responds correctly to an unseen data set –Terminate the procedure once the results of the validation test are satisfactory. Otherwise, another class of model is selected & repeat step 2 to 4
13
13 Least-Squares Estimators
14
14 Least-Squares Estimators General form: y = 1 f 1 (u) + 2 f 2 (u) + … + n f n (u) (14) where: –u = (u 1, …, u p ) T is the model input vector –f 1, …, f n are known functions of u – 1, …, n are unknown parameters to be estimated
15
15 Least-Squares Estimators The task of fitting data using a linear model is referred to as linear regression where: –u = (u 1, …, u p ) T is the input vector –f 1 (u), …, f n (u)regressors – 1, …, n parameter vector
16
16 Least-Squares Estimators We collect a training data set {(u i, y i ), i = 1, …, m} Equation (14) becomes: Which is equivalent to: A = y
17
17 Least-Squares Estimators Which is equivalent to: A = y –where A = y = A -1 y (solution) m*n matrixn*1 vectorm*1 vector unknown
18
18 Least-Squares Estimators We have –m outputs & –n fitting parameters to find Or m equations & n unknown variables Usually m is greater than n
19
19 Least-Squares Estimators Since the model is just an approximation of the target system & the data observed might be corrupted, therefore an exact solution is not always possible! To overcome this inherent conceptual problem, an error vector e is added to compensate A + e = y
20
20 Least-Squares Estimators Our goal consists now of finding that reduces the errors between and The problem: find, estimate
21
21 Least-Squares Estimators If e = y - A then: We need to compute:
22
22 Least-Squares Estimators Theorem [least-squares estimator] The squared error is minimized when satisfies the normal equation if is nonsingular, is unique & is given by is called the least-squares estimators, LSE
23
23 Statistical Properties of least-squares estimators
24
24 Statistical qualities of LSE Definition [unbiased estimator] An estimator of the parameter is unbiased if where E[.] is the statistical expectation
25
25 Statistical qualities of LSE Definition [minimal variance] –An estimator is a minimum variance estimator if for any other estimator *: where cov( ) is the covariance matrix of the random vector
26
26 Statistical qualities of LSE Theorem [Gauss-Markov]: –Gauss-Markov conditions: The error vector e is a vector of m uncorrelated random variables, each with zero mean & the same variance 2. This means that:
27
27 Statistical qualities of LSE Theorem [Gauss-Markov]: –Gauss-Markov conditions: The error vector e is a vector of m uncorrelated random variables, each with zero mean & the same variance 2. This means that:
28
28 Statistical qualities of LSE Theorem [Gauss-Markov] LSE is unbiased & has minimum variance. Proof:
29
29 Maximum likelihood (ML) estimator
30
30 Maximum likelihood (ML) estimator –ML is one of the most widely used technique for parameter estimation of a statistical distribution ML definition: –For a sample of n observations (of a probability density function ) x 1, x 2, …, x n, the likelihood function L is defined by:
31
31 Maximum likelihood (ML) estimator The criterion for choosing is: –“pick a value of that provides a high probability of obtaining the actual observed data x 1, x 2, …, x n ” –Therefore, ML estimator is defined as the value of which maximizes L: or equivalently:
32
32 Maximum likelihood (ML) estimator Example: ML estimation for normal distribution –For m observations x 1, x 2, …, x m, we have:
33
33 Maximum likelihood (ML) estimator Example: ML estimation for normal distribution –For m observations x 1, x 2, …, x m, we have:
34
34 Statistical Properties & the ML Estimator Equivalence between LSE & MLE Theorem –Under the Gauss conditions and if each component of the vector e follows a normal distribution then: LSE of = MLE of
35
35 Statistical Properties & the Maximum Likelihood Estimator (5.7) (cont.) Maximum likelihood (ML) estimator (cont.) –Equivalence between LSE & MLE –Theorem: Under the Gauss conditions & if each component of the vector e follows a normal distribution then the LSE of = MLE of
36
36 LSE for Nonlinear Models
37
37 LSE for Nonlinear Models Nonlinear models are divided into 2 families –Intrinsically linear –Intrinsically nonlinear Through appropriate transformations of the input- output variables & fitting parameters, an intrinsically linear model can become a linear model By this transformation into linear models, LSE can be used to optimize the unknown parameters
38
38 LSE for Nonlinear Models Examples of intrinsically linear systems
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.