Model Structures 1

Objective Recognize some discrete-time model structures which are commonly used in system identification such as ARX, FIR, ARMAX, OE. 2

ARX model structure In ARX model, the output y(k) is computed based on previous input and output values: This equivalent to Where e(k) is the noise at step k. Model parameters: {a 1, a 2, …, a na } and {b 1, b 2, …, b nb }. 3

Polynomial representation Defining the Backward shift operator q −1 : Then and similarly 4

ARX model in polynomial form Therefore, the ARX model is written compactly: Which has the following symbolic representation: Remark: The ARX model is quite general, it can describe arbitrary linear relationships between inputs and outputs. However, the noise enters the model in a restricted way, and later we introduce models that generalize this. 5

Returning to the explicit recursive representation: So in fact ARX model obeys the standard model structure in linear regression! Regressor vector: ϕ ∈ R na+nb, previous output and input values. Parameter vector: θ ∈ R na+nb, polynomial coefficients. Linear Regression Model 6

Consider now that we are given a vector of data u(k), y(k), k = 1,..., N, and we have to find the model parameters θ. For any k, we have: where ε(k) is now interpreted as an equation error. Objective: minimize the mean squared error: Identification Problem 7

We can write the following set of linear equations in matrix from: with notations Y ∈ R N and Φ ∈ R N×(na+nb). Linear System of Equations 8

As we know, the least squares solution is given by Note that the form above is impractical in system identification, since the number of data points N can be very large. Better form: ARX Solution 9

Consider we are given the following, separate, identification and validation data sets. plot(id); and plot(val); Remarks: Identification input: a PRBS signal, an approximation of (non- zero-mean) white noise. Validation input: a sequence of steps. Example 10

Example, continued Identifying an ARX model: model = arx(id, [na, nb, nk]); Arguments: Identification data of type iddata format. Array containing the orders of A and B and the minimum delay nk between inputs and outputs (useful for systems with time delays). 11

Example (continued): Model Validation Let us identify a second-order ARX model with: na = 2, nb = 1, nk = 1. For validation: compare(model, val); 12

Example (continued): Structure selection Better idea: try many different structures and choose the best one. Na = 1:15; Nb = 1:15; Nk = 1:5; NN = struc(Na, Nb, Nk); V = arxstruc(id, val, NN); struc generates all combinations of orders in Na, Nb, Nk. arxstruc identifies for each combination an ARX model (on the data in 1st argument), simulates it (on the data in the 2nd argument), and returns all the MSEs on the first row of V (see help arxstruc for the format of V). 13

Example (continued): Structure selection To choose the structure with the smallest MSE: N = selstruc(V, 0); %best fit to validation data For our data, N= [8, 7, 1]. Alternatively, graphical selection: N = selstruc(V, ’plot’); Then click on bar corresponding to best (red) model and the button “Select”. 14 Red — Best fit minimizes the sum of the squares of the difference between the validation data output and the model output.

Example (continued): Validation of best ARX model model = arx(id, N); compare(model,val); 15 A better fit is obtained.

Special case of ARX: FIR Setting A(q -1 ) = 1 (na = 0) in ARX model, we get: where h(j) is the impulse response of the system. Note that here h(0), the impulse response at time 0, is assumed 0 – i.e. system does not respond instantaneously to changes in input). 16

Difference between ARX and FIR Since ARX includes recursive relationships between current and previous outputs, it will be sufficient to take orders na and nb (at most) equal to the order of the dynamical system. FIR needs a sufficiently large order nb to model the entire transient regime of the impulse response (in principle, we only recover the correct model as nb → ∞). ⇒ more parameters ⇒ more data needed to identify them. 17

ARMAX model structure Number of parameters: na+nb+nc Name: AutoRegressive, Moving Average (referring to noise model) with eXogenous input (dependence on u) ARX model is a special case of ARMAX model (with nc = 0, or C(q -1 )=1 ). ARMAX can model more intricate relationships between the disturbance and the inputs and outputs. 18

Overall relationship ARMAX ⊃ ARX ⊃ FIR ARMAX to ARX: Less freedom in modeling disturbance. ARX to FIR: More parameters required. 19

Example 2 Consider again the previous data on which ARX was applied but let us now identify an ARMAX model mARMAX = armax(id, [na, nb, nc, nk]); Arguments: 1 Identification data. 2 Array containing the orders of A, B, C and delay nk. 20

Example 2, continued Considering the system is 2nd order with na = 2, nb = 2, nc = 2, nk = 1. Validation: compare(val, mARMAX); Good results! similar to ARX model (with na=8, nb=7, nk=1), but with fewer parameters ( =7). Flexible noise model pays off. 21

Output error (OE) model Other model forms are possible, e.g. Output Error, OE: This corresponds to simple, additive measurement noise on the output (the “output error”). To identify OE model we can use the command: mOE = oe(id, [nb, nf, nk]); 22