Tijl De Bie John Shawe-Taylor ECS, ISIS, University of Southampton A statistical learning approach to subspace identification of dynamical systems Tijl De Bie John Shawe-Taylor ECS, ISIS, University of Southampton

Warning! work in progress…

PASCAL workshop - Bohinj Overview Dynamical systems and the state space representation Subspace identification for linear systems Regularized subspace identification Kernel version for nonlinear dynamical systems Preliminary experiments PASCAL workshop - Bohinj

PASCAL workshop - Bohinj Dynamical systems Dynamical systems: Accepts inputs ut 2 <m Generates outputs yt 2 <l Is affected by noise… E.g.: yt = (a1yt-1 + a2yt-2) + (b0ut + b1ut-1) + nt u0, u1, …, ut, … y0, y1, …, yt, … Dynamical system PASCAL workshop - Bohinj

PASCAL workshop - Bohinj Dynamical systems Practical examples: Car: inputs = position of steering wheel, force applied to wheel, clutch position, road conditions,… outputs = position of the car Chemical reactor: inputs = inflow of reactants, warmth added,… outputs = temperature, pressure,… Other: bridges, vocal tract, electrical systems,… PASCAL workshop - Bohinj

State space representation Next position of car (output) depends on the current inputs, and on the current position and speed Temperature and pressure (outputs) depend on the current inputs and on the current composition of the mixture, temperature and pressure Summarize total effect of past inputs in the state xt 2 <n of the system: speed of car / mixture composition This leads to the state space representation: PASCAL workshop - Bohinj

State space representation State space representation (SSR): Memory, stored in the state vector: xt Summarizes the past u0, u1, …, ut, … y0, y1, …, yt, … State update equation: xt+1 = fstate(xt,ut,wt) Output equation: yt = foutput(xt,ut,vt) PASCAL workshop - Bohinj

State space representation Linear state space model: Interpretation: the states are latent variables with Markov dependency… Note: even simple systems like the car are nonlinear, but often linear is a good approximation around working point State update equation: xt+1 = Axt + But + wt Output equation: yt = Cxt + Dut + vt PASCAL workshop - Bohinj

State space representation Advantages of the SSR: Intuitive, often close to ‘first principles’ State observer (such as Kalman filter) allows to estimate states based on input and outputs  optimal control based on these Kalman states. This is, if the system (i.e. and ) is known… If the system is not specified: algorithms for system identification exist, often identifying a SSR PASCAL workshop - Bohinj

PASCAL workshop - Bohinj Overview Dynamical systems and the state space representation Subspace identification for linear systems Regularized subspace identification Kernel version for nonlinear systems Preliminary experiments PASCAL workshop - Bohinj

Subspace identification for linear systems System identification: Given: and Noise unknown but iid … (other technical conditions) Determine: system parameters, i.e. the system matrices Classical system identification focuses on asymptotic unbiasedness (sometimes consistency) of the estimators  ok for large samples Regularization issues are at most a side-note (+/- 10 pages in the 600 pages reference work by Ljung) Explanation: often datasets are relatively low dimensional and large PASCAL workshop - Bohinj

Subspace identification for linear systems Fact: If an estimate for the state sequence is known, determining the system matrices is a (least squares) regression problem: PASCAL workshop - Bohinj

Subspace identification for linear systems Estimate state sequence first ! Two observations: We can estimate state based on past inputs and past outputs (and of initial state x0) So, for any vector there are vectors and such that: PASCAL workshop - Bohinj

Subspace identification for linear systems Future outputs are depending on the current state and future inputs So, for any vector there are vectors and such that: PASCAL workshop - Bohinj

Subspace identification for linear systems Use notation: Similarly past and future outputs and State sequence: ‘Past inputs’ as columns, time shifted ‘Future inputs’ as columns, time shifted PASCAL workshop - Bohinj

Subspace identification for linear systems Then: which is strongly related to CVA (Canonical Variate Analysis), a well known subspace identification method… Solution: generalized eigenvalue problem (like CCA); as many significantly nonzero eigenvalues as the dimensionality of the state space Reconstruct state sequence as: However: generalization problems with high-dimensional input space and small sample sizes… PASCAL workshop - Bohinj

PASCAL workshop - Bohinj Overview Dynamical systems and the state space representation Subspace identification for linear systems Regularized subspace identification Kernel version for nonlinear systems Preliminary experiments PASCAL workshop - Bohinj

Regularized subspace identification Introduce regularization: Solution: generalized eigenvalue problem PASCAL workshop - Bohinj

Regularized subspace identification Introduce regularization: Notes: Different regularization: if output space is high dimensional, also on the and Different constraints (e.g. if is omitted and without regularization, we get exactly CVA) PASCAL workshop - Bohinj

PASCAL workshop - Bohinj Overview Dynamical systems and the state space representation Subspace identification for linear systems Regularized subspace identification Kernel version for nonlinear systems Preliminary experiments PASCAL workshop - Bohinj

Kernel version for nonlinear systems Hammerstein models (input nonlinearity): Examples: in the car examples relations between forces, angles of the steering wheel,… involve sinus functions; also: saturation in actuators correspond to (‘soft’) sign functions Using feature map on inputs , then: (with B having potentially infinitely many columns) PASCAL workshop - Bohinj

Kernel version for nonlinear systems Use representer theorem: where = vector containing (with the training samples) and containing dual variables… PASCAL workshop - Bohinj

Kernel version for nonlinear systems Then:  Kernel version of subspace identification algorithm; regularization is absolutely necessary here… Solution: similar generalized eigenvalue problem (cfr. Kernel-CCA) Extensions (further work): output nonlinearity (Wiener-Hammerstein systems); requires the inverse feature map… PASCAL workshop - Bohinj

PASCAL workshop - Bohinj Overview Dynamical systems and the state space representation Subspace identification for linear systems Regularized subspace identification Kernel version for nonlinear systems Preliminary experiments PASCAL workshop - Bohinj

Preliminary experiments Experiments with system (200 samples): Random Gaussian inputs with std=1, Gaussian noise PASCAL workshop - Bohinj

PASCAL workshop - Bohinj Further work Conclusions: Regularization ideas imported into system identification for large dimensionalities / small samples Kernel trick allows for identification of Hammerstein nonlinear systems Further work: Motivation of our type of regularization with learning theory bounds Wiener-Hammerstein models (also output nonlinearity) Extension of notions like controllability to such nonlinear models Design of Kalman filter and controllers based on this kind of Hammerstein systems PASCAL workshop - Bohinj

PASCAL workshop - Bohinj Thanks! Questions? PASCAL workshop - Bohinj