Robotics Research Laboratory 1 Chapter 7 Multivariable and Optimal Control

Robotics Research Laboratory 2 Time-Varying Optimal Control - deterministic systems

Robotics Research Laboratory 3

4 LQ problem (Linear Quadratic)–Finite time problem Using Lagrange multipliers

Robotics Research Laboratory 5

6

7

8

9

10

Robotics Research Laboratory 11 LQR (Linear Quadratic Regulator) -Infinite time problem ARE(Algebraic Riccati Equation) – analytic solution is impossible in most cases. – numerical solution is required.

Robotics Research Laboratory 12

Robotics Research Laboratory 13

Robotics Research Laboratory 14 Remark : Using reciprocal root properties in p372

Robotics Research Laboratory 15

Robotics Research Laboratory 16

Robotics Research Laboratory 17

Robotics Research Laboratory 18 Eigenvector Decomposition

Robotics Research Laboratory 19 inside the unit circle outside the unit circle

Robotics Research Laboratory 20

Robotics Research Laboratory 21

Robotics Research Laboratory 22

Robotics Research Laboratory 23

Robotics Research Laboratory 24

Robotics Research Laboratory 25

Robotics Research Laboratory 26

Robotics Research Laboratory 27 Cost Equivalents

Robotics Research Laboratory 28

Robotics Research Laboratory 29 Least Squares Estimation p  1 measurement vector p  1 measurement error vector p  n matrix n  1 unknown vector

Robotics Research Laboratory 30

Robotics Research Laboratory 31

Robotics Research Laboratory Sales fit and prediction Sales ($1000) Months

Robotics Research Laboratory 33 Weighted Least Squares

Robotics Research Laboratory 34 Recursive Least Square

Robotics Research Laboratory 35 old estimate new estimate covariance of old estimate

Robotics Research Laboratory 36 Sometime, it is a scalar. That is if we use just one new information.

Robotics Research Laboratory 37 Stochastic Models of Disturbance We have dealt with well-known well-defined, ideal systems. - disturbance (process, load variation) - measurement noise 0 real line new (range) sample space sample point in sample space (event) s

Robotics Research Laboratory 38

Robotics Research Laboratory 39

Robotics Research Laboratory 40

Robotics Research Laboratory 41

Robotics Research Laboratory 42

Robotics Research Laboratory 43

Robotics Research Laboratory 44

Robotics Research Laboratory 45

Robotics Research Laboratory 46

Robotics Research Laboratory 47

Robotics Research Laboratory 48 F(X, t 2 ) F(X, t 1 ) F(X, t 3 ) 0 1 X X 0 t1t1 t2t2 t3t3 t X(,  1 ) X(,  2 ) X(,  3 )

Robotics Research Laboratory 49 Remark:

Robotics Research Laboratory 50

Robotics Research Laboratory 51

Robotics Research Laboratory 52 A random process X(t) is Gaussian process if for any t 1, …, t m, and any m, the random vector X(t 1 ) … X(t m ) have the Gaussian distribution. A Gaussian process is completely characterized by its mean and its autocorrelation If Gaussian process X(t) is w.s.s, then it is strictly stationary. Assume that X(t) is wide sense stationary Let then is Fourier transform of. It is called a Spectral Density Matrix.

Robotics Research Laboratory 53 Remark: A random process X(t) is a Markov process if for all t 1 <t 2 <···<t m, all m, all x 1,…, x m A random processes X(t) is independent if the random vectors X(t 1 ) ··· X(t m ) are mutually independent for all t 1 <t 2 <···<t m and all m. Note: Andrei Andreevich Markov (1856 – 1922)

Robotics Research Laboratory 54 ex) Consider a scalar random process X(t), t  0 defined from where X(0) is zero mean Gaussian random variable with. 00 t X(t)X(t)

Robotics Research Laboratory 55

Robotics Research Laboratory 56 The density function X(t) is A random process w(t), t  0 is a white process if it is zero mean with the property that w(t 1 ) and w(t 2 ) are independent for all t 1  t 2 and where Q(t 1 ) is intensity

Robotics Research Laboratory 57 Remarks: i) If Q(t) is constant, i.e. Q(t) = Q then w(t) is w.s.s. and the spectral density is ii) A white process is not a mathematical rigorous random process. iii) A sample function for a white noise process can be thought as composed of superposition of large number of independent pulse of brief duration with random amplitude. iv) If the amplitude of the pulse is Gaussian, the w(t) is a Gaussian white noise. v) A white noise is a ‘derivative’ of a Wiener process (Brown motion)

Robotics Research Laboratory 58 ex) Similarly, Since {v(k)} is a white process, {X(k)} is a random process. X(0) should be specified. It is assumed that white process Wiener process

Robotics Research Laboratory 59

Robotics Research Laboratory 60

Robotics Research Laboratory 61

Robotics Research Laboratory 62 Remarks: A stable linear time-invariant discrete-time system has a pulse transfer function H(z). Suppose that the input u(k) is w.s.s. with a spectral density matrix. Then the output y(k) is w.s.s. and the spectral density of the output y(k) is In a scalar case, h(k-j) u(j)y(k)

Robotics Research Laboratory 63

Robotics Research Laboratory 64

Robotics Research Laboratory 65 ex)

Robotics Research Laboratory 66 white noise with intensity I Note: Norbert Wiener (1894 – 1964) Wiener filter for stationary I/O case in 1949 “Everything” can be generated by filtering white noise. L.T.I w(k)y(k) white process with intensity I colored noise

Robotics Research Laboratory 67

Robotics Research Laboratory 68 LQ + Kalman Filter ( ~ state feedback + observer by pole placement) LQG(Linear Quadratic Gaussian) problem - Partially informed states

Robotics Research Laboratory 69

Robotics Research Laboratory 70 Given y(0), y(1), … y(k), determine the optimal estimate such that an n  n positive definite matrix i.e., minimum variance of error Remarks: i) P(k) is minimum  *P(k)  is minimum where  is an arbitrary vector ii) P(k) is minimum Problem Formulation

Robotics Research Laboratory 71 Let the prediction-type Kalman filter have the form. -Predictor type, One-step-ahead estimator - where L(k) is time-varying y(k) is a measured output is an output from the model. Define as a reconstruction error.

Robotics Research Laboratory 72

Robotics Research Laboratory 73 where 0 0

Robotics Research Laboratory 74 minimize matrix scalar

Robotics Research Laboratory 75 Note: Kalman and Bucy filter for time-varying state space in 1960

Robotics Research Laboratory 76 Remarks: i) ii) a priori information are iii) due to system dynamics due to disturbance w(k) last term due to newly measured information iv) P(k) does not depend on the observation. Thus the gain can be precomputed in forward time and stored. v) steady-state Kalman filter – all constants

Robotics Research Laboratory 77

Robotics Research Laboratory 78

Robotics Research Laboratory 79

Robotics Research Laboratory 80 time transient L = 0.01 L = 0.05 L(k):optimal gain Steady-state

Robotics Research Laboratory 81 colored noise Frequency Domain Properties of Kalman Filter

Robotics Research Laboratory 82

Robotics Research Laboratory 83

Robotics Research Laboratory 84 Remarks: i)It gives an idea how the Kalman filter attenuates different frequency. ii)Kalman filter has zeros at the poles of the noise model. (notch filter)

Robotics Research Laboratory 85 Smoothing: To estimate the Wednesday temperature based on temperature measurements from Monday, Tuesday and Thursday. Filtering: To estimate the Wednesday temperature based on temperature measurements from Monday, Tuesday and Wednesday. Prediction: To estimate the Wednesday temperature based on temperature based on temperature measurements from Sunday, Monday and Tuesday.

Robotics Research Laboratory 86 Stochastic LQ Control Problem

Robotics Research Laboratory 87 LQG Control Problem

Robotics Research Laboratory 88

Robotics Research Laboratory 89 Stationary LQG Control Problem

Robotics Research Laboratory 90

Robotics Research Laboratory 91 Control and Estimation Duality