222. OPTIMAL CONTROL OF CONTINUOUS DYNAMIC SYSTEMS
General Continuous Optimal Control Problem
plant
u(t)
x(t)
The optimal control problem is to find the input control signal u*(t) on the time interval [to,T] that drives the plant (system)
along a trajectory x*(t) such that the cost function:
is minimised, with terminal conditions:
where

223. Calculus of Variations - Leibniz’s Rule
If is a function of t and:
where J(x) and h(.) are scalar functionals (i.e. functions of the function x(t)), then the variation:
where
( a column vector (= gradient))
At a stationary point dJ = 0

224. Solution of the Optimal Control Problem
Using Lagrange multipliers, we obtain the augmented performance index:
where is a constant multiplier and
is a function of time.
Define the Hamiltonian function:
giving:-

225. Then from Leibniz’s rule:-
Now by integration by parts
noting that

226. Hence:
For the particular case when to and x(to) are fixed,
dto = 0, dx(to) = 0 giving:-

227. For a stationary point, for all independent
Hence, the necessary conditions for a minimum are:-
(system model)
(costate equations)
(stationarity condition)
(boundary condition)
(final-state constraint)

228. Summary of Necessary Optimality Conditions
System description
System model:
Performance index:
Final state constraint:

229. Optimal controller
Hamiltonian:
Costate equation:
Boundary conditions:
Stationarity:
State equation:

230. Notes
1. The optimal control depends upon the solution of a two- point boundary value problem
2. For fixed time horizon problems (i.e. to and T fixed), dT = 0 and the boundary condition becomes:-
3. If x(T) is fixed, dx(T) = 0 and (T) is free.
4. If x(T) is free,
and, furthermore, if does not exist,
and, even further, if does not exist
5. In the time invariant case, it can be shown that
the Hamiltonian is constant along an optimal
trajectory

231. Example
u(t)
x(t)
plant

232. Application of the optimality conditions
Hamiltonian:
Co-state equation:
Stationarity:
Boundary conditions:
Hence, the two point boundary value problem is:

233. Solution of the two point boundary value problem
Write the two point boundary problem as:
The solution will be of the form:
where 1, 2 are the eigenvalues of A
e1, e2 are the corresponding eigenvectors.
calculated as follows:

234. Hence:

235. where 1 and 2 are evaluated to satisfy the boundary conditions: x(0) = 1, (T) = 0; giving:
Hence:

236. -2e-2t e-2t u(t) x(t) plant
and the optimal control signal and state response are:
Note: for large T
-2e-2t
x(t)
plant
u(t)
e-2t
Note: the determined optimal control is open loop

237. Feedback solution
If we assume:
Then:
and substitution from the co-state and state equations gives:
where the constant c needs to be evaluated to satisfy the boundary condition s(T) = 0. This gives c = 4T and:

238. Hence, the feedback control is:
where the time dependent feedback gain is:
x(t)
plant
u(t)
-k(t)
gain
Note as:

239. The closed-loop system can also be drawn in the form:
x(t)
plant
u(t)
k(t)
gain
ref = 0 + -

240. MATLAB results:

241. Sub-optimal control:
Use the steady-state value:

243. MATLAB Demo

244. Linear Quadratic Regulator (LQR)
System model:
Performance index:
where Q and S(T) are +ve semi def, R is +ve def; Q, S(T) and R are all symmetric, T is fixed. We wish to determine u*(t) on [0,T] to minimise J.
Hamiltonian:
Co-state equation:
Stationarity:
Boundary conditions:

245. Two-point boundary value problem:
which can be written as the hamiltonian system:
hamiltonian matrix

246. Feedback solution:
Assume:
This gives closed-loop feedback control:
and the two-point boundary value problem becomes
giving the matrix Riccati equation:
which can be “solved” backwards in time.

247. The optimal control is:
That is:
where K(t) (the Kalman gain) is:
plant
K(t)
controller
‘time-dependent’ + -
r = 0
x(t)
u(t)
-x(t)

248. Steady -state closed-loop control and suboptimal feedback
the matrix Riccati equation converges to:
and we can then use a constant feedback
gain
is given by solving the algebraic Riccati equation:
We then obtain the steady-state Kalman gain:

249. plant
controller
‘time-invariant’ + -
r = 0
x(t)
u(t)
-x(t)

250. Example x1 u x2 Here: Substitution into the matrix Riccati equation:
produces:

251. The Kalman gain matrix:
then gives:

252. Feedback control structurex1 x2 u
k1(t)
k2(t)
ref = 0 + -

253. Numerical solution of the Riccati equations gives:k1 k2

254. For large T, the steady-state solution of the Riccati equations gives:
noting that we take positive values of s12 and s22 to ensure that S is +ve def. This gives negative feedback Kalman gains:

255. and the sub-optimal control structure (optimal if ):x1 x2 u 1
ref = 0 + -

258. “symplectic” Let us look at the Hamiltonian matrix:
The eigenvalues of H are:
“symplectic”

260. The stable eigenvalues are those of the sub-optimal
closed-loop system.

261. Linear Quadratic Regulator Design Using MATLAB

262. Example
produces:
which agrees with our previous analysis.

263. MATLAB Demo

264. Constrained Input Problems The Minimum (Maximum) Principle
We will now consider the situation when the control signal u(t) is constrained to lie in an admissible region:
Consider the general problem:

265. for all admissible where * denotes an optimal value.
The procedure is similar to before, but with a significant alteration of the stationarity condition which now becomes the optimality condition:
for all admissible where * denotes an optimal value.
Note: The condition Hu =0 still applies if u is inside the region, but not when any component of u is on a bound.
L.S. Pontryagin
This is the MINIMUM PRINCIPLE.

266. Summary of Necessary Optimality Conditions When the Input is Constrained
System model:
Performance index:
Final state constraint:
System description

267. Optimal controller
Hamiltonian:
Costate equation:
Boundary conditions:
Optimality:
state equation:

268. Example x1 x2 u +1 -1
Hamiltonian:
Co-state equations:

269. Boundary conditions:
Optimality condition: To minimise H we require 2u to be as small as possible where: Hence, if 2 is +ve then use u = -1; if 2 is -ve then use u = +1. That is the optimal control is:
This is bang-bang control.

270. Solution of the co-state equations
Hence, 2(t) is linear and can change sign a maximum of only once.
Optimal control: Since the optimal control is one of four choices, depending upon the particular initial condition.
1) u(t) = -1 for all t
2) u(t) = -1 switching to +1
3) u(t) = +1 switching to -1
4) u(t) = +1 for all t

271. phase-plane trajectories
Hence:
which is the equation of a parabola passing through (x10,x20). Hence, the phase portrait will be a family of parabolas, one for each initial condition.

273. u=+1
u=-1
switching
curve
x10,x20

274. The only way to arrive at x1(T) = 0, x2(T) = 0, starting at (x10,x20), at the particular point shown, is to first apply u = -1. When the state arrives at the switching curve, the control is then switched to u = +1 which then drives the state to the origin. (Note that if (x10,x20) happens to be on the switching curve the u(t) would remain at +1 or -1 for all t)
The equation of the switching curve is:

275. giving the feedback law:x1 x2 u
feedback
law +1 -1

279. MATLAB Demo