1. Nonlinear Network Structures for Optimal Control
Automation & Robotics Research Institute (ARRI)
Nonlinear Network Structures for Optimal Control
Frank L. Lewis and Murad Abu-Khalaf
Advanced Controls, Sensors, and MEMS (ACSM) group

2. System
Cost
The Usual Suspects

4. NONLINEAR QUADRATIC REGULATOR
Generalized HJB Equation
Optimal Control (SVFB)
Hamilton-Jacobi-Bellman (HJB) Equation

5. PROBLEM- HJB usually has no analytic solution
SOLUTION- Successive Approximation
a stabilizing control
A contraction map
(Saridis)
Saridis and Beard used Galerkin
Approx to allow for GHJB solution
Converges to optimal solution
Gives u(x) in SVFB form

6. For Constrained Controls
NONLINEAR NONQUADRATIC REGULATOR
with
Nonquadratic form- Lyshevsky
PD if u

7. Natural, exact, no approximation New GHJB is
u(t) constrained if f(.) is a saturation function!
tanh(p) 1 p -1

8. Iterate: Problem- cannot solve HJB
Solution- Use Successive Approximation on GHJB
Iterate:
a stabilizing control

11. Problem- Cannot solve GHJB!
Solution- Neural Network to approximate V(i)(x)
Select basis set
(.) x1 x2 y1 y2 VT WT
inputs
hidden layer
outputs
Two-Layer Neural Network with adjustable output weights xn ym 1 2 3 L

12. Cost gradient approximation
Let
Nonzero residual!
Then GHJB is

13. Neural-network-based nearly optimal saturated control law.

14. To minimize the residual error in a LS sense
Evaluate the GHJB at a number of points on
Note, if
Then, GHJB is

15. NN Training Set! Evaluating this at N points gives
L x N coefficient matrix
Solve by LS
NN Training Set!

16. Select the N sample points xk
Uniform Mesh Grid in
Random selection- Montecarlo
Approximation error is
(Barron)
Approximation error is
Montecarlo overcomes NP-complexity problems!

17. ASIDE-
Useful for reducing complexity of fuzzy logic systems?
Uniform grid of
Separable Gaussian activation
functions for RBF NN

18. NN Training Set must be PE

19. Algorithm and Proofs work for any Q(x) in
Constrained input given by
CONSTRAINED STATE CONTROL
k large and even
MINIMUM-TIME CONTROL
For small R and
this is approx.

20. Example: Linear system

21. Region of asymptotic stability for the initial controller,

22. Region of asymptotic stability
for the nearly optimal controller,

23. Example: Nonlinear oscillator