Nonlinear Network Structures for Optimal Control Automation & Robotics Research Institute (ARRI) Nonlinear Network Structures for Optimal Control Frank L. Lewis and Murad Abu-Khalaf Advanced Controls, Sensors, and MEMS (ACSM) group

System Cost The Usual Suspects

NONLINEAR QUADRATIC REGULATOR Generalized HJB Equation Optimal Control (SVFB) Hamilton-Jacobi-Bellman (HJB) Equation

PROBLEM- HJB usually has no analytic solution SOLUTION- Successive Approximation a stabilizing control A contraction map (Saridis) Saridis and Beard used Galerkin Approx to allow for GHJB solution Converges to optimal solution Gives u(x) in SVFB form

For Constrained Controls NONLINEAR NONQUADRATIC REGULATOR with Nonquadratic form- Lyshevsky PD if u

Natural, exact, no approximation New GHJB is u(t) constrained if f(.) is a saturation function! tanh(p) 1 p -1

Iterate: Problem- cannot solve HJB Solution- Use Successive Approximation on GHJB Iterate: a stabilizing control

Problem- Cannot solve GHJB! Solution- Neural Network to approximate V(i)(x) Select basis set (.) x1 x2 y1 y2 VT WT inputs hidden layer outputs Two-Layer Neural Network with adjustable output weights xn ym 1 2 3 L

Cost gradient approximation Let Nonzero residual! Then GHJB is

Neural-network-based nearly optimal saturated control law.

To minimize the residual error in a LS sense Evaluate the GHJB at a number of points on Note, if Then, GHJB is

NN Training Set! Evaluating this at N points gives L x N coefficient matrix Solve by LS NN Training Set!

Select the N sample points xk Uniform Mesh Grid in Random selection- Montecarlo Approximation error is (Barron) Approximation error is Montecarlo overcomes NP-complexity problems!

ASIDE- Useful for reducing complexity of fuzzy logic systems? Uniform grid of Separable Gaussian activation functions for RBF NN

NN Training Set must be PE

Algorithm and Proofs work for any Q(x) in Constrained input given by CONSTRAINED STATE CONTROL k large and even MINIMUM-TIME CONTROL For small R and this is approx.

Example: Linear system

Region of asymptotic stability for the initial controller,

Region of asymptotic stability for the nearly optimal controller,

Example: Nonlinear oscillator