1. Introductory Control Theory

3. Control Theory The use of feedback to regulate a signal Controller Plant Desired signal x d Signal x Control input u Error e = x-x d (By convention, x d = 0) x’ = f(x,u)

4. What might we be interested in? Controls engineering  Produce a policy u(x,t), given a description of the plant, that achieves good performance Verifying theoretical properties  Convergence, stability, optimality of a given policy u(x,t)

5. Agenda PID control LTI multivariate systems & LQR control Nonlinear control & Lyapunov funcitons

6. PID control Proportional-Integral-Derivative controller A workhorse of 1D control systems

7. Proportional term u(t) = -K p x(t) Negative sign assumes control acts in the same direction as x x t Gain

8. Integral term u(t) = -K p x(t) - K i I(t) I(t) =  0 t x(t) dt (accumulation of errors) x t Residual steady-state errors driven asymptotically to 0 Integral gain

9. Instability For a 2 nd order system (momentum), P control x t Divergence

10. Derivative term u(t) = -K p x(t) – K d x’(t) x Derivative gain

11. Putting it all together u(t) = -K p x(t) - K i I(t) + K d x’(t) I(t) =  0 t x(t) dt

12. Parameter tuning

13. Example: Damped Harmonic Oscillator Second order time invariant linear system, PID controller  x’’(t) = A x(t) + B x’(t) + C + D u(x,x’,t) For what starting conditions, gains is this stable and convergent?

14. Stability and Convergence System is stable if errors stay bounded System is convergent if errors -> 0

15. Example: Damped Harmonic Oscillator x’’ = A x + B x’ + C + D u(x,x’) PID controller u = -K p x –K d x’ – K i I x’’ = (A-DK p ) x + (B-DK d ) x’ + C - D K i I

16. Homogenous solution Instable if A-DK p > 0 Natural frequency  0 = sqrt(DK p -A) Damping ratio  =(DK d -B)/2  0 If  > 1, overdamped If  < 1, underdamped (oscillates)

17. Multivariate Systems x’ = f(x,u) x  X  R n u  U  R m Because m  n, and variables are coupled, this is not as easy as setting n PID controllers

18. Linear Time-Invariant Systems Linear: x’ = f(x,u,t) = A(t)x + B(t)u LTI: x’ = f(x,u) = Ax + Bu Nonlinear systems can sometimes be approximated by linearization

19. Convergence of LTI systems x’ = A x + B u Let u = - K x  Then x’ = (A-BK) x The eigenvalues i of (A-BK) determine convergence  Each i may be complex  Must have real component between (-∞,0]

20. Linear Quadratic Regulator x’ = Ax + Bu Objective: minimize quadratic cost  x T Q x + u T R u dt Over an infinite horizon Error term“Effort” penalization

21. Closed form LQR solution Closed form solution u = -K x, with K = R -1 BP Where P solves the Riccati equation  A T P + PA – PBR -1 B T P + Q = 0  Derivation: calculus of variations

22. Solving Riccati equation Solve for P in A T P + PA – PBR -1 B T P + Q = 0 Existing iterative techniques, e.g. in Matlab

23. Nonlinear Control x’ = f(x,u) How to find u?  Next class How to prove convergence and stability?  Hard to do across X

24. Proving convergence & stability with Lyapunov functions Let u = u(x) Then x’ = f(x,u) = g(x) Conjecture a Lyapunov function V(x)  V(x) = 0 at origin x=0  V(x) > 0 for all x in a neighborhood of origin V(x)

25. Proving stability with Lyapunov functions Idea: prove that d/dt V(x)  0 under the dynamics x’ = g(x) around origin V(x)  t g(x)  t d/dt V(x)

26. Proving convergence with Lyapunov functions Idea: prove that d/dt V(x)  < 0 under the dynamics x’ = g(x) around origin V(x)  t g(x)  t d/dt V(x)

27. Proving convergence with Lyapunov functions d/dt V(x) = dV/dx(x) dx/dt(x) =  V(x) T g(x) < 0 V(x)  t g(x)  t d/dt V(x)

28. How does one construct a suitable Lyapunov function? It may not be easy… Typically some form of energy (e.g., KE + PE)

29. Handling Uncertainty All the controllers we have discussed react to disturbances Some systems may require anticipating disturbances To be continued…

30. Motion Planning and Control Replanning = control?

31. Motion Planning and Control Replanning = control? PRM planning Reactive Control Accurate models Explicit plans computed Computationally expensive Coarse models Policies engineered Computationally cheap Optimal control Real-time planning Model predictive control

32. Planning or control? The following distinctions are more important:  Tolerates disturbances and modeling errors?  Convergent?  Optimal?  Scalable?  Inexpensive?

33. Presentation Schedule Optimal Control, 3/30  Ye and Adrija Operational space and force control, 4/1  Yajia and Jingru Learning from demonstration, 4/6  Yang, Roland, and Damien Planning under uncertainty, 4/8  You Wei and Changsi Sensorless planning, 4/13  Santhosh and Yohanand Planning to sense, 4/15  Ziaan and Yubin