A Graph Search Algorithm for Optimal Control of Hybrid Systems Olaf Stursberg University of Dortmund Germany Work financially supported by the European Union within the project AMETIST (Advanced Methods for Timed Systems), IST

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 2 Motivation given:  Hybrid dynamic system with:  nonlinear continuous dynamics  continuous and discrete inputs  transitions with state resets  Specifications:  transfer from initial state to goal set  safety restriction (exclusion of unsafe states)  maximized performance / minimized costs [industrial relevance: start-up, shut-down or change-over of processing systems] Objective: Determine input trajectories such that the specs are met! location z1z1 z2z2 z3z3 x2x2 x1x1 initialization goal reset unsafe set x(t)x(t) v u

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 3 Why is this difficult? Standard approaches to continuous OC: Variational Calculus / Maximum Principle Necessary Conditions: Hamiltonian: adjunct DE:,... required: Bellman Principle / Dynamic Programming A total strategy is only optimal if the remaining partial strategy is optimal. Value function: HJB equation (sufficient condition): Hybrid Systems: not differentiable; required

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 4 Related Work Among others:  M. S. Branicky, V. S. Borkar, S. K. Mitter: “A unified framework for hybrid control: Model and optimal control theory”. IEEE Trans. Automatic Control, vol. 43, no. 1, 31–45,  H. Sussmann: “A maximum principle for hybrid optimal control problems,” Proc. 38th IEEE Conf. Decision and Control, 1999, 425–430.  M. Broucke, M. Di Benedetto, S. Di Gennaro, A. Sangiovanni-Vincentelli: “Optimal control using bisimulations”. Hybrid Systems: Comp. and Control, LNCS 2034, 2001, 175–188.  M. Shaikh, P. Caines: “On the optimal control of hybrid systems”. Hybrid Systems: Comp. and Control, ser, LNCS 2623, 2003, 466–481.  B. De Schutter: “Optimal control of a class of linear hybrid systems with saturation”. Proc. 38th IEEE Conf. Decision and Control, 1999, pp. 3978–3983.  B. Lincoln, A. Rantzer: “Optimizing linear system switching”. Proc. 40th IEEE Conf. Decision and Control, 2001, 2063–2068.  X. Xu, P. Antsaklis: “Quadratic optimal control problems for hybrid linear autonomous systems with state jumps”. Proc. American Control Conf., 2003, 3393–3398.  A. Bemporad, M. Morari: “Control of systems integrating logic, dynamics, and constraints,” Automatica, vol. 35, no. 3, pp. 407–427,  O. Stursberg and S. Engell: “Optimal control of switched continuous systems using mixed integer programming”. Proc. 15th IFAC World Congr. Automatic Control, vol. Th-A06-4, 2002.

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 5  continuous states:  continuous inputs:  finite set of discrete inputs:  finite set of locations:  invariants:, polyhedral for all z  transitions:  guards:, polyhedral, disjoint for each z  resets:  flow functions: s.t. defines a continuous vector field Hybrid Model - Syntax Hybrid automaton:

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 6 Hybrid Model - Semantics Time set: T = {t 0, t 1, t 2,...}contains event times Hybrid state:  k  (z k, x k )   with x k = x(t k ), z k = z(t k ) Input trajectories:  u = (u 0, u 1,...)  u,  v = (v 0, v 1,...)   v with u k, v k constant for t  [t k, t k+1 [ Feasible run of HA for given  0,  u and  v :   = (  0,  1,  2,...) with  k from: (i)continuous: and is the unique solution to the flow function for t  [0,  ];  (t)  inv(z k ) but  (t)  g((z k,  )) for t <  (ii)transition:(z k, z k+1 )  ,  (  )  g((z k, z‘)), and x k+1 = r((z k, z k+1 ),  (  ))  inv(z k+1 ) Note: all feasible runs      are deterministic. z2z2 z3z3 x2x2 x1x1 transition x(t)x(t)

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 7 Problem Statement Target region: (z tar, x tar )  tar  , with one z tar  Z, x tar  inv(z tar ) Forbidden sets: with F j  , continuous sets polyhedral Assume:time set T = {t 0, t 1,..., t f } is finite Optimal control task: determine such that is the solution to: subject to:        0 = (z 0, x 0 ),  f   tar,   F Chosen cost function  : t f in combination with weighted distances of  k to  tar

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 8 Previously: MILP-based Approach Problems complexity increases highly polynomially with prediction horizon  applicability to larger systems restricted Reasons:  large number of auxiliary variables and constraints required to express the transition dynamics algebraically  relaxations between different dynamics often inefficient (e.g. ) Characteristics:  point-wise linearized hybrid dynamics  reformulation as purely algebraic optimization problem  solution by mixed-integer linear programming (MILP)  applied iteratively in a moving horizon scheme

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 9 Principle:  separate the optimization of continuous and discrete degrees of freedom: (i) high level: search tree encoding the discrete DOF  v(t) (ii) low level: embedded NLP for the continuous DOF  u(t)  branch&bound and heuristics to prune the search tree efficiently  cost function evaluated by hybrid simulation Decomposition Approach Hybrid Automaton HA Specification:  0,  tar,  Graph Search Algorithm Embedded Nonlinear Programming Hybrid Simulation Neighborhood info  u,  v,   node n, v k Prediction horizon p

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 10 Graph Search (1)  acyclic graph encoding the possible -trajectories (finite length)  node: with:c a – accumulated costs up to  k c p – predicted costs for – priority for selection  ‘shortest path’ search: costs too high compared to best solution

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 11 Graph Search (2)  Search strategy:(1.) best-first / depth-first until 1 st solution is found (2.) breadth-first (or continue with best-first) Selection criterion:priorities (combine c a and c p )  Pruning:  node does not belong to a feasible run  hybrid state within F j  costs indicated that no optimal solution ( )  target reached  hybrid state in a neighborhood of another (and lower priority)

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 12 Adjacency and Similarity Observation: usually many ‘similar’ are investigated Similarity:for two different input trajectories, and contain inter- mediate states  i and  i ’ with z i = z i ’ and   i and  i ’ are called adjacent [assumption: remaining optimal paths incur the same costs] Priorities:if  i and  i ’ are adjacent, and for accumulated costs c a, c a ’ and predicted costs c p, c p ’, the priorities are:  (  ’) >  (  ) iff: (1.) c a (  ’) < c a (  ) (2.)c p (  ’) < c p (  )

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 13 Embedded Nonlinear Optimization Time set for the prediction:T p = {t k, …, t k+p }  T Optimization:  for : each component of v is relaxed to its range of values  solution by nonlinear programming (NLP) Evaluation of the cost function: hybrid simulation of HA for the input trajectories (involves evaluating the continuous dynamics, detecting the guard satisfaction, and executing the resets)  Result of the embedded optimization:  continuous input u k  accumulated costs c A (from  0 to  k+1 )  predicted cost c p (underestimation  lower bound)

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 14 Algorithm

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 15 Example - Description Discrete dynamics: M F1F1 F2F2 sHsH FCFC F3F3 V, T, cA, cBV, T, cA, cB Variables:  discrete inputs: F 1, F 2, s H  continuous inputs: F C, F 3  state variables: V, T, c A, c B Continuous dynamics: low level high level V  0.8 V  0.8 only for “high level” Tank reactor with 2 nd order reaction

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 16 Example – Results (1) Objectives:  reach nominal reaction (target) from an initially empty reactor  time optimality  avoid overflow and critical temperatures. Configurations:  select: best-first search (throughout)  determinebest: pruning based on adjacency after 1 st solution is found  prediction horizon: p = 2 Results:  termination after 959 nodes,  721 nodes fathomed due to adjacency, the remainder due to costs [theoretical number of nodes for the encountered path length: 3  ]  computation time: 484 CPU-sec (P4-1.5 GHz) (approx. one order of magnitude smaller than for the MILP solution)

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 17 Example – Results (2) Projection into the (V R, T R, c A ) space:  red: fathomed  blue: not fathomed  green: target, best found solution Corresponding trajectory

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 18 Conclusions Separation of continuous and discrete DOFs:  evaluates the original hybrid dynamics (not linearized models)  no algebraic encoding of the transition dynamics required  tree search only for true degrees of freedom (not for discrete auxiliary variables)  no relaxations between different continuous dynamics  unlike MPC: not all but the best solution discarded in each iteration Adjacency:  avoids exploration of almost identical evolutions  used for pruning or only for sorting the list of live nodes  result is in general only suboptimal

Olaf Stursberg Graph Search Algorithm for Optimal Control of Hybrid Systems 19 Current Work  improve state space coverage: neighborhoods determined by the progress in X  determine suitable choices for the diameters of neighborhoods  evaluate the performance for more complex autonomous discrete dynamics (includes costs for resets)