1. 1 Feedback control theory: An overview and connections to biochemical systems theory Sigurd Skogestad Department of Chemical Engineering Norwegian University of Science and Tecnology (NTNU) Trondheim, Norway VIIth International Symposium on Biochemical Systems Theory Averøy, Norway, 17-20 June 2002

2. 2 Motivation I have co-authored a book: ”Multivariable feedback control – analysis and design” (Wiley, 1996) –What parts could be useful for systems biochemistry? Control as a field is closely related to systems theory –The more general systems theory concepts are assumed known Here: Focus on the use of negative feedback Some other areas where control may contribute (Not covered): –Identification of dynamic models from data (not in my book anyway) –Model reduction –Nonlinear control (also not in my book)

3. 3 Outline 1.Introduction: Negative feedforward and feedback control 2.Introductory examples –Feedback is an extremely powerful tool (BUT: So simple that it is frequently overlooked) 3.Control theory and possible contributions 4.Fundamental limitation on negative feedback control 5.Cascade control and control of complex large-scale engineering system Hierarchy (cascades) of single-input-single-output (SISO) control loops 6.Design of hierarchical control systems Overall operational objectives Which variable to control (primary output) ? Self-optimizing control 7.Summary and concluding remarks

4. 4 Important control concepts Cause-effect relationship Classification of variables: –”Causes”: Disturbances (d) and inputs (u) –”Effects”: Internal states (x) and outputs (y) Typical state-space models: Linearized models (useful for control!):

5. 5 Typical chemical plant: Tennessee Eastman process Recycle and natural phenomena give positive feedback

6. 6 Control uses negative feedback XC x As xAxA FAFA

7. 7 Control Active adjustment of inputs (available degrees of freedom, u) to achieve the operational objectives of the system Most cases: Acceptable operation = ”Output (y) close to desired setpoint (y s )”

8. 8 Plant (uncontrolled system) Disturbance (d) Input (u) Output (y) Acceptable operation = ”Output (y) close to desired setpoint (y s )” Control: Use input (u) to counteract effect of disturbance (d) on y Two main principles: Feedforward control (measure d, predict and correct ahead) (Negative) Feedback control (measure y and correct afterwards)

9. 9 Plant (uncontrolled system) Disturbance (d) Input (u) Output (y) No control: Output (y) drifts away from setpoint (y s )

10. 10 Plant (uncontrolled system) Disturbance (d) Input (u) Output (y) Feedforward control: Measure d, predict and correct (ahead) Main problem: Offset due to model error FF-Controller≈ Plant model -1 Setpoint (y s ) Predict Offset

11. 11 Plant (uncontrolled system) Disturbance (d) Input (u) Output (y) FB Controller ≈ High gain Setpoint (y s ) Feedback control: Measure y, compare and correct (afterwards) Main problem: Potential instability e

12. 12 Outline 1.Introduction: Feedforward and feedback control 2.Introductory examples –(Negative) Feedback is an extremely powerful tool (BUT: So simple that it is frequently overlooked) 3.Control theory and possible contributions 4.Fundamental limitation on control 5.Cascade control and control of complex large-scale engineering system Hierarchy (cascades) of single-input-single-output (SISO) control loops 6.Design of hierarchical control systems Overall operational objectives Which variable to control (primary output) ? Self-optimizing control 7.Summary and concluding remarks

13. 13 Example G GdGd u d y Plant (uncontrolled system) 1 k=10 time 25

14. 14 G GdGd u d y

15. 15 Feedforward (FF) control G GdGd u d y Nominal G=G d → Use u = -d

16. 16 G GdGd u d y FF control: Nominal case (perfect model)

17. 17 G GdGd u d y FF control: change in gain in G

18. 18 G GdGd u d y FF control: change in time constant

19. 19 G GdGd u d y FF control: simultaneous change in gain and time constant

20. 20 G GdGd u d y FF control: change in time delay

21. 21 Feedback (FB) control G GdGd u d y Feedback controller ysys e=y s -y Negative feedback: u=f(e) ”Counteract error in y by change in u’’

22. 22 Feedback (FB) control Feedback controller e=y s -y u Simplest: On/off-controller u varies between u min (off) and u max (on) Problem: Continous cycling

23. 23 Feedback (FB) control Feedback controller e=y s -y u Most common in industrial systems: PI-controller

24. 24 G GdGd u d y Back to the example

25. 25 G GdGd u d y C ysys e Feedback PI-control: Nominal case Input u Output y

26. 26 G GdGd u d y C ysys e Integral (I) action removes offset offset

27. 27 G GdGd u d y C ysys e Feedback PI control: change in gain

28. 28 FB control: change in time constant G GdGd u d y C ysys e

29. 29 FB control: simultaneous change in gain and time constant G GdGd u d y C ysys e

30. 30 FB control: change in time delay G GdGd u d y C ysys e

31. 31 FB control: all cases G GdGd u d y C ysys e

32. 32 G GdGd u d y FF control: all cases

33. 33 Summary example Feedforward control is NOT ROBUST (it is sensitive to plant changes, e.g. in gain and time constant) Feedforward control: gradual performance degradation Feedback control is ROBUST (it is insensitive to plant changes, e.g. in gain and time constant) Feedback control: sudden performance degradation (instability) Instability occurs if we over-react (loop gain is too large compared to effective time delay). Feedback control: Changes system dynamics (eigenvalues) Example was for single input - single output (SISO) case Differences may be more striking in multivariable (MIMO) case

34. 34 Feedback is an amazingly powerful tool

35. 35 Stabilization requires feedback Input u Output y

36. 36 Why feedback? (and not feedforward control) Counteract unmeasured disturbances Reduce effect of changes / uncertainty (robustness) Change system dynamics (including stabilization) No explicit model required MAIN PROBLEM Potential instability (may occur suddenly)

37. 37 Outline 1.Introduction: Feedforward and feedback control 2.Introductory examples –Feedback is an extremely powerful tool (BUT: So simple that it is frequently overlooked) 3.Control theory and possible contributions 4.Fundamental limitation on control 5.Cascade control and control of complex large-scale engineering system Hierarchy (cascades) of single-input-single-output (SISO) control loops 6.Design of hierarchical control systems Overall operational objectives Which variable to control (primary output) ? Self-optimizing control 7.Summary and concluding remarks

38. 38 Overview of Control theory Classical feedback control (1930-1960) (Bode): –Single-loop (SISO) feedback control –Transfer functions, Frequency analysis (Bode-plot) –Fundamental feedback limitations (waterbed). Focus on robustness Optimal control (1960-1980) (Kalman): –Optimal design of Multivariable (MIMO) controllers –Model-based ”feedforward” thinking; no robustness guarantees (LQG) –State-space; Advanced mathematical tools (LQG) Robust control (1980-2000) (Zames, Doyle) –Combine classical and optimal control –Optimal design of controllers with guaranteed robustness (H ∞ ) Nonlinear control (1950 - ) – ”Feedforward thinking”, Mechanical systems Adaptive control (1970-1985) (Åstrøm)

39. 39 Control theory Design

40. 40 Relationship to system biochemistry/biology: What can the control field contribute? Advanced methods for model-based centralized controller design –Probably of minor interest in biological systems –Unlikely that nature has developed many multivariable control solutions Understanding of feedback systems –Same inherent limitations apply in biological systems Understanding and design of hierarchical control systems –Important both in engineering and biological systems –BUT: Underdeveloped area in control ”Large scale systems community”: Out of touch with reality

41. 41 Outline 1.Introduction: Feedforward and feedback control 2.Introductory examples –Feedback is an extremely powerful tool (BUT: So simple that it is frequently overlooked) 3.Control theory and possible contributions 4.Fundamental limitation on control 5.Cascade control and control of complex large-scale engineering system Hierarchy (cascades) of single-input-single-output (SISO) control loops 6.Design of hierarchical control systems Overall operational objectives Which variable to control (primary output) ? Self-optimizing control 7.Summary and concluding remarks

42. 42 Inherent limitations Simple measure: Effective delay θ eff Fundamental waterbed limitation (”no free lunch”) for second- or higher-order system: Does NOT apply to first-order system

43. 43 Inherent limitations in plant (underlying uncontrolled system) Effective delay: Includes inverse response, high-order dynamics Multivariable systems: RHP-zeros (unstable inverse) – generalization of inverse response Unstable plant. Not a problem in itself, but –Requires the active use of plant inputs –Requires that we can react sufficiently fast ”Large” disturbances are a problem when combined with –Large effective delay: Cannot react sufficiently fast –Instability: Inputs may saturate and system goes unstable All these may be quantified: For exampe, see my book

44. 44 Outline 1.Introduction: Feedforward and feedback control 2.Introductory examples –Feedback is an extremely powerful tool (BUT: So simple that it is frequently overlooked) 3.Control theory and possible contributions 4.Fundamental limitation on control 5.Cascade control and control of complex large-scale engineering systems Hierarchy (cascades) of single-input-single-output (SISO) control loops 6.Design of hierarchical control systems Overall operational objectives Which variable to control (primary output) ? Self-optimizing control 7.Summary and concluding remarks

45. 45 Problem feedback: Effective delay θ Effective delay PI-control = ”original delay” + ”inverse response” + ”half of second time constant” + ”all smaller time constants”

46. 46 PI-control G1G1 u d y C ysys e G2G2

47. 47 Improve control? Some improvement possible with more complex controller –For example, add derivative action (PID-controller) –May reduce θ eff from 5 s to 2 s –Problem: Sensitive to measurement noise –Does not remove the fundamental limitation (recall waterbed) Add extra measurement and introduce local control –May remove the fundamental waterbed limitation Waterbed limitation does not apply to first-order system –Cascade

48. 48 Cascade control w/ extra meas. (2 PI’s) G1G1 u d y C1C1 ysys G2G2 C2C2 y2y2 Without cascade With cascade y 2s

49. 49 Cascade control Inner fast (secondary) loop: –P or PI-control –Local disturbance rejection –Much smaller effective delay (0.2 s) Outer slower primary loop: –Reduced effective delay (2 s) No loss in degrees of freedom –Setpoint in inner loop new degree of freedom Time scale separation –Inner loop can be modelled as gain=1 + effective delay Very effective for control of large-scale systems

50. 50 Control configuration with two layers of cascade control y 1 - primary output (with given setpoint = reference value r 1 ) y 2 - secondary output (extra measurement) u 3 - main input (slow) u 2 - Extra input for fast control (temporary – reset to nominal value r 3 ) More complex cascades

51. 51 Hierarchical structure in chemical industry

52. 52 Engineering systems Most (all?) large-scale engineering systems are controlled using hierarchies of quite simple single-loop controllers –Commercial aircraft –Large-scale chemical plant (refinery) 1000’s of loops Simple components: on-off + P-control + PI-control + nonlinear fixes + some feedforward Same in biological systems

53. 53 Outline 1.Introduction: Feedforward and feedback control 2.Introductory examples –Feedback is an extremely powerful tool (BUT: So simple that it is frequently overlooked) 3.Control theory and possible contributions 4.Fundamental limitation on control 5.Cascade control and control of complex large-scale engineering system Hierarchy (cascades) of single-input-single-output (SISO) control loops 6.Design of hierarchical control systems Overall operational objectives Which variable to control (primary output) ? Self-optimizing control 7.Summary and concluding remarks

54. 54 Hierarchical structure Brain Local control in cells Organs

55. 55 Alan Foss (“Critique of chemical process control theory”, AIChE Journal,1973): The central issue to be resolved... is the determination of control system structure. Which variables should be measured, which inputs should be manipulated and which links should be made between the two sets?

56. 56 Alternatives structures for optimizing control What should we control? Hierarchical Centralized Brain Cells

57. 57 Alternatives structures for optimizing control Hierarchical Centralized What should we control? (Control theory has little to offer) Control theory has a lot to offer

58. 58 WHAT SHOULD WE CONTROL? Example: 10 km Run Overall objective: Minimum time No major disturbances What should we control? –constant speed? easy to measure with clock. –constant heart beat? –constant level of sugar? –Constant level of lactic acid? Example: 10 km cross-country skiing Overall objective: minimum time Disturbance = hill. What should we control? –Constant speed no longer optimal. – Could have a mix depending on disturbance (constant feed when flat, lactic acid in hill?, max speed downhill turn) Example: Cell Overall objective = optimize cell growth? What should we control? – constant oxygen?

59. 59 Self-optimizing control (Skogestad, 2000) Self-optimizing control is achieved when a constant setpoint policy results in an acceptable loss L (without the need to reoptimize when disturbances occur) Loss L = J - J opt (d) J = cost (overall objective to be minimized)

60. 60 Good candidate controlled variables c (for self-optimizing control) Requirements: The optimal value of c should be insensitive to disturbances c should be easy to measure and control The value of c should be sensitive to changes in the degrees of freedom (Equivalently, J as a function of c should be flat) For cases with more than one unconstrained degrees of freedom, the selected controlled variables should be independent. Singular value rule (Skogestad and Postlethwaite, 1996): Look for variables that maximize the minimum singular value of the appropriately scaled steady-state gain matrix G from u to c

61. 61 Stepwise procedure for design of control system in chemical plant I. TOP-DOWN Step 1. DEFINE OVERALL CONTROL OBJECTIVE Step 2. DEGREE OF FREEDOM ANALYSIS Step 3. WHAT TO CONTROL? (primary outputs) control active constraints unconstrained: “self-optimizing variables” Mainly economic considerations: Little control knowledge required! Stepwise procedure chemical plant

62. 62 II. BOTTOM-UP (structure control system): Step 4. REGULATORY CONTROL LAYER 5.1Stabilization 5.2Local disturbance rejection (inner cascades) ISSUE: What more to control? (secondary variables) Step 5. SUPERVISORY CONTROL LAYER Decentralized or multivariable control (MPC)? Pairing? Step 6. OPTIMIZATION LAYER (RTO) Stepwise procedure chemical plant

63. 63 Step 1. Overall control objective What are the operational objectives? Quantify: Minimize scalar cost J Usually J = economic cost [$/h] + Constraints on flows, equipment constraints, product specifications, etc. Stepwise procedure chemical plant

64. 64 Step 2. Degree of freedom (DOF) analysis N m : no. of dynamic (control) DOFs (valves) Stepwise procedure chemical plant

65. 65 Step 3. What should we control? (primary controlled variables) Intuition: “Dominant variables” Systematic: Define cost J and minimize w.r.t. DOFs –Control active constraints (constant setpoint is optimal) –Remaining DOFs: Control variables c for which constant setpoints give small (economic) loss Loss = J - J opt(d) when disturbances d occurs Stepwise procedure chemical plant

66. 66 Application: Recycle process J = V (minimize energy) N m = 5 3 economic DOFs 1 2 3 4 5 Given feedrate F 0 and column pressure: Constraints: Mr < Mrmax, xB > xBmin = 0.98 Stepwise procedure chemical plant

67. 67 Recycle process: Loss with constant setpoint, c s Large loss with c = F (Luyben rule) Negligible loss with c =L/F or c = temperature Stepwise procedure chemical plant

68. 68 Recycle process: Proposed control structure for case with J = V (minimize energy) Active constraint M r = M rmax Active constraint x B = x Bmin Stepwise procedure chemical plant

69. 69 Effect of implementation error on cost Stepwise procedure chemical plant

70. 70 II. Bottom-up assignment of loops in control layer Identify secondary (extra) controlled variable Determine structure (configuration) of control system (pairing) A good control configuration is insensitive to parameter changes! Industry: most common approach is to copy old designs Stepwise procedure chemical plant

71. 71 Step 4. Regulatory control layer Purpose: “Stabilize” the plant using local SISO PID controllers to enable manual operation (by operators) Main structural issues: What more should we control? (secondary cv’s, y 2 ) Pairing with manipulated variables (mv’s) y 1 = c y 2 = ? Stepwise procedure chemical plant

72. 72 Selection of secondary controlled variables (y 2 ) The variable is easy to measure and control For stabilization: Unstable mode is “quickly” detected in the measurement (Tool: pole vector analysis) For local disturbance rejection: The variable is located “close” to an important disturbance (Tool: partial control analysis). Stepwise procedure chemical plant

73. 73 Summary Procedure plantwide control: I. Top-down analysis to identify degrees of freedom and primary controlled variables (look for self-optimizing variables) II. Bottom-up analysis to determine secondary controlled variables and structure of control system (pairing). Stepwise procedure chemical plant Skogestad, S. (2000), “Plantwide control -towards a systematic procedure”, Proc. ESCAPE’12 Symposium, Haag, Netherlands, May 2002. Larsson, T. and S. Skogestad, 2000, “Plantwide control: A review and a new design procedure”, Modeling, Identification and Control, 21, 209-240. Skogestad, S. (2000). “Plantwide control: The search for the self-optimizing control structure”. J. Proc. Control 10, 487- 507. See also the home page of Sigurd Skogestad: http://www.chembio.ntnu.no/users/skoge/

74. 74 Biological systems ”Self-optimizing” controlled variables have presumably been found by natural selection Need to do ”reverse engineering” : –Find the controlled variables used in nature –From this identify what overall objective the biological system has been attempting to optimize

75. 75 Conclusion Negative Feedback is an extremely powerful tool Complex systems can be controlled by hierarchies (cascades) of single- input-single-output (SISO) control loops Control extra local variables (secondary outputs) to avoid fundamental feedback control limitations Control the right variables (primary outputs) to achieve ”self- optimizing control”

76. 76 Outline 1.Introduction: Feedforward and feedback control 2.Introductory examples –Feedback is an extremely powerful tool (BUT: So simple that it is frequently overlooked) 3.Control theory and possible contributions 4.Fundamental limitation on control 5.Cascade control and control of complex large-scale engineering system Hierarchy (cascades) of single-input-single-output (SISO) control loops 6.Design of hierarchical control systems Overall operational objectives Which variable to control (primary output) ? Self-optimizing control 7.Summary and concluding remarks

77. 77 Paper by Doyle (special issue of Science on Systems Biology, March 2002) SUMMARY Robustness Speculation: Most of the supposedly important genes are related to control –Compare with commercial airplane or chemical plant HOT: mechanism for power laws that challenges the self-optimized- criticality and edge-of-chaos concepts (Santa Fe Institute)