1. April 29, 2008 UC Berkeley EECS, Berkeley, CA 1 Anytime Control Algorithms for Embedded Real-Time Systems L. Greco, D. Fontanelli, A. Bicchi Interdepartmental Research Center “E. Piaggio” University of Pisa

2. April 29, 2008 UC Berkeley EECS, Berkeley, CA 2 Introduction  General tendency in embedded systems: implementation of many concurrent real-time tasks on the same platform  overall HW cost and development time reduction  Highly time-critical control tasks traditionally scheduled with very conservative approaches  rigid, hardly reconfigurable, underperforming architecture  Modern multitasking RTOS (e.g. in automotive ECUs), schedule their tasks dynamically, adapting to varying load conditions and QoS requirements.

3. April 29, 2008 UC Berkeley EECS, Berkeley, CA 3 Introduction  Real-time preemptive algorithms (e.g., RM or EDF) can suspend task execution on higher-priority interrupts  Guarantees of schedulability – based on estimates of Worst-Case Execution Time (WCET) – are obtained at the cost of HW underexploitation: e.g., RM can only guarantee schedulability if less than 70% CPU is utilized  In other terms: for most CPU cycles, a longer time is available than the worst-case guarantee  The problem of Anytime Control is to make good use of that extra time

4. April 29, 2008 UC Berkeley EECS, Berkeley, CA 4  Anytime algorithms and filters…  The execution can be interrupted any time, always producing a valid output;  Increasing the computational time increases the accuracy of the output ( imprecise computation )  Can we apply this to controllers? Anytime Paradigm + + + +

5. April 29, 2008 UC Berkeley EECS, Berkeley, CA 5 Example (I) + + - + + +

6. April 29, 2008 UC Berkeley EECS, Berkeley, CA 6 Example (II) Regulation Problem – RMS comparison Not feasible Conservative: stable but poor performance

7. April 29, 2008 UC Berkeley EECS, Berkeley, CA 7 Example (III) Regulation Problem – RMS comparison Unstable! Greedy: maximum allowed  i

8. April 29, 2008 UC Berkeley EECS, Berkeley, CA 8 Hierarchical Design: controllers must be ordered in a hierarchy of increasing performance; Switched System Performance: stability and performance of the switched system must be addressed; Practicality: implementation of both control and scheduling algorithms must be simple (limited resources); Composability: computation of higher controllers should exploit computations of lower controllers (recommended). Issues in Anytime Control

9. April 29, 2008 UC Berkeley EECS, Berkeley, CA 9 Consider a linear, discrete time, invariant plant and a family of stabilizing feedback controllers Controller i provides better performance than controller j if i > j (but WCET i > WCET j ) Problem Formulation The closed-loop system is

10. April 29, 2008 UC Berkeley EECS, Berkeley, CA 10 Sampling instants: Time allotted to the control task: Worst Case Execution Times: Time map: Scheduler Description

11. April 29, 2008 UC Berkeley EECS, Berkeley, CA 11 A simple stochastic description of the random sequence can be given as an i.i.d. process At time t, the time slot is such that all controllers but no controller can be executed Scheduler Description Stochastic Scheduler as an I.I.D. Process Pr

12. April 29, 2008 UC Berkeley EECS, Berkeley, CA 12 Description Transition probability matrix: Steady state probabilities: More general description with a finite state, discrete-time, homogeneous, irreducible aperiodic Markov chain Scheduler Description Stochastic Scheduler as a Markov Chain

13. April 29, 2008 UC Berkeley EECS, Berkeley, CA 13 m-step (lifted system) Theorem: The MJLS is exponentially AS-stable if and only if such that the m-step condition holds 1-step (average contractivity) [P. Bolzern, P. Colaneri, G.D. Nicolao – CDC ’04] Almost Sure Stability Definition: The MJLS is exponentially AS-stable if such that, and any initial distribution  0, the following condition holds Sufficient conditions

14. April 29, 2008 UC Berkeley EECS, Berkeley, CA 14 Upper bound on the index of the executable controller Controller is computed, unless a preemption event forces Switching Policy Preliminaries and Analysis Switching policy map Examples: Conservative Policy (non-switching, always av.) Greedy Policy (if already AS-stable)

15. April 29, 2008 UC Berkeley EECS, Berkeley, CA 15 Switching Policy Synthesis Problem Formulation Problem: Given and the invariant scheduler distribution, find a switching policy such that the resulting system is a MJLS with invariant probability distribution The computational time allotted by the scheduler cannot be increased; The probability of the i -th controller can be increased only by reducing the probabilities of more complex controllers. How can we build a switching policy ensuring ?

16. April 29, 2008 UC Berkeley EECS, Berkeley, CA 16 Use of an independent, conditioning Markov chain  Same structure (number of states) of the scheduler chain : in the next sampling interval at most the i -th controller is computed (if no preemption occurs) Stochastic Policy How does the conditioning chain interact with the scheduler’s one?

17. April 29, 2008 UC Berkeley EECS, Berkeley, CA 17 Note: the extended chain  has n 2 states Merging Markov Chains Mixing Theorem: Consider two independent finite-state homogeneous irreducible aperiodic Markov chains  and  with state space and respectively. The stochastic process is a finite-state homogeneous irreducible aperiodic Markov chain characterized by

18. April 29, 2008 UC Berkeley EECS, Berkeley, CA 18 The goal is to produce a process with a desired stationary probability with cardinality n After mixing, use an aggregation function derived from the schedulability constraints The i -th controller is executed if and only if: (i.e. limiting controller) (i.e. preemption) (aggregated process) Merging Markov Chains Aggregating

19. April 29, 2008 UC Berkeley EECS, Berkeley, CA 19 Remark: The aggregated process is a linear combination of two chains. Hence: Merging Markov Chains Aggregating (II) Remark: The state evolution of the JLS driven by is the same as the one produced by an equivalent MJLS driven by the Markov chain, constructed associating to the index, hence the controlled system. Therefore:

20. April 29, 2008 UC Berkeley EECS, Berkeley, CA 20 Markov Policy 1-step contractive formulation Anytime Problem – (Linear Programming) Find a vector such that

21. April 29, 2008 UC Berkeley EECS, Berkeley, CA 21 Example (Reprise) + + - + + +

22. April 29, 2008 UC Berkeley EECS, Berkeley, CA 22 Example - Furuta Pendulum Regulation Problem – RMS comparison Markov policy Improvement: > 55%

23. April 29, 2008 UC Berkeley EECS, Berkeley, CA 23 A 1-step contractive solution may not exist, but an m-step solution always exists for some m, since the minimal controller is always executable Look for a solution to the Anytime Problem for increasing m Markov Policy m-step contractive formulation (I) Key Idea: The switching policy supervises the controller choice so that some control patterns are preferred w.r.t. others

24. April 29, 2008 UC Berkeley EECS, Berkeley, CA 24 Lifted Scheduler chain ( n m states) Conditioning chain not lifted ( n m states) : strings of symbols Chain : Mixing: Aggregating: Same as 1-step problem Switching policy: every m steps a bet in advance for an m -string (elementwise minimum) Markov Policy m-step contractive formulation (II)

25. April 29, 2008 UC Berkeley EECS, Berkeley, CA 25 Example (TORA) (I) + + - + + +

26. April 29, 2008 UC Berkeley EECS, Berkeley, CA 26 Example (TORA) (II) Regulation Problem – RMS comparison Not feasible Conservative: stable but poor performance Greedy

27. April 29, 2008 UC Berkeley EECS, Berkeley, CA 27 Example (TORA) (III) Regulation Problem – RMS comparison Markov policy 4-step solution Most likely control pattern:        

28. April 29, 2008 UC Berkeley EECS, Berkeley, CA 28 Tracking and Bumpless  In tracking tasks the performance can be severely impaired by switching between different controllers  The activation of higher level controller abruptly introduces the dynamics of the re-activated (sleeping) states (low-to-high level switching)  The use of bumpless-like techniques can assist in making smoother transitions  Practicality considerations must be taken into account in developing a bumpless transfer method

29. April 29, 2008 UC Berkeley EECS, Berkeley, CA 29 Example (F.P.) (V) Tracking Problem – RMS comparison Not feasible Conservative: stable but poor performance

30. April 29, 2008 UC Berkeley EECS, Berkeley, CA 30 Example (F.P.) (VI) Tracking Problem – Reference & output comparison Markov policy Markov Bumpless policy

31. April 29, 2008 UC Berkeley EECS, Berkeley, CA 31 Example (F.P.) (VII) Tracking Problem – Greedy Policy Unstable! Greedy: maximum allowed  i

32. April 29, 2008 UC Berkeley EECS, Berkeley, CA 32 Example (F.P.) (VIII) Tracking Problem – RMS comparison Markov policy Markov Bumpless policy

33. April 29, 2008 UC Berkeley EECS, Berkeley, CA 33 Example (TORA) (IV) Tracking Problem – RMS comparison Not feasible Conservative: stable but poor performance Greedy

34. April 29, 2008 UC Berkeley EECS, Berkeley, CA 34 Example (TORA) (V) Tracking Problem – Reference & output comparison Markov policy Markov Bumpless policy Greedy

35. April 29, 2008 UC Berkeley EECS, Berkeley, CA 35 Example (TORA) (VI) Tracking Problem – RMS comparison Markov policy Markov Bumpless policy Greedy

36. April 29, 2008 UC Berkeley EECS, Berkeley, CA 36 Performance (not just stability) under switching must be considered for tracking Ongoing work is addressing: –hierarchic design of (composable) controllers for anytime control –numerical aspects of the m-step solution –implementation on real systems Conclusions

37. April 29, 2008 UC Berkeley EECS, Berkeley, CA 37 Anytime Control Algorithms for Embedded Real-Time Systems L. Greco, D. Fontanelli, A. Bicchi Interdepartmental Research Center “E. Piaggio” University of Pisa