1. Motivation Supervisory control of systems –composed of (synchronized) modules –too large to fit in memory/be computed –modules change with time: (dis)appear –where continuous uptime is needed –adaptive control is needed –early warning of problems

2. DES Model The system G is modeled as a (synchronous) composition of modules M j Time is discrete and increments with one after each occurrence of an event The composition can vary with time. G t =M 1t ||M 2t ||…||M nt t  {0,1,…} Modules can appear or disappear

3. System control Control is exercised using the limited lookahead online paradigm “Supervisory control using variable looahead policies”, Chung, Lafortune, Lin, 1994 (“Variable lookahead supervisory control with state information”, Hadj-Alouane, Lafortune, Lin, 1994)

4. Problems Computing the synchronous product of modules Optimizing the control Alternative to marking in the system (accepted language)

5. Dynamic product How to compute M=M 1  M 2  …  M n so that if M i changes, M can be recomputed inexpensively? The operation  is commutative and associative and there is an identity element e=  *, M i  e=M i

6. Stack-like M=(…(M 1  M 2 )  …)  M n When M i changes M 1  …  M i-1 remains intact Older elements are “stacked” first If there are stable components, they will “sink” Not suitable for systems with non-persistent components M1M1 M2M2 M3M3 M4M4 M5M5

7. Tree-like M=(((M 1  M 2 )  (M 3  M 4 ))…)  (…(M n-1  M n )) When M i changes log n number of  is required to recompute M Suitable for systems with any pattern of stability of modules Increased requirements on memory M1M1 M2M2 M3M3 M4M4

8. Stack-tree Using a tree, inner nodes are modules as well M=(…(M 1  M 2  M 3 )  M 4  (M 5  M 6  M 7 ))…)  M i  (…(M n-2  M n-1  M n )…) If top components change, less than log n number of  is required to recompute M New modules go at the top, stable modules “sink” to the base of the tree Not suitable for non-persistent modules Increased complexity, need to use the identity element (Reduced requirements on memory)

9. Stack-tree 1245 36 15 891112 10 13 714 disappers moves down ** added

10. Recomputing M When the composition of modules changes, the lookahead window has to be recomputed Legality of states determined first on substructures to reduce the size of the complete product of the composition Keep information about the Cartesian product of the basic-module states to facilitate the advancement of the horizon

11. Optimization of control Cost function v:  *  R v(s) =   s  K (legal language) v can be computed for example using the monetary value of events in a real system. Expenses will have positive values, revenue will be negative Simplest v is 0 when not  –results in the original limited lookahead control policy

12. Algorithm Simple modification of algorithm in “Super. ctrl using var. lookahead policies” –generate the tree of the possible continuations of a string –var. ctrl with state information should be used

13. Original algorithm if from state come uncontrollable events, recursively compute the cost for each event and assign max value (or until  ) if only controllable event, recursively compute the cost for each one and assign min value (or until 0) if state at border of window, return 0 or  depending on policy –optimistic –conservative

14. Modified algorithm Traversal stops when state is identified as illegal or when it hits the boundary When traversing hits boundary, v(s) is returned –need to keep history (generated s) Unlike original algorithm, traversal will not terminate when a state is identified as legal –for a better approximation of cost –costly when tree depth is very large

15. Examples Trucks fetch material Legal specs: –Only one truck can fetch material at a given time –Exactly 40 units of material are needed c(goS)=5, c(goB)=6, c(fetch10)=10, c(fetch20)=12 All events controllable (for simplicity) goS fetch10 goB fetch10, fetch 20 Small truck (S) Big truck (B)

16. Cost function & compositions v(s)=  if s  legal language v(s)=  c(  i ) -  N*10 else –s=  1  2 …  n ; N=units of material fetched G=S 1 ||S 2 t=0,1 G=S 1 ||B 2 t=2,3 G=B 2 t > 3 In the examples, the cost function isn’t always computed using the full history to keep the examples simple

17. Example 1 (depth=1) Time 0 goS 1 goS 2 v=5 Leave both

18. Example 1 Time 1 (executed goS 1 ) fetch10 1 goS 2 v=  v=-90 Leave fetch10 1

19. Example 1 Time 2 (executed fetch10 1 ) goS 1 goB 2 v=6 v=5 Leave goS 1

20. Example 1 Time 3 (executed goS 1 ) fetch10 1 goB 2 v=  v=-90 Leave fetch10 1

21. Example 1 Time 4 (executed fetch10 1 ) goB 2 v=6 Leave goB 2

22. Example 1 Time 5 (executed goB 2 ) fetch10 2 fetch20 2 v=-188 v=-90 Leave fetch20 2 …Task completed

23. Example 1 Discussion –At time 2 there was a wrong choice in disabling goB 2 instead of goS 1, since big truck can bring more material –This was caused by the limited window size (no satisfactory prediction was possible)

24. Example 2 (depth=2) Time 0 goS 1 goS 2 Leave both goS 2 fetch10 1 goS 1 fetch10 2 v=-85 v=  v=-85 v= 

25. Example 2 Time 1 (executed goS 1 ) goS 2 fetch10 1 v=  Leave fetch10 1 goS 1 goS 2 v=-85

26. Example 2 Time 2 (executed fetch10 1 ) goS 1 goB 2 Leave goB 2 goB 2 fetch10 1 fetch10 2 fetch20 2 v=-182 v=-84 v=-85 v=  goS 1 v= 

27. Example 2 Time 3 (executed goB 2 ) fetch10 2 fetch20 2 v=  Leave fetch20 2 goS 1 goB 2 goS 1 goB 2 v=-182 v=-183 v=-84 v=-85 goS 1

28. Example 2 Time 4 (executed fetch20 2 ) goB 2 Leave goB 2 fetch10 2 fetch20 2 v=  v=-84

29. Example 2 Time 5 (executed goB 2 ) fetch10 2 fetch20 2 Leave fetch10 2 …Task completed goB 2 v=  v=-84

30. Example 2 Discussion –The control is not optimal, since v(goS 1 fetch10 1 goB 2 fetch20 2 goB 2 fetch10 2 ) = v(goS 1 fetch10 1 goS 1 fetch10 1 goB 2 fetch20 2 )+1 –Non-optimality arises because the system varies with time. Sometimes it can also happen that the controller will base the control decisions on a module which later will not be available

31. Example 3 Consider the big truck can carry only 20 units. Thus at time 4 we get a runtime error Theoretically, the system can continue with goB 2 to time 5 and then throw an RTE goB fetch 20 Big truck (B) goB 2 RTE fetch20 2 v= 

32. Example 3 Discussion –Time varying made the system throw an RTE –Let’s relax the requirements and leave only legality rule #1 (only one truck can fetch at a time) –We’ll use the cost function to enforce the requirement for 40 units

33. New cost function v(s)=  if s  legal language v(s)=  c(  i ) -  N*10 if N  40 v(s)=  c(  i ) +  N*10 - 400 else –s=  1  2 …  n ; N=units of material fetched

34. Example 4 At time 4, there is only a big truck available while only 10 units are required There is no RTE, only the operation is very costly goB 2 fetch20 2 v= 118 << 

35. Example 4 If the big truck can fetch both 10 and 20 units (time 5): fetch10 2 fetch20 2 Leave fetch10 2 …Task completed goB 2 v=118 v=-84

36. Example 4 Discussion –Resulting behavior is “‘legal’ when possible, best effort otherwise” –Using the cost function to enforce requirements is much more flexible and leaves room when the system varies with time –Approach is language-oriented as opposed to state-machine-oriented

37. General discussion The optimality of the solution depends on how well the cost function can approximate the real cost of a string –limited future (lookahead window) –full-depth traversal of subtrees –history Algorithm is flexible and makes best attempts to control system Algorithm can be “tweaked” to consider –costs of disabling –allow threshold control of branching –notification of pending RTE’s –etc.

38. “Goals” instead of marking (preliminary idea) Using marked languages in limited lookahead systems may lead to RTE’s due to wrong control Using a cost function to guide execution can give a “second chance” to the system to recover (similar to rollback in databases) Suitable for systems with continuous uptime (may have “checkpoints”) System might not need to explore full tree depth if it recognizes the accomplishment of a goal