1. Timed Model-based Programming: Executable Specifications for Robust Mission-Critical Sequences Michel Ingham, Seung Chung, Paul Elliott, Oliver Martin, Tazeen Mahtab, Greg Sullivan and Brian Williams Model-based Embedded Robotic Systems Group Space Systems Laboratory Massachusetts Institute of Technology

2. Objectives & Outline Timed Model-based Execution “in a nutshell” Timed Model-based Programming: a visual programming paradigm Illustration of Timed Model-based Execution Executive implementation: Timed Control Sequencer Executive implementation: Timed Mode Estimation Executive implementation: Symbolic Reactive Planner

3. Mission-critical sequences: Launch & deployment Planetary fly-by Orbital insertion Entry, descent & landing Motivation images courtesy of NASA

4. Problem Statement Traditional programming can lead to “brittle” sequences:  complexity of control specification  complexity of plant interactions  lack of robustness (management of off-nominal behavior) Time is central to the execution of mission-critical sequences:  plant spec: component behavior includes latency and evolution  control spec: hard-coded delays in sequence capture state knowledge Robust executive must consider time in its control and behavior models, in addition to reactively managing complexity Approach: Timed Model-based Programming

5. Timed Model-based Programming Approach for graphically encoding and executing robust mission-critical spacecraft sequences. Addresses issues of sequence complexity and low-level system interactions Adopts timed, state-based control specifications Reasons through probabilistic, timed models of nominal and off-nominal plant behavior.

6. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

7. Descent engine to “standby”: off heating 30-60 sec standby Mars Entry Example engine to standby planetary approach separate lander switch to inertial nav rotate to entry-orient & hold attitude

8. Mars Entry Example engine to standby Spacecraft approach: 270 mins delay relative position wrt Mars not observable based on ground computations of cruise trajectory planetary approach separate lander switch to inertial nav rotate to entry-orient & hold attitude

9. Mars Entry Example engine to standby planetary approach separate lander switch to inertial nav rotate to entry-orient & hold attitude Switch navigation mode: “Earth-relative” = Star Tracker + IMU Switch navigation mode: “Inertial” = IMU only

10. Mars Entry Example engine to standby Rotate spacecraft: command ACS to entry orientation planetary approach separate lander switch to inertial nav rotate to entry-orient & hold attitude

11. Mars Entry Example engine to standby Rotate spacecraft: once entry orientation achieved, ACS holds attitude planetary approach separate lander switch to inertial nav rotate to entry-orient & hold attitude

12. Mars Entry Example engine to standby Separate lander from cruise stage: planetary approach separate lander switch to inertial nav rotate to entry-orient & hold attitude cruise stage lander stage pyro latches

13. Mars Entry Example engine to standby planetary approach separate lander switch to inertial nav rotate to entry-orient & hold attitude Separate lander from cruise stage: when entry orientation achieved, fire primary pyro latch cruise stage lander stage pyro latches

14. Mars Entry Example engine to standby planetary approach separate lander switch to inertial nav rotate to entry-orient & hold attitude Separate lander from cruise stage: when entry orientation achieved, fire primary pyro latch cruise stage lander stage

15. Mars Entry Example engine to standby planetary approach separate lander switch to inertial nav rotate to entry-orient & hold attitude Separate lander from cruise stage: in case of failure of primary latch, fire backup pyro latch cruise stage lander stage

16. Mars Entry Example engine to standby planetary approach separate lander switch to inertial nav rotate to entry-orient & hold attitude Separate lander from cruise stage: in case of failure of primary latch, fire backup pyro latch cruise stage lander stage

17. Key Features of Executive engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander simple state-based control specifications models are writable/inspectable by systems engineers handle timed plant & control behavior automated reasoning through low- level plant interactions fault-aware (in-the-loop recoveries)

18. TMBP for Mars Science Lab Technology Demo 2005 MSL Mission (2009) courtesy NASA JPL

19. Related Work Model-based Programming Timed Formal Modeling TMBP Williams, Ingham, Chung & Elliott, ‘03 Timed Control Programs, Timed Plant Models, Semi-Markov Semantics RMPL and Control Sequencer Alur & Dill, ‘94 Henzinger, Manna & Pnueli, ‘92 Kwiatkowska, et al., ‘00 Largouet & Cordier, ‘00 State-based Specifications Goal-driven Execution Constraint Programming Synchronous Programming Model-based Execution deKleer & Williams, ‘87-‘89 Williams & Nayak, ‘96-’97 Kurien & Nayak, ‘00 Deductive Estimation & Control Saraswat, Jagadeesan & Gupta, ‘94 Constraint Modeling Closed-loop Control Firby, ‘89; Simmons, ‘98; Gat, ‘96 Harel, ‘87; Kesten & Pnueli, ‘92 Berry & Gonthier, ‘92 Halbwachs, ‘93 Visual Representations Embedded Programming Constructs Mission Data System Dvorak, Rasmussen, et al., ‘00 Non-deterministic Timed Transitions

20. Objectives & Outline Timed Model-based Execution “in a nutshell” Timed Model-based Programming: a visual programming paradigm Illustration of Timed Model-based Execution Executive implementation: Timed Control Sequencer Executive implementation: Timed Mode Estimation Executive implementation: Symbolic Reactive Planner

21. Timed Model-based Program

23. Timed Hierarchical Constraint Automata graphical specification language for control programs in spirit of Timed StateCharts (Kesten & Pnueli, Harel) writable, inspectable by systems engineers composite locations primitive locations  compact encoding: multiple locations can be simultaneously marked

24. Timed Hierarchical Constraint Automata graphical specification language for control programs in spirit of Timed StateCharts (Kesten & Pnueli, Harel) writable, inspectable by systems engineers goal constraint (hidden state) clock initialization  act on hidden state  clocks provide timing mechanism

25. Timed Hierarchical Constraint Automata graphical specification language for control programs in spirit of Timed StateCharts (Kesten & Pnueli, Harel) writable, inspectable by systems engineers  conditioned on time & state constraints transition guard maintenance constraint

26. Timed Hierarchical Constraint Automata graphical specification language for control programs in spirit of Timed StateCharts (Kesten & Pnueli, Harel) writable, inspectable by systems engineers sequential parallel iteration preemption

27. Timed Model-based Program

28. Timed Concurrent Constraint Automata variant of factored POSMDP (time continuous, but observations and decisions at discrete points) constraints guarded & timed probabilistic transitions nominal modes fault modes p  (t) t 0.1 0.2 P  = 99.9% modal rewards

29. Objectives & Outline Timed Model-based Execution “in a nutshell” Timed Model-based Programming: a visual programming paradigm Illustration of Timed Model-based Execution Executive implementation: Timed Control Sequencer Executive implementation: Timed Mode Estimation Executive implementation: Symbolic Reactive Planner

30. Timed Model-based Executive Architecture

31. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

32. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander Control Sequencer executes THCA

33. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander Deductive Controller provides state estimates and command sequences that achieve goals t 30 60 p  (t) obs: goal: Standby

34. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

35. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

36. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

37. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

38. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

39. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

40. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

41. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

42. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

43. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

44. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander Model-based executive provides robustness in the goal-driven control loop obs: goal: Separated primary pyro misfired! backup pyro fired

45. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

46. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

47. Mars Entry Example engine to standby planetary approach switch to inertial nav rotate to entry-orient & hold attitude separate lander

48. Full Demonstration Proof-of-concept on a representative mission scenario: “Full” Entry, Descent and Landing scenario Control program (57 locations, 16 state vars, 6 clock vars) Plant model (~25 components, avg. 3-4 modes per component)

49. Demonstration: Highlights Key Capabilities Nominal operations: –execution conditioned on state constraints –execution conditioned on time constraints –nominal mode tracking through commanded and timed transitions –accept configuration goal and generate appropriate command sequence (single-step, multi-step reconfigurations) Operations in the presence of faults: –fault diagnosis through commanded transitions –fault diagnosis through timed transitions –recovery by repair (deductive controller) –recovery by leveraging physical/functional redundancy (control sequencer)

50. TMBP for Mars Science Lab Technology Demo 2005 MSL Mission (2009) courtesy NASA JPL

51. Objectives & Outline Timed Model-based Execution “in a nutshell” Timed Model-based Programming: a visual programming paradigm Illustration of Timed Model-based Execution Execution semantics Executive implementation: Timed Control Sequencer Executive implementation: Timed Mode Estimation Future directions and contributions

52. TMBP Semantics Plant model: –dense time, discrete states –variables –factored POSMDP full assignments  over all vars in  transitions upon transition, subset of clocks are reset nominal transition initial state probability transition probability observation probability state reward s  o

53. TMBP Semantics Control program: –program locations –clocks –deterministic automaton initial program location transitions between locations, conditioned on last & current state, current clock values config. goal clock init. assignments  to all clocks in

54. TMBP Semantics Interleaving model of execution cycle = discrete event + continuous phase Legal execution of TMBP: Cycle start time… Plant state Pgm location Pgm clocks  start with valid

55. TMBP Semantics Interleaving model of execution cycle = discrete event + continuous phase Legal execution of TMBP: Cycle start time… Plant state Pgm location Pgm clocks  consistent with  in nominal case, is max-reward state in or is prefix of loop-free plant trajectory leading to max-reward state in 

56. TMBP Semantics Interleaving model of execution cycle = discrete event + continuous phase Legal execution of TMBP: Cycle start time… Plant state Pgm location Pgm clocks  is a legal clock value sequence, i.e. for each active clock in,

57. TMBP Semantics Interleaving model of execution cycle = discrete event + continuous phase Legal execution of TMBP: Cycle start time… Plant state… Pgm location… Pgm clocks…  and so on…

58. Objectives & Outline Timed Model-based Execution “in a nutshell” Timed Model-based Programming: a visual programming paradigm Illustration of Timed Model-based Execution Executive implementation: Timed Control Sequencer Executive implementation: Timed Mode Estimation Executive implementation: Symbolic Reactive Planner

59. Control Sequencer Implementation

60. THCA Execution Algorithm 1.update active clocks 2.check maintenance constraints 3.assert clock initializations & state goals 4.request MR to take action 5.obtain new state estimate from ME 6.await incomplete goals 7.take enabled transitions 8.mark new set of locations 9.return to step 2 reactive preemption clock persistence goal-driven executionclosed-loop execution progress due to goal achievement or preemption

61. Objectives & Outline Timed Model-based Execution “in a nutshell” Timed Model-based Programming: a visual programming paradigm Illustration of Timed Model-based Execution Executive implementation: Timed Control Sequencer Executive implementation: Timed Mode Estimation Executive implementation: Symbolic Reactive Planner

62. Deductive Controller Implementation

63. Mode Estimation Given latest commands and observations, what is most likely current state? Belief state update to estimate state for POMDPs: state: set of mode assignments for all components state transition: one mode transition for each component

64. Mode Estimation Full belief state update computationally infeasible –state space too large Need to explore in best-first order –formulate OCSP, to identify likely extensions to current trajectories (“shortest path” problem) –solve with OPSAT – Clause-directed A* 1 OM speedupv– Ragno SM best-first search best-first search most-likelycandidateoptimalfeasiblemodes conflicts (infeasible modes) consistent with model & obs? consistent with model & obs? conflict database conflict database

65. Timed Mode Estimation For physical plants modeled as TCCA (POSMDP): define “system state” = plant states + plant clocks Good news: can leverage existing OPSAT engine! Bad news: state space gets much larger…

66. TCCA Mode Estimation Algorithm Given state s (i), control action  (i), observation o (i+1) & current time t abs : 1.update clock t j for each component 2.setup OCSP : decision vars x, such that dom[x j ] = reachable target modes objective function f(x) = prior prob of state x, constraint C(x), such that is consistent 3.compute most likely solution using OPSAT 4.for any component whose mode has changed, reinitialize clock to zero

67. Objectives & Outline Timed Model-based Execution “in a nutshell” Timed Model-based Programming: a visual programming paradigm Illustration of Timed Model-based Execution Executive implementation: Timed Control Sequencer Executive implementation: Timed Mode Estimation Executive implementation: Symbolic Reactive Planner

68. Directions for Future Work Formal Verification (Model Checking) for Timed Plant Models, Timed Control Programs Extension to Hybrid Model-based Programming –control programs can specify trajectories in terms of continuous and/or discrete states –fold continuous estimators & controllers into Deductive Controller Compiled Deductive Controller Managing Uncertainty Online –Prediction of Future Unsafe States –Reactive Planning Under Uncertainty –Active Probing to Expose Failures

69. Eliminate Structural Decomposition Compilation Out1 Out2 Out3 In1 In2 S1 S2 S1S2 Abstraction Projected Prime Implicates In1 Out1 Out3 In2 S1 S2 CSP Smaller sub- problems

70. Verification of RMPL Model-based Programs Goal: To extract probabilistic information that gives confidence in the likelihood of program’s successful execution. Problem: Given the high-level goal specification of an RMPL control program, find the most likely program executions that fail to achieve goal by time step N. Approach: 1. Convert program + goal to OCSP. 2. Solve for most likely failure trajectories using OpSat. 3. Abstract important information from trajectories e.g. group failures that have a common cause. 4. Present as counterexamples to user.

71. State Prediction Predict the likelihood of reaching an unsafe state during the execution of a control program. –Why? Anticipate, i.e. react to the likely future events. –How? Using the reactive planner, compute the expected sequence of commands that achieves the goals specified by the control program. Predict the outcome of executing each command through projection. Issue: –State space explosion problem—memory and time. Approach: –Predict the state of each component individually using the transition dependency graph used by the reactive planner. Hazzard

72. TMBP for Mars Science Lab Technology Demo 2005 MSL Mission (2009) courtesy NASA JPL