1. Aero/Astro Open House MERS Research Group Model-based Embedded and Robotic Systems Group Space Systems Laboratory Massachusetts Institute of Technology Friday, March 30, 2001

2. Motivation Apollo 13 quintuple fault Mars Polar Lander failed due to a faulty sensor. Autonomous systems handle Faults Anomalies Communication Commanding Europa Probe Distant Explorers Mercury Orbiter Cooperative Exploration Mars Outpost Earth Imager

3. Model-based Autonomy Specify common sense, compositional models. Models are reusable and easy to articulate at the concept stage Generative approach covers a broad set of behaviors. Insensitive to design changes Planning Monitoring & DiagnosisReconfigurationReplanning Execution ClosedValveOpen Stuckopen Stuckclosed OpenClose 0. 01 0.01 0.01 inflow = outflow = 0 Programmers generate functions based on common sense hardware and software models. Model-based ProgrammingModel-based Execution Consider as a model-based stochastic, optimal controller Able to perform significant search and deduction within the control loop. Fills in details between common sense models and the environment

4. Self-Repairing Explorers 1. Flow = zero and Pressure 1 = nominal 2. Pressure 2 = nominal 3. Acceleration = zero It can not be the case that all valves are open given the current observations. Determine a mode for each component that explains all observations. Diagnosis is performed in several steps Determine what the conflicting information is given sensor information. In this case, it is that the Flow = zero and the Pressure = nominal. Given this conflicting information, determine a set of components that lead to the given observations, which are the valves in the propulsion system. Then, determine the component modes that explain all observations. In this case, this would be that the highlighted valve is closed instead of open. Gives spacecraft the ability to remain operational in the face of failures.

5. Model-based Autonomous Reactive Systems (MARS) RMPL models unify planning, diagnosis, learning, scheduling and execution representations Planner uses high level goals to drive the operations of the system. Executive will break down the operations from the planner into more specific system commands as well as monitor the system for failures.

6. RMPL models (below) compile to Hierarchical Constraint Automata These automata can be directly executed. Only generates non-destructive plans Operate on reactive time scales Uses precompiled policies to achieve real time performance Model-based Executive (Titan) Mode estimation and reconfiguration rely on propositional logic for fast inference.

7. Model-Based Execution “Tell” constraint: execution layer calls upon the reactive planning layer to take the appropriate steps necessary to achieve the specified hidden state (EngineA = Standby): - Turn on Device Driver A - Issue the “standby-cmd” OrbitInsert():: { (EngineA = Standby), (EngineB = Standby), (Camera = Off), do { when (EngineA = Standby) AND (Camera = Off) donext (EngineA = Firing) } watching (EngineA = Failed), when (EngineA = Failed) donext { when (EngineB = On) AND (Camera = Off) donext (EngineB = Firing) } Write control program by asserting and checking abstract states (which may be “hidden”, i.e. not directly controllable/observable), rather than operating on observable and control variables. Underlying Mode Estimation and Reactive Planning layer takes care of deducing the spacecraft state from the observables, and figuring out how to achieve a specified state. “Ask” constraint: execution layer calls upon the mode estimation layer to deduce the current system state, including diagnosis of transition to (EngineA = Failed): - While (EngineA = Firing), IMU data indicates (Thrust = zero) - ME deduces that most likely current state is (EngineA = Failed). Standby Engine Model Off Failed off-cmd standby-cmd 0.01 (thrust = full) AND (power_in = nominal) Firing 0.01 standby-cmd fire-cmd (thrust = zero) AND (power_in = zero) (thrust = zero) AND (power_in = nominal)

8. Real-time Deduction Off-line Operations c e e dd d _ Model RMPL Representation On-line Operations [ (valve1 = open) and (flow = zero) ] [ (valve2 = open) and not(valve3 = open) and (flow = zero) ] [ (valve2 = stuck closed) and (flow = high) ] Propositions specify conflicts in system. Propositions specifying conflicts Observations valve 1 = open valve 2 = stuck closed

9. MINIature Mode Estimation MESSENGER Software Architecture Mini-ME embodies fast mode estimation using off line operations.

10. Distributing the Model-based Paradigm Code for describing: Concurrent Task Management Resource Management Temporal Constraints Multiple Method Selection Deductive Mode Estimator & Reactive Planner FSC RTC Onboard Sequencer Model Deductive Mode Estimator & Reactive Planner FSC RTC Onboard Sequencer Model Deductive Mode Estimator & Reactive Planner FSC RTC Onboard Sequencer Model Distributed Temporal Planner Distributed Temporal Planner Distributed Temporal Planner High-Level Mission Planning

11. Model-based Planner Input: Planning models specified as RMPL models Processing: Transforms RMPL models into Temporal Plan Network (TPN) representation. Output: Temporally-constrained network of events Model-based Planner Planner Plan Runner Plan (i.e Network of Temporally constrained events) episode.start(params) episode.stop(params) success(episode) fail(episode) RMPL Planning Model Path Planning Input: Temporally-constrained network of events from planner. Episode completion or failure from executive Processing: Trim edges from network Output: Sequence of episode start and stop commands. Start commands: Initiate the episode at specified time Stop commands: Force executive to terminate the episode if executive has not done so by upper time bounds.

12. Path Planning The path planner finds a collision free path between two points Updates the distance between the two points (points A and B) accordingly Autonomous vehicles follow the path with the best cost, in this case the shortest distance A B Distance from point A to point B via Path 1 = 150km Distance from point A to point B via Path 2 = 120km A B Path 1 Path 2 or Distance from point A to point B via Path 1 = 200km Distance from point A to point B via Path 2 = 250km Before path planning After path planning Represents way-points for the vehicles to follow

13. Research Extensions & Issues Hierarchical Plan Graph –Nodes expand to more complex graph Issues Plan and re-plan in a distributed manner, while being robust to communication losses –Create a network representation of a plan that results in consistent temporal plans Develop planning and execution languages that can handle real world problems in real-time Implement a path planner that quickly finds a path from source to destination Control heterogeneous teams of cooperative robotic explorers

14. Monitoring and Diagnosis of Discrete/Continuous Systems Modern Physical Artifacts: composed of interacting hardware and software components exhibit complex continuous/discrete dynamic behavior high demands on performance and availability autonomous operation without human intervention Monitoring and Diagnosis track operational mode for supervisory control system determine health state of the artifact provide means for reconfiguration and degraded operation to bypass potential catastrophic system failures

15. Motivating Example: Mars Polar Lander Most likely fault: Fault of the touchdown sensor caused a premature engine shutdown at 40 m. Possible Solution: A hybrid monitoring and diagnosis unit that monitors the continuous process of descend and the control program’s execution would have been able to detect such a subtle fault and prevent the engine shutdown.

16. Hybrid Modeling of combined hardware/software system Hybrid Probabilistic Automata model M … set of discrete modes x, u, y … continuous state, input, output F … continuous dynamics for each mode (ODE) T … probabilistic transitions between modes E.g.: probabilistic transitions: system transitions either to a new nominal mode or to a failure mode Research Issues: Composition of HPA component models -> Concurrent Hybrid Probabilistic Automata RMPL: unified compositional language for system’s modeling allows to describe complex hardware, software components and their system-wide interaction -> compiles to Concurrent Hybrid Probabilistic Automata Hybrid Probabilistic Automata (HPA) is a hidden Markov model, encoded as a set of modes that exhibit a continuous dynamical behavior, which are expressed by differential or difference equations.

17. Hybrid State Estimator maintains a set of likely hybrid state estimates Hybrid Mode Estimation and Diagnosis Hybrid State Estimator: Research Issues: Exponential explosion of possible hybrid state estimates -> focus on the most likely one by applying AI techniques currently formulated for HPA – extension to concurrent HPA models Real-time operation: “anytime” algorithm Hybrid mode estimator determines for each trajectory the possible transitions, and specifies the candidate trajectories to be tracked by the continuous state estimators.

18. Learning the System Model Our Knowledge about the system might be incomplete: how do we learn the parameters of the concurrent PHA? 1. F … parameter estimation for the nonlinear ODE system that governs the continuous behavior -> parameter estimation for nonlinear dynamic systems 2. T … estimate the transition probabilities -> parameter estimation for hidden Markov models Research Issues: Decompose models, select applicable data, coordinate and combine estimates. How do we converge quickly with sparse data and avoid getting stuck in local minima? How do we generalize automated decomposition and simplification methods to hybrid automata? Is there a unification of mode identification and estimation that exploits continuous and discrete, compile time decomposition? 7 parameters, 9 sensors

19. Testbeds Rover Testbed Advanced Live Support System - BIOPlex The Model-based Embedded and Robotics Systems group maintains a facility to evaluate various aspects of rover control using one ATRV and three ATRV-Jr. Rovers exhibit a rich set of continuous and discrete behavior and therefore will serve as the major testbed for evaluating and validating our proposed monitoring, diagnosis, model learning and refinement methods. Mode estimation will involve determining the rover’s internals state and its state with respect to its surrounding world. We will provide a rich set of sensory inputs, such as load sensors for actuators, tactile sensors, sonars, etc. to allow a comprehensive mode estimation. Evaluation of the proposed hybrid health management techniques will also include mode estimation of on-board scientific instruments, such as an spectroscopic sensor. Previously funded research at JSC has led to a variety of advanced live support (ALS) testbeds. We will apply the hybrid mode estimation techniques to the ALS system dynamic simulation testbed which provides a simple, integrated simulation of a BIO- Plex-like ALS system. The simulation models energy, water recovery, air revitalization, plant growth and crew activities. This example will challenge our approach by describing a highly complex mechanism consisting of tightly coupled sub-systems, a large number of sensors and actuators, and the interaction with crew members living in this artificial habitat. This testbed will also serve as the major evaluation platform for our model learning techniques.