1. Model-based Programming of Fault Aware Systems Brian C. Williams CSAIL, MIT

2. Three Challenges 1.Creating Fault-Aware Systems 2.Elevating Programming to Coaching 3.Programming Robust Dexterous Explorers

3. Three Days to Find a Rock Target Day 4 During the Day Science Activities Day 1 Long-Distance Traverse (<20-50 meters) Day 2 Initial Position; Followed by “Close Approach” During the Day Autonomous On- Board Navigation Changes, as needed Day 2 Traverse Estimated Error Circle Day 3 Science Prep (if Required) Day 2 Traverse Estimated Error Circle Courtesy JPL, NASA ARC 2. Elevating Programming to Coaching

4. 3. Programming Robust Dexterous Explorers

5. Creating Fault-Aware Systems: “Grabbing Success From the Jaws of Failure” Mars Exploration Rovers: Uploaded software patch shortly before entry, descent and landing. Memory leak crashes processor shortly after landing. Dragging wheel Stuck in sand trap. Courtesy JPL Subtle interactions between software, digital hardware, analog hardware and environment. Frequent novel failures. Couplings are too vast to pre-enumerate.

6. Executable Specifications Offer A Starting Point For Robustness Embedded programs interact with plant sensors and actuators: Read sensors Set actuators Embedded Program S Plant Obs Cntrl Programmer must map between state and sensors/actuators. Issue: State mapping implicit in specification. Difficult to catch specification errors Limited flexibility to adapt at run-time Executable specifications: Specification formally verified. Code synthesized from spec. Specification

7. Model-based Programs Interact Directly with State Embedded programs interact with plant sensors and actuators: Read sensors Set actuators Model-based programs interact with plant state: Read state Write state Embedded Program S Plant Obs Cntrl Model-based Embedded Program S Plant Programmer must map between state and sensors/actuators. Model-based executive maps between state and sensors/actuators. S’ Model-based Executive ObsCntrl  Produces executable specifications that are state and fault aware.

8. Model-based Programming of Fault Aware Systems Model-based programming languages elevate the programming task to state-based storyboarding and modeling. –System engineers program their high-level intentions in terms of how they would like the state of the world to evolve. –Programmers describe the embedded world using commonsense models of normal and faulty behavior. Model-based Executives implement these intentions by reasoning on the fly and at compile time. –They continually hypothesize the likely states of the world, given what they observe. –They continually plan and execute actions in order to achieve the programmer’s intentions robustly.

9. Example: Cassini and Deep Space One

10. Deep Space 1 Remote Agent Experiment Given Goal Specification and Declarative Models (HW,Ops): May 17-18th experiment: Mission-level Fault Protection Generate plan for course correction and thrust Diagnose camera as stuck on –Power constraints violated, abort current plan and replan Perform optical navigation Perform ion propulsion thrust May 21th experiment: Engineering-level Fault Protection Diagnose faulty device and –Repair by issuing reset. Diagnose switch sensor failure. –Determine harmless, and continue plan. Diagnose thruster stuck closed and –Repair by switching to alternate method of thrusting. Back to back planning RA was a toolbox, not a seamless language

11. Orbital Insertion Example EngineAEngineB Science Camera EngineAEngineB Science Camera Turn camera off and engine on

12. Control Sequencer Deductive Controller System Model CommandsObservations Control Program Plant State goalsState estimates Generates target goal states conditioned on state estimates Executes concurrently Preempts Queries (hidden) states Asserts (hidden) state OrbitInsert():: (do-watching ((EngineA = Firing) OR (EngineB = Firing)) (parallel (EngineA = Standby) (EngineB = Standby) (Camera = Off) (do-watching (EngineA = Failed) (when-donext ( (EngineA = Standby) AND (Camera = Off) ) (EngineA = Firing))) (when-donext ( (EngineA = Failed) AND (EngineB = Standby) AND (Camera = Off) ) (EngineB = Firing)))) hierarchical constraint automata over abstract state s Titan Model-based ExecutiveRMPL Model-based Program

13. Control Sequencer Deductive Controller System Model CommandsObservations Control Program Plant Titan Model-based ExecutiveRMPL Model-based Program State goalsState estimates Generates target goal states conditioned on state estimates Executes concurrently Preempts Queries (hidden) states Asserts (hidden) state Closed Valve Open Stuckopen Stuckclosed OpenClose 0. 01 0.01 0.01 inflow = outflow = 0 Probabilistic constraint automata (PCA)

14. Example: The model-based program sets engine = firing, and the deductive controller.... Selects valves on backup engine that will achieve thrust; plans needed actions. Estimates that a valve failed - stuck closed 2. Selects valve configuration; 3. plans actions to open six valves Fuel tank Oxidizer tank 1. Estimates that thrust is off, and the engine is healthy Mode Estimation Mode Reconfiguration Mode Estimation

15. Control Sequencer Deductive Controller System Model Commands Observations Control Program Plant Titan Model-based ExecutiveRMPL Model-based Program State goalsState estimates Control Sequencer: Generates goal states conditioned on state estimates Mode Estimation: Tracks likely States Mode Reconfiguration: Tracks least-cost state goals Executes concurrently Preempts Asserts and queries states Chooses based on reward Fire backup engine Valve fails stuck closed least cost reachable goal state First Action Current Belief State

16. Possible Behaviors Visualized by a Trellis Diagram Assigns a value to each variable (e.g.,3,000 vars), consistent with all state constraints (e.g., 12,000).  Intractable Probability clustered around few states. Encode trellis diagram symbolically. Enumerate only k most likely states, by exploiting conditional independence.  Enumeration framed as Optimal CSP. The Plant’s Behavior

17. Diagnosing Complex Systems Mars Polar Lander Mission lost due to interaction between software monitors and Hall effect sensors. Earth Observing One Software failures vastly more frequent than hardware failures. [Chien ASE] Challenges: 1.Monitor complex hardware/software behaviors 2.Delayed symptoms 3.Monitoring efficiency Hierarchical PCA Estimation [Mikaelian, Williams, Sachenbacher AAAI-05] [Williams, Chung, Gupta IJCAI-01]

18. Vision-based Navigation Scenario MERS Rover Testbed Probability Time 0 Power=off Power On and Take Picture 1 Nominal Cam=Broken Battery=low Sensor=Broken 2 Observe Sensor voltage = low Battery=low Cam=Broken Sensor=Broken Image not corrupt ………………..……………. 6 S/W=> Sensor=Broken Battery=low Cam=Broken

19. Modeling Complex Processes PHCA H/W models S/W specs [Williams, Chung, Gupta 2001] Complex Behavior Embedded Program (Esterel)  Hierarchical Automata Probabilistic Constraint  Hierarchical Probabilistic Program (RMPL)Constraint Automata PHCA = PCA + HCA from plant model from control program [Mikaelian, Williams & Martin, AAAI 05]

20. Mapping Probabilistic Embedded Programs (RMPL) to Hierarchical Probabilistic Constraint Automata

21. Diagnostic Process PHCA H/W models S/W specs (code) [Williams, Chung, Gupta 2001] Complex Behavior Offline compilation phase N-Stage OCSP/VCSP Delayed Symptoms parameter N Online solution phase [Mikaelian, Williams & Martin, AAAI 05]

22. Delayed Symptoms: Encode PHCA as an N-stage Filter Based on an Optimal CSP T3 On T4 T5 turnOn turnOff Power=zero Power=nominal T6 T7 Observable variables: command, Power State (location) variables: Off, On, Broken Auxiliary variables encoding transition constraints (multiple simultaneous transitions may be possible per PHCA): T1->T7 Edges/brackets indicate constraints among variables

23. Diagnostic Process Tree Decomp Implicate Gen Offline compilation phase N-Stage OCSP/COP PHCA H/W models S/W specs (code) Efficiency [Gottlob, Leone, Scarcello 2000; Kask, Dechter, Larossa 2003] Online solution phase observationscommands Optimal Constraint Solver Dynamic update of OCSP/COP 1.Lazy Dynamic Programming 2.Set-based B&B w ADDs 3.Forward Conflict-directed BF Search [Mikaelian, Williams & Martin, AAAI 05] Cost based on Viterbi

24. Demonstration Scenarios MIT SPHERES Testbed Models NASA Earth Observing One (EO-1) Models - Advanced Land Imager - Hyperion Instrument - Wideband Advanced Recorder Processor EO-1 (12 components) -Global Metrology Subsystem SPHERES 1 (5 components) SPHERES 2 (18 components)

25. Results: Online (1.6 GHz Pentium M) Solver: [Sachenbacher and Williams, CP 2004]

26. Recovering From Failure

27. Hybrid Estimation Hybrid case: estimate hybrid state from noisy observations Continuous state Discrete mode Hybrid probabilistic constraint automata –Stochastic transitions between discrete modes –Different continuous dynamics for each mode nominal failed 0.001 0 0.999 1 actuator

28. Kalman Filters Track Subset of Trajectories ok 0 0 0 1 failed 0 1 ok 1 0 1 failed 0 1 t = 0t = 1t = 2 failed 0 1 … … … … … … … … … [Blackmore, Funiak, Williams AAAAI 05]

29. Mixed Greedy/Stochastic Sampling Concentrated PosteriorFlat Posterior 1.K-best: performs best for concentrated priors. 2.Rao-Blackwell particle filtering performs best for flat prior. 3.Mixed strategy, balances best of both.

30. Estimation Results 1.Transfer scenario empty both WAM2

31. Estimation Results 3.Stuck scenario

32. Commanding as Coaching: Finding a Rock in Less than Three Days Target Day 4 During the Day Science Activities Day 1 Long-Distance Traverse (<20-50 meters) Day 2 Initial Position; Followed by “Close Approach” During the Day Autonomous On-Board Navigation Changes, as needed Day 2 Traverse Estimated Error Circle Day 3 Science Prep (if Required) Day 2 Traverse Estimated Error Circle Courtesy JPL, NASA ARC

33. Model-Predictive Method Selection To ensure safe, optimal execution, the control sequencer: Receives descriptions of possible contingencies. Dynamically selects consistent methods over future horizon, Adapts to uncertainty by selecting execution times dynamically, Monitors outcomes and plans contingencies. Continuous Temporal Planner Control Program Commands Observables Mode Estimation Mode Reconfiguration Model of Subsystems Plan Runner RMPL imageScienceTargets(Rover1, Rover2) { { [5,10] Rover1.goto(p4); [5,10] Rover1.goto(p5); [2,5] Rover1.imageTargets(); [5,10] Rover1.goto(p3); }, { [5,10] Rover2.goto(p1); [5,10] Rover2.goto(p2); [2,5] Rover2.findTargets(); [5,10] Rover2.goto(p3); } Temporal Plan Network control sequencer deductive controller

34. Control Sequencer Continually Searches for Optimal Consistent Threads of Execution Start End Rover1.goto(p4) Rover2.goto(p1) Rover1.imageTargets Rover1.goto(p5) Rover1.goto(p3) p1 p2 p3 p4 p5 1 2 Rover1.goto(p5) Rover1.imageTargets Rover2.goto(p2) Rover2.imageTargets Rover2.goto(p3) Rover2.goto(p2) Rover2.goto(p3) Rover2.imageTargets Ask site1 =  obstructed Failure Throw: Type: imageTargets Reason: site1 = obstructed Tell site1 =  obstructed obstructed imageScienceTargets(Rover1, Rover2) { [5,10] Rover1.goto(p4); choose { { do { [5,10] Rover1.goto(p5); } maintaining( site1 =  obstructed); [2,5] Rover1.imageTargets(); } { [2,5] Rover1.imageTargets(); [5,10] Rover1.goto(p5); } }; [5,10] Rover1.goto(p3); }, { [5,10] Rover2.goto(p1); choose { { do { [2,5 ]Rover2.imageTargets(); } maintaining ( site1 =  obstructed); [5,10] Rover2.goto(p2); [5,10] Rover2.goto(p3); } { [5,10] Rover2.goto(p2); [5,10] Rover2.goto(p3); [2,5] Rover2.imageTargets(); } Catch: Type: imageTargets Handler: Rover1.goto(p4) Tell: site1 = obstructed

35. A Walk on Mars

36. Model-based Programs Specify Qualitative Gaits [Muybridge, 1955] Stop-action photographic study of human and animal motion Gaits depicted as sequences of distinct qualitative poses Flexible spatial and temporal constraints

37. Hybrid executive coordinates controllers - to sequence biped through qualitative state plan Executive is like a marionetteer

38. Nominal Walking Angular momentum tightly conserved during normal walking Allows for linearizing controllers that decouple state variables and makes them directly controllable [Hofmann, et al; 2004]

39. Feasible trajectories must go through goal regions

40. Flow tubes denote all feasible trajectories

41. Center of Mass CM tube constrained by foot position tubes: Foot positions define support polygon.. Center of foot Pressure CP constrained to be inside support polygon. CM coupled to CP.

42. Disturbance displaces trajectory in state space Dispatcher selects trajectory within tubes online. If disturbance not too large, displacement stays in tube. Activity still executes successfully.

43. Disturbance displaces trajectory in state space If disturbance too large, trajectory pushed outside tube. Goal region not achievable at the required time. Plan failure detected immediately leaving more room for recovery.

45. Control Sequencer System Model CommandsObservations Control Program Plant State goalsState estimates Executes concurrently Preempts Queries (hidden) states Asserts (hidden) state Model-based ExecutiveRMPL Model-based Program Mode Reconfiguration Mode Estimation Closed Valve Open Stuckopen Stuckclosed OpenClose 0. 01 0.01 0.01 inflow = outflow = 0 Biomimetic hybrid constraint automata hierarchical timed hybrid constraint automata Self-repairingCoachedAgile

46. Model-based Programming Provides a programmer idealization in which state is directly observable, while managing robustness automatically. A wide range of systems can be modeled and reasoned about using variants of constraint automata –Probabilistic, decision-theoretic, timed, hierarchical, hybrid. Execution reasons over abstract descriptions of possible trajectories over a limited horizon. –Symbolic trellis, temporal plan networks, flow tubes. High performance achievable through Forward, Conflict-directed Optimal Search