1. Approved for Public Release, Distribution Unlimited Pervasive Self-Regeneration through Concurrent Model-Based Execution Brian Williams (PI) Paul Robertson MIT Computer Science and Artificial Intelligence Laboratory

2. Approved for Public Release, Distribution Unlimited 7/20/042 Outline What we are trying to do. Demonstration scenario and test bed. Implications of successful results. Technical approach. What is new. Anticipated challenges. Technology transition. Looking forward: next steps.

3. Approved for Public Release, Distribution Unlimited 7/20/043 What we are trying to do Why software fails: –Software assumptions about the environment become invalid because of changes in the environment. –Software changes introduce incompatibilities. –Software is attacked by a hostile agent. What can be done when software fails: –Recognize that a failure has occurred. –Diagnose what has failed – and why. –Find an alternative way of achieving the intended behavior. Runtime Models

4. Approved for Public Release, Distribution Unlimited 7/20/044 Building upon a proven technology base. By extending RMPL to support software failure, we can extend robustness in the face of hardware failures to robustness in the face of software failures. Many of the same issues pertain: –Detection of faulty behavior –Diagnosis of the fault given faulty behavior –Reconfiguration of the software to achieve intended behavior using different software components. –Select among alternatives to maximize utility.

5. Approved for Public Release, Distribution Unlimited 7/20/045 Deliverables Model-based programming tools –A language for modeling (RMPL): The process in terms of desired state evolutions; The components; and The environment. Model-based executives that provide: –Safe optimal dispatch. –Redundant methods. –Continuous monitoring and diagnosis. –Regeneration and optimization.

6. Approved for Public Release, Distribution Unlimited 7/20/046 Architectural overview S Plant Obs Cntrl Model-based Embedded Programs S Continuous Reactive Commanding Continuous Mode/State Estimation Model Desiderata: languages that are Suspicious Monitor intentions and plans Self-Adaptive Exploits and generates contingencies State and Fault Aware Anticipatory –“Model-predictive languages” –Plans and verifies into the future –Predicts future states –Plans contingencies

7. Approved for Public Release, Distribution Unlimited 7/20/047 Basic assumptions for our approach Objectives for the RMPL language –Low overhead The effort required of the programmers to achieve robustness should be small compared to the effort required to implement the base capability. –Pervasive The technology should be applied not only to major components but to all components in the system. –Incremental Increased robustness can be achieved incrementally by adding greater modeling.

8. Approved for Public Release, Distribution Unlimited 7/20/048 Innovative claims Features of our approach: –Fault-aware processes. –Fault-adaptive. –Model-based. –Synthesizes a fault-adaptive process to achieve state evolutions. –Reasons from MODELS of correct and faulty behavior of supporting service components. –Constructs novel recovery actions in the face of novel faults. Specific approaches: –Dynamic selection from redundant methods. –Self-optimization: select the optimal candidates. –Continuous monitoring. –Incremental addition of robustness by adding monitoring procedures incrementally. –Synthesis of repair procedures from models.

9. Approved for Public Release, Distribution Unlimited 7/20/049 Demonstration scenario End to End Self-regeneration of Command & Control Systems: Robot must plan and execute motion to one or more targets. Robot must perform designated tasks at selected destinations. Robot utilizes various sensors and actuators in achieving its task. Robot utilizes various software components in interpreting its sensor data and manipulating its actuators. Changing environment will cause software components to fail. Failed software components will be detected and diagnosed. Alternate configurations of the software will be found that can maintain mission objectives while maximizing utility—in real- time. Fault injection testing: We will inject faults into the test bed including (1) environment changes, (2) incompatibilities, (3) software attacks.

10. Approved for Public Release, Distribution Unlimited 7/20/0410 Test bed summary Robot scenario involves: –Path planning and execution. –Goal selection with risks and rewards. –Visual and other sensors and actuators utilized in navigation and task execution. –Reacts to: Failures in the software. –Exploiting redundant methods. –By re-planning to maintain optimality. Discoveries in the environment. –Obstacles –Suitability of using selected sensors given terrain lighting etc. Attack

11. Approved for Public Release, Distribution Unlimited 7/20/0411 Rover test bed Allows real-world testing of planning and execution software Consists of a reconfigurable environment with one ATRV- 2 and three ATRV-JRs.

12. Approved for Public Release, Distribution Unlimited 7/20/0412 Rover test bed setup Differential drive Laser range scanner Sonar sensors Wheel encoders (odometry) Stereo camera Inclinometer GPS receiver Compass Antennas for wireless LAN Sensors give information on motion and environment. Onboard PC allows for real-time computation and command processing.

13. Approved for Public Release, Distribution Unlimited 7/20/0413 Implications of successful results Robotic systems that can operate autonomously to achieve goals in a complex and changing environment. –Modeling environment Software that detects and works around “bugs” resulting from incompatible software changes. –Modeling software components Software that detects and recovers from software attacks. –Modeling attack scenarios Software that automatically improves as better software components and models are added. Higher level of command and control for robotic missions.

14. Approved for Public Release, Distribution Unlimited 7/20/0414 Military roles for autonomous robots Reconnaissance –Rover makes its way to designated places and reports back—such as with pictures. Search and Rescue –Rover enters a dangerous area and reports back on the presence of wounded people. Mapping –Rover explores (a building) producing a map annotated with pictures in support of ground forces. Munitions Delivery –Rover goes to designated locations and delivers munitions. (like predator UAV)

15. Approved for Public Release, Distribution Unlimited 7/20/0415 Technical approach We will extend the technology developed for execution, hardware fault detection, diagnosis and reconfiguration successfully used in Deep Space One to be applied to software fault awareness and reconfiguration. Software components look like hardware components but reconfiguration is less restricted. A greater emphasis on environment modeling is necessary because most software faults will be because of environmental changes.

16. Approved for Public Release, Distribution Unlimited 7/20/0416 Systems Should be Suspicious Sensor Interpretation: Nominal: Command Confirmation Fault Detection Failure: Fault Isolation Fault Diagnosis Traditional Commanding is Open Loop Continuous Estimation of Modes and State Model Real Time Robust Systems Should be Fully State Aware And Should Use To Dynamically Monitor Specifications

17. Approved for Public Release, Distribution Unlimited 7/20/0417 Commanding Involves Repairing a Correct State For long-lived systems, failure is the rule: Must navigate around failures. Must operate with varying resources and capabilities. Should select best means amongst multiple alternatives. Nominal commanding is Traditionally pre-determined Robust systems should select the best action in context Continuous Reactive Commanding Continuous Mode/State Estimation Model

18. Approved for Public Release, Distribution Unlimited 7/20/0418 How Do We Guide Self-Regenerative Systems? - By Interacting Directly with State Embedded programs interact with sensors and actuators. Model-based programs interact with state Embedded Program S Services Obs Cntrl Model-based Embedded Program S Services Programmers map between sensors, actuators to states. Model-based executives map between sensors, actuators to states.

19. Approved for Public Release, Distribution Unlimited 7/20/0419 Model-based Programs Interact Directly with State S Plant Obs Cntrl Model-based Embedded Programs Model-based Executive S Plant Model State estimates Reactive Commanding Mode Estimation State goals RMPL State assertion State query Conditional Execution Preemption Iteration Concurrent execution Fault Occurs Current Belief State Reconfigure least cost reachable goal state First Action

20. Approved for Public Release, Distribution Unlimited 7/20/0420 Real-Time Execution Flight H/W Fault Monitors Planning Experts (incl. Navigation) Model-based Autonomy Architecture Ground System RAX Manager Model- based Fault Protection Mission Manager Executive Planner/Scheduler

21. Approved for Public Release, Distribution Unlimited 7/20/0421 Real-Time Execution Flight H/W Fault Monitors Planning Experts (incl. Navigation) Ground System RAX Manager Model-based Fault Protection Mission Manager Executive Planner/Scheduler Remote Agent Goal States Model-based Autonomy Architecture

22. Approved for Public Release, Distribution Unlimited 7/20/0422 Real-Time Execution Flight H/W Fault Monitors Planning Experts (incl. Navigation) Ground System RAX Manager Model-based Fault Protection Mission Manager Executive Planner/Scheduler Remote Agent Model-based Autonomy Architecture

23. Approved for Public Release, Distribution Unlimited 7/20/0423 Real-Time Execution Flight H/W Fault Monitors Planning Experts (incl. Navigation) Ground System RAX Manager Model- based Fault Protection Mission Manager Procedural Executive Planner/Scheduler Remote Agent Low-Level Fault Protection Model-based Autonomy Architecture

24. Approved for Public Release, Distribution Unlimited 7/20/0424 Real-Time Execution Flight H/W Fault Monitors Planning Experts (incl. Navigation) Ground System RAX Manager Model- based Executive Mission Manager Procedural Executive Planner/Scheduler Remote Agent High-Level Fault Protection Model-based Autonomy Architecture

25. Approved for Public Release, Distribution Unlimited 7/20/0425 Remote Agent Experiment May, 1999 May 17-18th experiment: High-level Fault Protection Generate plan for course correction and thrust Diagnose camera as stuck on –Power constraints violated, abort current plan and replan Perform optical navigation Perform ion propulsion thrust May 21th experiment: Low-level Fault Protection Diagnose faulty device and –Repair by issuing reset. Diagnose switch sensor failure. –Determine harmless, and continue plan. Diagnose thruster stuck closed and –Repair by switching to alternate method of thrusting. Back to back planning RA was a toolbox, want a seamless language

26. Approved for Public Release, Distribution Unlimited 7/20/0426 Technology transition The structure of our solution facilitates transition. –Implements self-regenerative system using language+executive, as opposed to a set of regeneration utilities. –Easily wraps self-regeneration around existing components, through a clean separation of the “executive” and “plant.” –Has supported a long history of successful technology transition. Papers and other publications

27. Approved for Public Release, Distribution Unlimited 7/20/0427 Looking forward: next steps Reasoning about complex software models. Coordination while preserving privacy of subsystem information. Extending the technology to work with complex distributed self-regenerative systems. –Regeneration requires peer to peer coordination of systems. –Subtle faults that are distributed across multiple systems. –Faults that are detected in systems in which we do not have direct control—and must negotiate fault resolution.

28. Approved for Public Release, Distribution Unlimited 7/20/0428 Appendix A

29. Approved for Public Release, Distribution Unlimited 7/20/0429 Model-based Execution Kernels as Stochastic Optimal Controllers Deductive Controller Plant mode estimation mode reconfiguration s’(t) commands(t) f f s (t) g g Observations(t) Model Goal States

30. Approved for Public Release, Distribution Unlimited 7/20/0430 Operators and programmers reason through system- wide interactions to: isolate faults isolate faults diagnose causes diagnose causes Diagnosis

31. Approved for Public Release, Distribution Unlimited 7/20/0431 OPSAT Conflict- directed A* Conflict- directed A* Optimalfeasiblemodes Conflict Checkedkernel ISAT Generate Most-likely Candidate Diagnoses: conflict-directed A* (< 10 states visited) Test Against Model and Observables: Incremental Satisfiability (avg. < 10 % off ideal) Learn from counter examples (conflicts) to guide generation.

32. Approved for Public Release, Distribution Unlimited 7/20/0432 Compare Most Likely Candidate to Observations

33. Approved for Public Release, Distribution Unlimited 7/20/0433 Isolate Conflicting Modes A conflict, C, is an assignment to a subset of the mode variables that is inconsistent with the model and observations.

34. Approved for Public Release, Distribution Unlimited 7/20/0434 Generate Next Most Likely Candidate Every consistent mode assignment must differ from the conflict for at least one mode variable

35. Approved for Public Release, Distribution Unlimited 7/20/0435 Generate Next Most Likely Candidate Every consistent mode assignment must differ from the conflict for at least one mode variable

36. Approved for Public Release, Distribution Unlimited 7/20/0436 Testing Candidate Detects Another Conflict

37. Approved for Public Release, Distribution Unlimited 7/20/0437 Consistent mode assignment must differ from both conflicts Generate Next Most Likely Candidate

38. Approved for Public Release, Distribution Unlimited 7/20/0438 Consistent: Most Likely Diagnosis

39. Approved for Public Release, Distribution Unlimited 7/20/0439 Operators and programmers reason through system- wide interactions to: isolate faults isolate faults diagnose causes diagnose causes Diagnosis repair repair reconfigure reconfigure ConfigurationManagement

40. Approved for Public Release, Distribution Unlimited 7/20/0440 Conflicts Focus MR Goal: Achieve Thrust Find least cost modes that entail the current goal. A conflict, C, is an assignment to a subset of the mode variables that entails the negation of the goal.

41. Approved for Public Release, Distribution Unlimited 7/20/0441 Goal: Achieve Thrust Conflicts Focus MR Find least cost modes that entail the current goal. A conflict, C, is an assignment to a subset of the mode variables that entails the negation of the goal.

42. Approved for Public Release, Distribution Unlimited 7/20/0442 Goal: Achieve Thrust Conflicts Focus MR Find least cost modes that entail the current goal. A conflict, C, is an assignment to a subset of the mode variables that entails the negation of the goal.

43. Approved for Public Release, Distribution Unlimited 7/20/0443 Goal: Achieve Thrust Conflicts Focus MR Find least cost modes that entail the current goal. A conflict, C, is an assignment to a subset of the mode variables that entails the negation of the goal.

44. Approved for Public Release, Distribution Unlimited 7/20/0444 Closed Open Stuck open Stuck closed Open Close Cost 5 Prob.9 Models are Concurrent, Constraint Encodings of Partially Observable Markov Decision Process Vlv = closed => Outflow = 0; vlv=open => [Outflow] S = [Inflow] S and [Outflow] R = [Inflow] R vlv=stuck open => [Outflow] S = [Inflow] S and [Outflow] R = [Inflow] R vlv=stuck closed=> Outflow = 0; Unknown One automaton for each device. Communication through shared variables.

45. Approved for Public Release, Distribution Unlimited 7/20/0445 Operators and programmers reason through system- wide interactions to: isolate faults isolate faults diagnose causes diagnose causes monitor monitor confirm commandsconfirm commands track goalstrack goals Estimating Modes

46. Approved for Public Release, Distribution Unlimited 7/20/0446 Mode Estimation Left engine onObserve “no thrust” Enumerated by decreasing probability. Find most likely reachable states consistent with observations. Every component transitions at each time step.

47. Approved for Public Release, Distribution Unlimited 7/20/0447 Operators and programmers reason through system- wide interactions to : Controlling Modes Estimating Modes monitor monitor track goals track goals confirm commands confirm commands isolat faults isolat faults diagnose faults diagnose faults repair repair reconfigure reconfigure execute execute avoid failures avoid failures change control policies change control policies

48. Approved for Public Release, Distribution Unlimited 7/20/0448 1.Find least cost state that entails the current goal and is reachable from the current state. 2.Find first action that moves towards this state. Model-based Reactive Planning Mode Reconf. Mode Est. Command Configuration goals Model goal statecurrent state Reactive Planning Burton Model-based Executive IJCAI97

49. Approved for Public Release, Distribution Unlimited 7/20/0449 Architectural overview S Plant Obs Cntrl Model-based Embedded Programs S Continuous Reactive Commanding Continuous Mode/State Estimation Model Desiderata: languages that are Suspicious Monitor intentions and plans Self-Adaptive Exploits and generates contingencies State and Fault Aware Anticipatory –“Model-predictive languages” –Plans and verifies into the future –Predicts future states –Plans contingencies

50. Approved for Public Release, Distribution Unlimited 7/20/0450 Anticipated Challenges Software is harder than hardware: –Hardware recovers by leveraging backups, while software leverages alternative methods. –Models of software are more complex to specify. –While hardware’s topology is static, software’s topology dynamically changes. Performance –Software systems result in larger state spaces, making real-time response a challenge.