1. Lecture 2: Reactive Systems Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments

2. © Gal Kaminka 2 Previously, on Robots … Robots are agents in physical or virtual environments Persistent, situated, responsive Environments have different characteristics Static/dynamic, accessible?, deterministic?, …. Since we are lazy, we want robots to do things for us Robots must consider task when deciding what to do Action-selection problem: What to do now in service of the task?

3. © Gal Kaminka 3 Physical Environments Dynamic Non-deterministic Inaccessible Continuous

4. © Gal Kaminka 4 A Crash Course in AI Planning Planning: An approach to action-selection problem Very long history, since the very beginning of AI 1971, seminal paper: STRIPS (Fikes and Nilsson) Still cited and taught today, despite much progress STRIPS originally developed for SRI robot, Shakey This is ironic Later on, STRIPS planning was rejected for robot control

5. © Gal Kaminka 5 AI Planning Definition: State { On(A,table), On(B,table), On(C,table), In-hand(nil) } ABC { On(A,B), On(B,table), On(C,table), In-hand(D) } A BC D

6. © Gal Kaminka 6 AI Planning Definition: Operators Operators change state of the world Preconditions, add-list, delete-list Pick-A(?x) Preconditions: On(A, ?x), In-hand(nil) Add: In-hand(A) Delete: In-hand(nil), On(A, ?x) Put-A(?y) Preconditions: In-hand(A), not On(?x, ?y) Add: On(A,?y), In-hand(nil) Delete: In-hand(A)

7. © Gal Kaminka 7 STRIPS Planning Given: Initial state of the world Operators Goal state Produce: Plan: Ordered list of instantiated operators Will change the world from initial state to goal state

8. © Gal Kaminka 8 Planning Example AB C Initial State A B C Goal State A B C After Pick-B(table) A B C After Put-B(A) A B C After Pick-C(table) After Put-C(B)

9. © Gal Kaminka 9 Planning on robots Sense initial state using sensors Create a full plan given goal state (given task) Feed plan, step-by-step to motors No need to sense again What’s wrong with this? (Hint: Think about Schoppers’ paper)

10. © Gal Kaminka 10 Deliberative Control Deliberative: Has internal state (typically a model of the world) Uses this internal state to make decisions Decisions made between alternatives Sense ModelThinkAct

11. © Gal Kaminka 11 When plans goes wrong Dynamic environment State changes even if no operator applied Non-deterministic State changes not according to operator specs Inaccessible Cannot sense entire state of the world Continuous Predicate-based description of world is discrete

12. © Gal Kaminka 12 Reactive control Reactive: No internal state Direct connection from sensors to actions S-R (stimulus response) systems No choices, no alternatives Sense Hard WiringAct

13. © Gal Kaminka 13 Universal Planning Have a plan ready for any possible contingency Scouts: Be prepared! From any initial state, know how to get to goal state Input: Operators, goal state Do not need to give initial state Output: Decision tree What operator to take, depending on environment state Not a single ordered list of operators

14. © Gal Kaminka 14 Universal planning algorithm A B C After Pick-B(table) A B C Goal State A B C After Pick-C(table) A B C After Put-B(A) AB C Initial State

15. © Gal Kaminka 15 Robot Control Algorithm Using Universal Planning Robot given task (goal, operators) Uses universal planner to create universal plan Robot senses environment Goal state reached? No: Execute operator according to decision tree Yes (keep persistency)

16. © Gal Kaminka 16 Advantages of Universal Planning Guaranteed to use optimal (shortest) plan to goal A very good thing! Optimal solution to action selection problem Robust to failures Robust in dynamic and non-deterministic domains

17. © Gal Kaminka 17 Problems with Universal Planning Assumes accessibility Assumes perfect sensors Assumes discrete actions (operators)

18. © Gal Kaminka 18 Universal plan as mapping sensors to actions Universal plan can be viewed as a function Sensor readings to actions u: S  A Essentially a table: For each state, give action Schoppers uses a decision-tree representation

19. © Gal Kaminka 19 Problems: Planning Time What is the planning time? Planning time grows with the number of states Since we have to enumerate operator for every state What is the number of states in an environment? Worst case: All possible combinations of sensor readings (state predicates)

20. © Gal Kaminka 20 Problems: Universal Plan Size Plan size grows with the number of possible states “Curse of dimensionality” X1 X2 Pick Put X1 X2 Pick Put X2 X1 Pick Put

21. © Gal Kaminka 21 Problems: Stupid executioner Schoppers: Baby goes around knocking blocks around? Ginsberg: What if baby repeatedly knocks down the same block? Universal plans may get into cycles This is because no deliberation is done Universal planner relies on simple executioner Sense, consult table, act Same as regular planner – except for sensing

22. © Gal Kaminka 22 שאלות?

23. © Gal Kaminka 23 Brooks’ Subsumption Architecture Multiple levels of control: Behaviors Avoid Object Wander Explore Map Monitor Change Identify objects Plan changes

24. © Gal Kaminka 24 Why does this work? It breaks the ideal universal plan into behaviors avoids the curse of dimensionality Behaviors (levels) interact to generate behavior Note that programmer is responsible for task- oriented design Goes both below and above universal plans Hand programmed: approximate plan Not automatically generated

25. © Gal Kaminka 25 Subsuming Layers How to make sure overall output is coherent? e.g., avoid object is in conflict with explore Subsumption hierarchy: Higher levels modify lower Avoid Object Wander Explore Map

26. © Gal Kaminka 26 Coherence using subsumption Key principle Higher layers control input/outputs of lower layers In practice: Can be difficult to design a lot depends on programmer Single module per layer can be restrictive Increasingly challenging to control at higher levels

27. © Gal Kaminka 27 Irony Ginsberg’s article pretty much killed universal planning Though occasional papers still published Reactive control very popular in practice But due to theory problem, no more automated planners! So we get lots of reactive plans, but no planning

28. © Gal Kaminka 28 Irony again Ginsberg was right: Approximating universal plan is possible Tends to be useful only in fairly low level locomotion control Approximation is what Brooks had done Which is why he often gets the credit for the revolution

29. © Gal Kaminka 29 Starting next week Behavior-Based Control: Expanding on Subsumption