1. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm1 The Reactive Paradigm Describe the Reactive Paradigm in terms of the 3 robot primitives and its organization of sensing List the characteristics of a reactive robotic system, and discuss the connotations of surrounding the reactive paradigm Describe the two dominant methods for combining behaviors in a reactive architecture: subsumption and potential field summation Be able to program a behavior using pfields Be able to construct a new potential field from primitive pfields and sum pfields to generate an emergent behavior Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary

2. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm2 Review: Lessons from Biology Programs should decompose complex actions into behaviors. Complexity emerges from concurrent behaviors acting independently Agents should rely on straightforward activation mechanisms such as IRM Perception filters sensing and considers only what is relevant to the task (action-oriented perception) Behaviors are independent but the output may be used in many ways including: combined with others to produce a resultant output or to inhibit others Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary

3. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm3 Hierarchical Organization is “Horizontal” Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary

4. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm4 More Biological is “Vertical” Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary Higher level behaviors reuse or inhibit more primitive behaviors

5. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm5 Sensing is Behavior-Specific or Local Behaviors can “share” perception without knowing it This is behavioral sensor fusion Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary

6. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm6 Reactive Robots Most apps are programmed with this paradigm Biologically based: –Behaviors (independent processes), released by perceptual or internal events (state) –No world models or long term memory –Highly modular, generic –Overall behavior emerges SENSEACT RELEASER behavior Overview History Reactive USAR Summary

7. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm7 Example 1: Robomow Behaviors? Random Avoid –Avoid(bump=obstacle) –Avoid(wire=boundary) Stop –Stop(tilt=ON) All active www.friendlymachines.com Overview History Reactive USAR Summary

8. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm8 Example 2: My Real Baby Behaviors? Touch-> Awake Upside down & Awake-> Cry Awake & Hungry -> Cry Awake & Lonely -> Cry Note can get crying from multiple behaviors Note internal state (countdown timer on Lonely) www.irobot.com Overview History Reactive USAR Summary

9. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm9 Reactive Behaviors Connotations –Execute rapidly Can be implemented in hardware –Have no memory Characteristics of reactive architectures –Robots are situated agents operating in an ecological niche –Behaviors are basic building blocks for robotic actions, and overall behavior of robot emerges from their interaction Independent, concurrent Schema of a behavior may have a coordinated control program, but there is no external controller of all behaviors for a task Architecture may use combination, suppression, or cancellation for interaction –Only local, behavior-specific sensing in permitted No world model, representation from sensing is ego-centric –Modular Tasks are broken down into behaviors; behaviors can be tested independently; new behaviors can be built from more primitive ones –Based on animal models of behavior

10. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm10 Reactive Historically, there are two main styles of creating a reactive system –Subsumption architecture Layers of behavioral competence How to control relationships –Potential fields Concurrent behaviors How to navigate They are equivalent in power In practice, see a mixture of both layers and concurrency

11. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm11 Subsumption: Rodney Brooks From http://www.spe.sony.com/classics/fastcheap/index.html Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary

12. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm12 Subsumption Philosophy Modules should be grouped into layers of competence Modules in a higher lever can override or subsume behaviors in the next lower level –Suppression: substitute input going to a module –Inhibit: turn off output from a module No internal state in the sense of a local, persistent representation similar to a world model. Architecture should be taskable: accomplished by a higher level turning on/off lower layers Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary

13. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm13 Level 0: runaway HALT COLLIDE PS MS RUN AWAY PS MS runaway 0 wander 1 follow-corridor 2 Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary sensor s motor actions behaviors when obstacle comes near turn around, run away; when collision imminent, stop, turn around, run away direction magnitude

14. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm14 Example Perception: Polar Plot Plot is ego-centric Plot is distributed (available to whatever wants to use it) Although it is a representation in the sense of being a data structure, there is no memory (contains latest information) and no reasoning (2-3 means a “wall”) Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary if sensing is ego-centric, can often eliminate need for memory, representation

15. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm15 Level 1: Wander runaway 0 wander 1 follow-corridor 2 Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary encoders AVOID PS MS WANDER PSMS Note sharing of Perception, fusion Avoid suppresse s (replaces) output from runaway What would Inhibition do?

16. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm16 Class Exercise runaway 0 wander 1 move2light 2 LIGHT PHOTO- TROPHISM S Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary

17. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm17 Level 2: Follow-Corridors runaway 0 wander 1 follow-corridor 2 STAY-IN-MIDDLE PSMS Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary Computed from shaft encoders Intended course

18. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm18 Class Exercise Design the roomba with subsumption

19. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm19 Subsumption Review What is the Reactive Paradigm in terms of primitives? –Sense, act What is the Reactive Paradigm in terms of sensing? –Local to each behavior Does the Reactive Paradigm solve the Open World problem? –Open world is non-monotonic; need truth maintenance mechanism –Reactive paradigm has no memory, no truth maintenance How does the Reactive Paradigm eliminate the frame problem? –No world model, so no frame problem What is the difference between a behavior and a level of competence? –Not schema theoretic; level of competence groups schema-like modules into abstract behaviors What is the difference between suppression and inhibition in subsumption? –Suppression acts like a gate; inhibition like on/off switch Review Organization -SA -beh. specific Subsumption -Philosophy -Level 0 -Level 1 -Level 2 Summary

20. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm20 Potential Fields: Ron Arkin From http://www.cc.gatech.edu/aimosaic/faculty/arkin From http://www.cc.gatech.edu/aimosaic/robot-lab/MRLhome.html

21. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm21 Potential Fields Philosophy The motor schema component of a behavior can be expressed with a potential fields methodology –A potential field can be a “primitive” or constructed from primitives which are summed together –The output of behaviors are combined using vector summation From each behavior, the robot “feels” a vector or force –Magnitude = force, strength of stimulus, or velocity –Direction But we visualize the “force” as a field, where every point in space represents the vector that it would feel if it were at that point

22. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm22 Example: Run Away via Repulsion

23. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm23 5 Primitive Potential Fields uniform perpendicular tangential

24. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm24 Draw These Now! Common fields in behaviors Uniform –Move in a particular direction, corridor following Repulsion –Runaway (obstacle avoidance) Attraction –Move to goal Perpendicular –Corridor following Tangential –Move through door, docking (in combination with other fields) random –do you think this is a potential field? what would it look like?

25. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm25 Class Exercise Name the field you’d use for –Moving towards a light –Avoiding obstacles Attractive Repulsive

26. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm26 Magnitude profiles Constant magnitude Linear drop off Exponential drop off P. 127

27. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm27 Programming a potential field Example: repulsive field with linear dropoff, only one sensor Vdirection = -180 degrees Vmagnitude = (D-d)/D if d <= D where D is range of potential field 0 otherwise (magnitude controls velocity) Implementation in C on p. 129-130 With more than one sensor – fig. =4.18, p. 134 - 136

28. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm28 Problems Impact of update rates –if time between updates is too long Path can be jerky Can overshoot Robots can’t change velocity and direction immediately Fields may sum to 0

29. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm29 Combining Fields for Emergent Behavior obstacle goal If robot were dropped anywhere on this grid, it would want to move to goal and avoid obstacle: Behavior 1: MOVE2GOAL Behavior 2: RUNAWAY The output of each independent behavior is a vector, the 2 vectors is summed to produce emergent behavior obstacle goal

30. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm30 Note: In this example, repulsive field only extends for 2 meters; the robot runs away only if obstacle within 2 meters Note: in this example, robot can sense the goal from 10 meters away Fields and Their Combination

31. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm31 Path Taken If robot started at this location, it would take the following path It would only “feel”the vector for the location, then move accordingly, “feel” the next vector, move, etc. Pfield visualization allows us to see the vectors at all points, but robot never computes the “field of vectors” just the local vector Robot only feels vectors for this point when it (if) reaches that point

32. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm32

33. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm33 Discussion Could you represent the Arctic Tern feeding behavior with potential fields? –what happens with multiple red dots? –can you inhibit with potential fields?

34. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm34 Example: follow-corridor or follow-sidewalk PerpendicularUniform Combined Note use of Magnitude profiles: Perpendicular decreases

35. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm35 Class Exercise: Draw Fields for Wall-Following (assume that robot stands still if no wall) Just half of a follow-corridor, but…

36. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm36 But how does the robot see a wall without reasoning or intermediate representations? Perceptual schema “connects the dots”, returns relative orientation PS: Find-wall MS: Perp. MS: Uniform S Sonars orientation

37. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm37 OK, But why isn’t that a representation of a wall? It’s not really reasoning that it’s a wall, rather it is reacting to the stimulus which happens to be smoothed (common in neighboring neurons)

38. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm38 Level 0: Runaway Note: multiple instances of a behavior vs. 1: Could just have 1 Instance of RUN AWAY, Which picks nearest reading; Doesn’t matter, but this Allows addition of another Sonar without changing the RUN AWAY behavior

39. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm39 Level 1: Wander Wander is Uniform, but Changes direction aperiodically

40. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm40 Level 2: Follow Corridor Follow-corridor Should we Leave Run Away In? Do we Need it?

41. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm41 Pfields Advantages –Easy to visualize –Easy to build up software libraries –Fields can be parameterized –Combination mechanism is fixed, tweaked with gains Disadvantages –Local minima problem (sum to magnitude=0) –Jerky motion

42. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm42 Example: Docking Behavior Arkin and Murphy, 1990, Questa, Grossmann, Sandini, 1995, Tse and Luo, 1998, Vandorpe, Xu, Van Brussel, 1995. Roth, Schilling, 1998, Santos-Victor, Sandini, 1997 Orientation, ratio of pixel counts  tangent vector Total count  attraction vector Selective attraction field, width of +-45 degrees

43. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm43 Docking Behavior Video

44. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm44 Class Discussion: When Does a Field End? Imagine the case of a “SodaPup” robot (MIT) –task: find and pick up a Coca-Cola can –environment: red cans are only red object in world

45. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm45 Pfield advantages Easy to visualize Easy to combine Can be parameterized (different ranges, drop off, etc.)

46. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm46 disadvantages Local minima –Solutions: Add noise Navigation templates Express potential fields as harmonic functions – no local minima of 0, but computationally expensive

47. 4 Introduction to AI Robotics (MIT Press)Chapter 4: The Reactive Paradigm47 Pfields Summary Reactive Paradigm: SA, sensing is local –Solves the Open World problem by emulating biology –Eliminates the frame problem by not using any global or persistent representation –Perception is direct, ego-centric, and distributed Two architectural styles are: subsumption and pfields Behaviors in pfield methodologies are a tight coupling of sensing to acting; modules are mapped to schemas conceptually Potential fields and subsumption are logically equivalent but different implementations Pfield problems include –local minima (ways around this) –jerky motion –bit of an art