1. Chapter 4: The Reactive Paradigm
Describe the Reactive Paradigm in terms of the 3 robot primitives and its organization of sensing
List the characteristics of a reactive robotic system, and discuss the connotations of surrounding the reactive paradigm
Describe the two dominant methods for combining behaviors in a reactive architecture: subsumption and potential field summation
Be able to program a behavior using pfields
Be able to construct a new potential field from primitive pfields and sum pfields to generate an emergent behavior
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

2. Review: Lessons from Biology
Programs should decompose complex actions into behaviors. Complexity emerges from concurrent behaviors acting independently
Agents should rely on straightforward activation mechanisms such as IRM
Perception filters sensing and considers only what is relevant to the task (action-oriented perception)
Behaviors are independent but the output may be used in many ways including: combined with others to produce a resultant output or to inhibit others
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

3. Hierarchical Organization is “Horizontal”
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

4. More Biological is “Vertical”
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

5. Sensing is Behavior-Specific or Local
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
Behaviors can “share” perception without knowing it
This is behavioral sensor fusion
Ask students what’s the difference between this and the layers in RCA…
These aren’t layers, they’re concurrent behaviors. The layers in RCA reflected a
Commitment to partitioning the world model, here any behavior can have access to any perception.
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

6. Chapter 4: The Reactive Paradigm
Reactive Robots
RELEASER
behavior
SENSE
ACT
Overview
History
Reactive
USAR
Summary
Most apps are programmed with this paradigm
Biologically based:
Behaviors (independent processes), released by perceptual or internal events (state)
No world models or long term memory
Highly modular, generic
Overall behavior emerges
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

7. Chapter 4: The Reactive Paradigm
Example 1: Robomow
Behaviors?
Random
Avoid
Avoid(bump=obstacle)
Avoid(wire=boundary)
Stop
Stop(tilt=ON)
All active
Overview
History
Reactive
USAR
Summary
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

8. Chapter 4: The Reactive Paradigm
Example 2: My Real Baby
Behaviors?
Touch-> Awake
Upside down & Awake-> Cry
Awake & Hungry -> Cry
Awake & Lonely -> Cry
Note can get crying from multiple behaviors
Note internal state (countdown timer on Lonely)
Overview
History
Reactive
USAR
Summary
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

9. Chapter 4: The Reactive Paradigm
Historically, there are two main styles of creating a reactive system
Subsumption architecture
Layers of behavioral competence
How to control relationships
Potential fields
Concurrent behaviors
How to navigate
They are equivalent in power
In practice, see a mixture of both layers and concurrency
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

10. Subsumption: Rodney Brooks
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
From
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

11. Subsumption Philosophy
Modules should be grouped into layers of competence
Modules in a higher lever can override or subsume behaviors in the next lower level
Suppression: substitute input going to a module
Inhibit: turn off output from a module
No internal state in the sense of a local, persistent representation similar to a world model.
Architecture should be taskable: accomplished by a higher level turning on/off lower layers
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

12. Chapter 4: The Reactive Paradigm
runaway 0
wander 1
follow-corridor 2
Level 0: Runaway
HALT
COLLIDE PS MS
RUN AWAY
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

13. Example Perception: Polar Plot
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
if sensing is ego-centric, can
often eliminate need for
memory, representation
Plot is ego-centric
Plot is distributed (available to whatever wants to use it)
Although it is a representation in the sense of being a data structure, there is no memory (contains latest information) and no reasoning (2-3 means a “wall”)
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

14. Chapter 4: The Reactive Paradigm
runaway 0
wander 1
follow-corridor 2
Level 1: Wander
encoders
AVOID PS MS
WANDER
Note sharing of
Perception, fusion
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
What would
Inhibition do?
Avoid is needed because would tend to runaway to areas already visited, never make any progress
Green oval: why suppression? What would inhibition do?
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

15. Chapter 4: The Reactive Paradigm
Class Exercise
runaway 0
wander 1
move2light 2
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
LIGHT
PHOTO-
TROPHISM S
In class question: Why suppression rather than inhibition?
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

16. Level 2: Follow-Corridors
runaway 0
wander 1
follow-corridor 2
Level 2: Follow-Corridors
STAY-IN-MIDDLE PS MS
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

17. Chapter 4: The Reactive Paradigm
Class Exercise
Design the roomba with subsumption
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

18. Chapter 4: The Reactive Paradigm
Subsumption Review
What is the Reactive Paradigm in terms of primitives?
What is the Reactive Paradigm in terms of sensing?
Does the Reactive Paradigm solve the Open World problem?
How does the Reactive Paradigm eliminate the frame problem?
What is the difference between a behavior and a level of competence?
What is the difference between suppression and inhibition in subsumption?
Review
Organization
-SA
-beh. specific
Subsumption
-Philosophy
-Level 0
-Level 1
-Level 2
Summary
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

19. Potential Fields: Ron Arkin
From
From
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

20. Potential Fields Philosophy
The motor schema component of a behavior can be expressed with a potential fields methodology
A potential field can be a “primitive” or constructed from primitives which are summed together
The output of behaviors are combined using vector summation
From each behavior, the robot “feels” a vector or force
Magnitude = force, strength of stimulus, or velocity
Direction
But we visualize the “force” as a field, where every point in space represents the vector that it would feel if it were at that point
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

21. Example: Run Away via Repulsion
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

22. 5 Primitive Potential Fields
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

23. Draw These Now! Common fields in behaviors
Uniform
Move in a particular direction, corridor following
Repulsion
Runaway (obstacle avoidance)
Attraction
Move to goal
Perpendicular
Corridor following
Tangential
Move through door, docking (in combination with other fields)
random
do you think this is a potential field? what would it look like?
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

24. Chapter 4: The Reactive Paradigm
Class Exercise
Name the field you’d use for
Moving towards a light
Avoiding obstacles
Attractive
Repulsive
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

25. Combining Fields for Emergent Behavior
obstacle
goal
obstacle
goal
obstacle
If robot were dropped anywhere on this grid,
it would want to move to goal and avoid obstacle:
Behavior 1: MOVE2GOAL
Behavior 2: RUNAWAY
The output of each independent behavior is a vector,
the 2 vectors is summed to produce emergent behavior
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

26. Fields and Their Combination
Note: in this example, robot can sense the
goal from 10 meters away
Note: In this example, repulsive field only extends for 2 meters;
the robot runs away only if obstacle within
2 meters
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

27. Chapter 4: The Reactive Paradigm
Path Taken
Robot only feels
vectors for this
point when it (if)
reaches that point
If robot started at this location, it would take the following path
It would only “feel”the vector for the location, then move accordingly, “feel” the next vector, move, etc.
Pfield visualization allows us to see the vectors at all points, but robot never computes the “field of vectors” just the local vector
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

28. Chapter 4: The Reactive Paradigm
Discussion
Could you represent the Arctic Tern feeding behavior with potential fields?
what happens with multiple red dots?
can you inhibit with potential fields?
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

29. Example: follow-corridor or follow-sidewalk
Perpendicular
Uniform
Note use of
Magnitude profiles:
Perpendicular decreases
Combined
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

30. Chapter 4: The Reactive Paradigm
Class Exercise: Draw Fields for Wall-Following (assume that robot stands still if no wall)
Just half of a follow-corridor, but…
If the stand-off distance is the same as the detection range, the robot is likely to lose sight of the wall
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

31. Chapter 4: The Reactive Paradigm
But how does the robot see a wall without reasoning or intermediate representations?
Perceptual schema “connects the dots”, returns relative orientation
orientation
MS: Perp.
PS:
Find-wall S
MS: Uniform
Sonars
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

32. OK, But why isn’t that a representation of a wall?
It’s not really reasoning that it’s a wall, rather it is reacting to the stimulus which happens to be smoothed (common in neighboring neurons)
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

33. Chapter 4: The Reactive Paradigm
Level 0: Runaway
Note: multiple instances of
a behavior vs. 1:
Could just have 1
Instance of RUN AWAY,
Which picks nearest reading;
Doesn’t matter, but this
Allows addition of another
Sonar without changing the
RUN AWAY behavior
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

34. Chapter 4: The Reactive Paradigm
Level 1: Wander
Wander is
Uniform, but
Changes direction
aperiodically
Green oval: why suppression? What would inhibition do?
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

35. Level 2: Follow Corridor
Should we
Leave
Run Away
In? Do we
Need it?
Green oval: why suppression? What would inhibition do?
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

36. Chapter 4: The Reactive Paradigm
Pfields
Advantages
Easy to visualize
Easy to build up software libraries
Fields can be parameterized
Combination mechanism is fixed, tweaked with gains
Disadvantages
Local minima problem (sum to magnitude=0)
Box canyon problem
Jerky motion
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

37. Example: Docking Behavior
Arkin and Murphy, 1990, Questa, Grossmann, Sandini, 1995, Tse and Luo, 1998, Vandorpe, Xu, Van Brussel, Roth, Schilling, 1998, Santos-Victor, Sandini, 1997
Orientation, ratio of pixel counts  tangent vector
Total count  attraction vector
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

38. Docking Behavior Video
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

39. Class Discussion: When Does a Field End?
Imagine the case of a “SodaPup” robot (MIT)
task: find and pick up a Coca-Cola can
environment: red cans are only red object in world
behavior: ??
Remind them about Jon Connell forgetting to inhibit– so it would drop off the red coke can, then see it and pick it up again– an infinite loop!
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm

40. Chapter 4: The Reactive Paradigm
Pfields Summary
Reactive Paradigm: SA, sensing is local
Solves the Open World problem by emulating biology
Eliminates the frame problem by not using any global or persistent representation
Perception is direct, ego-centric, and distributed
Two architectural styles are: subsumption and pfields
Behaviors in pfield methodologies are a tight coupling of sensing to acting; modules are mapped to schemas conceptually
Potential fields and subsumption are logically equivalent but different implementations
Pfield problems include
local minima (ways around this)
jerky motion
bit of an art
Introduction to AI Robotics (MIT Press)
Chapter 4: The Reactive Paradigm