1. Designing a Reactive Implementation
List the steps involved in designing a reactive behavioral system.
Use schema theory to program behaviors using object-oriented programming principles.
Design a complete behavioral system, including coordinating and controlling multiple concurrent behaviors.
Describe the two methods for assembling primitive behaviors into abstract behaviors: finite state machines and scripts.
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Pop quiz: fill in a behavior table for phototropism (Lab #4)
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

2. Behaviors as objects in OOP
Schema is a class with optional coordinated control program
Perceptual schema, motor schema and behavior are all subclasses of schema (class diagram p. 157)
Behavior has (class diagram p. 157)
One or more perceptual schemas
One or more motor schemas
One or more behaviors
Primitive behavior has only one perceptual schema, one motor schema, no coordinated control program; often prgrammed as a single method
Behaviors that are assembled from other behaviors or have multiple perceptual and motor schemas are abstract behaviors (not the same as abstract class in OOP)
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

3. Example: primitive move-to-goal behavior
Robot placed in empty office with red coke cans and white cups placed randomly, blue recycling bins in two corners, different colored trash bin in 2 corners. Goal: pick up the most trans and place in correct bin; red and blue are more easily perceived, so typical strategy was t recycle coke cans first
Basic behavior: move_to_goal(goal_color)
Note: PFields is a class with 5 primitive potential fields, serves as a library; percept is local to behavior
Object
Behavioral analog
identifier
Data
Percept
Goal_angle
Goal_strength
Methods
Perceptual_schema
Motor_schama
Extract_goal(goal_color)
Pfields.attraction(goal_angle, goal_strength)
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

4. Example: abstract follow-corridor behavior
Class diagrams for 2 different approaches shown on p. 161
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

5. Chapter 5: Designing a Reactive Implementation
Releasers in OOP
Releaser is a perceptual schema not bound to a motor schema
Two approaches:
Look for red with extract_color, signal main program when red is seen, main instantiates move_to_goal(red)
Let behavior always be active, but if red is not seen, behavior returns identity function (0,0,0) for potential field
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

6. Steps in Designing a Reactive Behavioral System
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Have them note the similarities to the Waterfall Life Cycle in software engineering
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

7. Chapter 5: Designing a Reactive Implementation
CSM 1994 UGV Competition
(this is from
1996, but it’s
a way better
picture)
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Fully autonomous vehicle, all on-board, navigate a course 10ft wide, about 800ft long
Obstacles
Carry a 20 pound payload
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

8. 1-3. Describe the Task, Robot, Environment
Camcorder on a
Panning mast, going to
A framegrabber
33MHz 486 PC running
Lynx (commercial unix)
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Sonar on a panning
mast
Note the hardware limitations: b/w framegrabber and ridiculously slow processor by today’s standards. Good time to discuss Moore’s Law.
Have to use computer vision… black &white, slow processor speeds (didn’t get to design the robot)
White (bright) should be in the center of the image
Reflections on grass are white, but random so average out
If stay in middle, never encounter an obstacle!
150ms update rate needed for steering to stay in control at ~1.5mph
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

9. 4. Describe how robot should act
Follow the line and stay in the middle
Follow-line
Only need the Camcorder!
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

10. 5-6 Refine and test each behavior
Follow-line
Worked with toilet paper (indoors),
Worked with athletic line tape (outdoor)
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

11. 7 Test with other behaviors
“Full dress rehearsal”
Oops, bales of hay are bright compared to grass, change the centroid to cause collision
Go back to step 4:
Follow line until “see” an obstacle, then just go straight until things return to normal
Hard to do visually in real time
Sonar! Look to the side and when something is close, it’s a bale, so go straight
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

12. Chapter 5: Designing a Reactive Implementation
Final System
Round 1
OOPS: sonar connection off so it hit the bale
Round 2
White shoes and dandelions, plus Killer Bale
Demo round
Hill vs. implicit flat earth assumption
Round 3
Trapped by sand, but $5K richer!
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

13. Chapter 5: Designing a Reactive Implementation
Main Points
Let the WORLD BE ITS OWN BEST REPRESENTATION and CONTROL STRUCTURE
“line” wasn’t a “line” just centroid of brightest pixels in the image
Pick up trash: if can is in gripper, then go to the recycle bin
Design process was iterative; rarely get a workable emergent behavior on the first try
There is no single right answer
Could have been done with subsumption, pfields, rules, whatever
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

14. Chapter 5: Designing a Reactive Implementation
Other Lessons Learned
Don’t count on getting much work done in the field!
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
“A river runs through it…”
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

15. Case Study and Field Trip: Polar Robotics
Existing robots
DANTE, DANTE II
Nomad
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

16. Chapter 5: Designing a Reactive Implementation
DANTE
Task
explore volcanoes in Antarctica (reduce risk to vulcanologists)
Environment
steep, rocky slopes
Robot
legged “framewalker”
Behaviors
rappel down
Refine and test
Test with other behaviors
1997 South America, failed
1998 Antarctica, med. success
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

17. Chapter 5: Designing a Reactive Implementation
Lessons to Be Learned
A robot is part of a system, must design system
example tether breaking
Testing and evaluation really does take ~30% of the time allocated (just like software engineering)
and more to fix the problem
NASA technical readiness levels
Level 1: just an idea
Level 6: demonstrated on a realistic test bed
Level 9: has flown, fully operational
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

18. Chapter 5: Designing a Reactive Implementation
Nomad
Task
search for and analyze metorities
Environment
Antarctica
Robot
hybrid mobility, LADAR (laser radar), omnivision, sample sensors
Behaviors
search an area using GPS, avoid negative/positive obstacles, stop at possible metorities and alert user. Can be controlled by human
Refine and Test
Test with other behaviors
1999
2000 success
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

19. Chapter 5: Designing a Reactive Implementation
Nomad Omnivision
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

20. Hyperion (really about Mars)
Task
navigate Arctic
Environment
Arctic
Robot
wheeled, uses solar
Behaviors
Refine and test
Test with other behaviors
2002 success
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

21. NASA HMP / Mars Society Flashline Project Robotics Support
Confined space inspection of Mars habitat
Marsupial deployment for fault identification
Human robot team for collaborative exploration of
hazardous areas & narrow fissures in search of micro-organisms
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

22. Case Study in Pfields: Docking
Needed for marsupial robots
Note
use of reactivity
simple perception
pfields
improvement over human
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

23. Daughter-centric Docking Behavior
Potential field
methodology:
orientation,
Xbar=
tangent vector
overall size=
attraction vector
(Arkin, Murphy 90)
Silver Bullet:
AMD K-6 400MHz
clearance=+/-2cm
+/-5 degrees
Bujold: Inuktun mVGTV
Pan only, color camera,
optics at about 2ms,
skid steering,
no on-board processor
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

24. Docking Behavior Results
48 data points (3 per 16 stations) on time to completion
by angle
by radial position
Larger angles, the slower, and farther away the slower
2 failures: 3, 7
Fastest: 26 sec.
Slowest: 81 sec.
Pfields weren’t
intended to work
behind the
gate
Optics dictated 120 deg, 2m
Approach Zone
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

25. Docking Behavior Results: teleoperation
17% faster
Performance comparison:
- 66 data points, 22 teleoperators
- paired sample t-test (conf>95%)
Communications data:
-36 data points, 6 teleoperators
Teleoperators did better only at 1,3 where the robot wasn’t
supposed to be able to dock from. Noticed that teleoperators
used a different strategy; opportunity for improvement
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

26. Chapter 5: Designing a Reactive Implementation
Docking Vision
Spherical Coordinate Transform space instead of HSV, RGB
Biomimetic vision: gradient, looming
O(m x n) lighting insensitive color segmentation
8-10 frames/sec on commercial hardware vs. 30 frames/sec for Cognachrome hardware
Robust algorithm
2-colored landmark to prevent confusion
imprinting and time of day look up tables (LUT)
if time-out, mother moves
a color is a
circle on the plane
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

27. Light insensitive color segmentation
LUT
(time of day,
weather)
>15 tries
Compute
region
statistics
Perform
connected
components
Initial
values
Extract
regions
Imprint
Adapt
new values for region
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

28. Indoor Data Collection and Demonstrations
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

29. Outdoor Demonstrations
Imprint at dawn (light mist), return 6 hours later
As expected, mediocre performance
from dusk to night (~50%) due to
headlight reflection
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

30. Additional Case Studies: Commercial Products
Robomow
cutting grass in residential areas
My Real Baby
toy dolls
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

31. Chapter 5: Designing a Reactive Implementation
Robomow
Behaviors?
Random
Avoid
Avoid(bump=obstacle)
Avoid(wire=boundary)
Stop
Stop(tilt=ON)
All active
Overview
History
Reactive
USAR
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

32. Chapter 5: Designing a Reactive Implementation
My Real Baby
Behaviors?
Touch-> Awake
Upside down & Awake-> Cry
Awake & Hungry -> Cry
Awake & Lonely -> Cry
Note can get crying from multiple behaviors
Note internal state (countdown timer on Lonely)
Overview
History
Reactive
USAR
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

33. Commercial Case Studies
Purely reactive
May not need subsumption or pfields, very straightforward
State of practice is much less complicated than state of art!
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

34. Design Tool: Behavior Table
An agent is attracted to light. If it sees light, it heads in that direction.
If it encounters an obstacle, it turns either left or right, favoring the
direction of the light. If there is no light, it sits and waits. But if an
obstacle appears, the agent runs away.
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
1. photropism
2. Obstacle avoidance
This is using the lab #4 on subsumption behaviors.
Note that atLight is really part of move2light, since it’s the STOPPING CASE for move2light
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

35. Chapter 5: Designing a Reactive Implementation
Behavior Table
An agent is attracted to light. If it sees light, it heads in that direction.
If it encounters an obstacle, it turns either left or right, favoring the
direction of the light. If there is no light, it sits and waits. But if an
obstacle appears, the agent runs away.
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Releaser
Behavior
Motor Schema
Percept
Perceptual Schema
Light
phototropism
move2Light():Attraction
Light: direction & strength
Brightest(dir), atLight()
Range
<tasked level>
Obstacle avoidance
avoid(): turn left or right;
runaway()
proximity
Obstacle()
Notice that the perceptual schemas were hidden in the previous drawing.
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

36. Chapter 5: Designing a Reactive Implementation
Robomow
Releaser
Behavior
Motor Schema
Percept
Perceptual Schema
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Notice that the perceptual schemas were hidden in the previous drawing.
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

37. Chapter 5: Designing a Reactive Implementation
My Real Baby
Releaser
Behavior
Motor Schema
Percept
Perceptual Schema
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Notice that the perceptual schemas were hidden in the previous drawing.
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

38. Assemblages of Behaviors
What to do if have a SEQUENCE of behaviors as well as CONCURRENCY?
Two equivalent methods
Finite state automata (FSA)
Scripts
(other ways too)
Two approaches:
Stuff the coordination mechanism into a separate controlling program (like a main())
Hurts portability, adding new behaviors “on top”
Stuff the coordination mechanism into a meta-behavior or abstract behavior using recursive-ness of schemas
Coordination mechanism is the coordinated control program part of the behavioral schema
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
They should have generated a scenario for what they want the robot to do
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

39. Example of a Sequence: Catch Suzy!
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

40. Chapter 5: Designing a Reactive Implementation
Example Continued
“Tracking Phase”
move to goal
avoid obstacle
“Pick up Phase”
fine position
pick up
“Return Home Phase”
same behaviors,
just DIFFERENT
INSTANTIATION OF
GOAL
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

41. Chapter 5: Designing a Reactive Implementation
FSA: M={K,S,d,s,F}
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
K: all the states, each is “q”- behaviors
d: transition function, d(q,s)= new behavior
S: inputs that agent can “see”, each is s– stimulus/affordances
q0: Start state(s)- part of K
F: Terminating state(s)- part of K
have them fill in the table: q, small sigma, delta
Behavior table p. 175
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

42. Class Exercise: Catch Suzy
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

43. Chapter 5: Designing a Reactive Implementation
EX: Pick Up Trash
Behavior table p. 179
FSA p. 181
Abstract behavior using subsumption p. 183
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

44. Chapter 5: Designing a Reactive Implementation
FSA Summary
“State” doesn’t mean “state of world” (bad); it means more “where the innate releasing mechanism is currently at” (good)
Advantages
Formal mechanism
Have to fill in the table, so a way of spotting unforeseen effects
Disadvantages
Hard to keep up with implicit behaviors
Ex. Avoid obstacle is often running throughout ALL states, gets tedious to express it formally
Tend to add explicit variable for transitions, rather than rely on emergent behavior from environment
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

45. Chapter 5: Designing a Reactive Implementation
Scripts
UGV competition really didn’t have a sequence of behaviors, just toggling back and forth
Scripts are equivalent to FSA but may be more natural or readable for sequences
Think of robot and task in terms of a screenplay
Resulting script for an abstract behavior is usually the same as the programming logic derived from FSA
Ex: pick up trash, p
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

46. Chapter 5: Designing a Reactive Implementation
Scripts
Script
Behavior Analog
Examples
Goal
Task
Find victims in rubble
Places
Environment, applicability or
“taskability” for new tasks
Collapsed buildings
Actors
Behaviors
explore(), dance(), avoid(), crawl, move2void, move2victim
Props, cues
Percepts
Voids: Dark, concave
Victims: heat, motion, color
Causal Chain
Sequence of behavior
Explore, dance, move2void, crawl, move2victim, dropRadio
Subscripts
Exception handling
If lose communications, return home
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

47. Chapter 5: Designing a Reactive Implementation
Scripts Summary
Will see more of it in Ch10: Topological Navigation
Advantages
A more storyboard like way of thinking about the behaviors
If-then, switch style of programming like FSA
Since a Script is “in” a behavior, other behaviors such as avoid can be running concurrently without having to appear “in” the script
Exception handling is a big plus in Real Life
Disadvantages
Can be a bit of overkill for simple sequences, especially with C++
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation

48. Chapter 5: Designing a Reactive Implementation
Summary
Design of reactive systems is identical to design of object-oriented software systems
Highly modular, can test behaviors independently
Follows the basic steps in the Waterfall Lifecycle
Describe task, robot, environment, how robot should act, refine behaviors, test independently, test together.
Except lots more iteration, making it more like Spiral
Behavior tables can help keep track of what’s a behavior and its releasers and percepts
Sequences of behaviors, rather than combinations, can be controlled by a separate routine or by adding a routine to the coordinated control program in the behavioral schema
FSA and Scripts are two main methods
Design
-Beh. Table
-Steps
Case Study
Coordination
-FSA
-Scripts
Summary
Introduction to AI Robotics (MIT Press)
Chapter 5: Designing a Reactive Implementation