1. Robot Vision Today: Reactive Control & Vision
Next Time: Localization & Navigation
(or Where am I and How to get there?
Camera-carrying robot enters Great Pyramid

2. Remote-Controlled Rats
Spectrum of Control
Teleoperation: Human Control
Autonomous (AI) Control
Shared Human – Robot Control
Remote-Controlled Rats

3. Reactive/Behavior-Based Control
Sense
Act
Ignores world models
“The world is its own best model”
Tightly couples perceptions to actions
No intervening abstract representations
Primitive Behaviors are used as building blocks
Individual behaviors can be made up of primitive behaviors
Reactive: no memory
Behavior-Based: Short Term Memory (STM)

4. Reactive/Behavior-Based Control Design
Design Considerations
What are the primitive behaviors?
What are the individual behaviors?
Individual behaviors can be made up of primitive and other individual behaviors
How are behaviors grounded to sensors and actuators?
How are these behaviors effectively coordinated?
If more than one behavior is appropriate for the situation, how does the robot choose which to take?

5. Design for robot soccer
What primitive behaviors would you program?
What individual behaviors?
What situations does the robot need to recognize
If the “pass behavior” is active and the “shoot behavior” is active, how does it choose?

6. Situated Activity Design
Robot actions are based on the situations in which it finds itself
Robot perception is characterized by recognizing what situations it is in and choosing an appropriate action

7. Implementing Behaviors
Schema: knowledge + process
Perceptual Schema :interpretation of sensory data
Releasers :instantiates motor schema
Motor Schema: actions to take.

8. Schema for Toad Feeding Behavior

9. Design of Behaviors represented by a State Transition Table
q Є K
Set of states (behaviors)
σ Є Σ
Set of releasers δ
Transition function s
State Robot starts in
q Є F
Set of terminating states
Trash Pick-up Example

10. Visual Representation in a Finite State Automata

11. Cooperative Coordination
Behavioral Fusion
Requires the ability to concurrently use the output of more than one behavior at a time
Consider what happens when a toad sees two flies
Behavior Fusion via vector summation

12. Competitive Coordination
Action Selection Method
Behaviors compete using an activation level
The response associated with the behavior with the highest activation level wins
Activation level is determined by attention (sensors) and intention (goals)

13. Competitive Coordination
Suppression Network Method
Response is determined by a fixed prioritization in which a strict behavioral dominance hierarchy exists.
Higher priority behaviors can inhibit or suppress lower priority behaviors.

14. Subsumption Architecture
A suppression network architecture built in layers
Each layer gives the system a set of pre-wired behaviors
Layers reflect a hierarchy of intelligence.
Lower layers are basic survival functions (obstacle avoidance)
Higher layers are more goal directed (navigation)
The layers operate asynchronously (Multi-tasking)
Lower layers can override the output from behaviors in the next higher level
Rank ordering

15. Foraging Example

16. More Complex Example: Robot Follow a Corridor

17. Using Multiple Behaviors can require the Robot to Multi-task
Multi-tasking is having more than one computing processing run in parallel.
True parallel processing requires multiple CPU’s.
IC functions can be run as processes operating in parallel The computer processor is actually shared among the active processes
main is always an active process
Each process, in turn, gets a slice of processing time (5ms)
Each process gets its own default program stack of 256bytes
A process, once started, continues until it has received enough processing time to finish (or until it is “killed” by another process)
Global variables are used for inter-process communications

18. IC: Functions vs. Processes
Functions are called sequentially
Processes can be run simultaneously
start_process(function-call);
returns a process-id
processes halt when function exits or parent process exits
processes can be halted by using
kill_process(process_id);
hog_processor(); allows a process to take over the CPU for an additional 250 milliseconds, cancelled only if the process finishes or defers
defer(); causes process to give up the rest of its time slice until next time
More info:

19. Example.ic The robot looks left and right
If it sees RED to one side it turns to face it
If it sees RED ahead it beeps
If the stop button is pressed it plays a song and quits

20. Reactive: Good & Bad Works with the Open World Assumption
Provides a timely response in a dynamic environment where the environment is difficult to characterize and contains a lot of uncertainty.
Unpredictable
Low level intelligence
Cannot manage tasks that require LTM or planning
Tasks requiring localization and order dependent steps

21. Computer Vision Uses the electromagnetic spectrum to produce an image.
Visible light, x-rays, thermal, infrared

22. Representation
Image: picture like format where there is a direct physical correspondence to the scene being viewed.
Implies there are multiple readings in a grid
Pixels: “picture element”
Measure depends on the type of spectra being used
Image function: converts a signal to a pixel value

23. CCD Cameras Charged-Couple device: detects visible light
Light fall on an array of metal-oxide semiconductor capacitor (MOS)
Line transfer or frame transfer
A/D conversion
Slow frame rate
Frame buffers
Framegrabber

24. Representations Grayscale 8-bit 256 discrete gray values
0 black, 255 white

25. Representations Different method for representing color
RGB space: red, green, blue
HSI space: hue, saturation, and intensity
Hue is the dominant wavelength
Saturation is the lack of whiteness
Intensity is the quantity of light
Linear transformation between RGB and HSI

26. Comparison of Region Segmentation

27. RGB
Color Space
24 bit color (8 bits per color)

28. RGB Representations Interleaved Separate RGB values stored together
Red = image[row][col][0]
Green = image[row][col][1]
Blue = image[row][col][2]
Separate
RGB values stored as separate 2D arrays
Red = image_red[row][col]
Green = image_green[row][col]
Blue = image_Blue[row][col]

29. Region Segmentation
Identifying a region in an image with a particular color
Thresholding
Binary image

30. Region Segmentation

31. cmucamlib Routines To use CMU camera routines put
#use "cmucamlib.ic" at the top of your file
Call init_camera(); to initialize camera before any other camera calls -- will beep and complain if HB cannot talk to camera (check the dongle switch)
Use clamp_camera_yuv(); to automatically set camera for the current lighting conditions. Camera should be pointed at a white surface when this call is being made (it waits for start button to be pressed). It takes 15 seconds for this function to complete!

32. cmucamlib Routines (cont.)
Call track_blue(); & track_orange(); to check for color blobs that CMUcam can see. These functions return 0 if they find no color blob, or the confidence of the blob detected. A good confidence is 80 and up. A confidence of 4 or 5 is poor.

33. cmucamlib Routines (cont.)
The track_color information is stored in globals:
track_size stores the approximate number of pixels matching in the blob
track_x stores the pixel x coordinate of the centroid of the color blob
track_y stores the pixel y coordinate of the color blob (note: 0,0 is the center; 40,80 is upper right and
-40,-80 is lower left)
track_area stores the size of the bounding rectangle of the color blob
track_confidence stores the confidence for seeing the blob

34. cmucamlib Routines (cont.)
For experts: use trackRaw(…); to specify a particular color for tracking. Returns 0 if no such blob is found, -1 is there is a communication error, or the confidence. Also check out setWin(…);
More details and low level functions are given in the comments at the beginning of cmucamlib.ic and cmucam3.ic

35. Example: cmucamlib-demo.ic
/* demonstrate color blob sensing for poof balls and blue paper */
#use "cmucamlib.ic"
void main() {
init_camera(); // initialize the camera in YUV mode
clamp_camera_yuv(); // clamp camera white balance in YUV mode
while(!stop_button()) { // hold down Stop for a long time
if (track_blue() > 4) { // you could make this 0 bigger
// number, like 80 for example
printf("blue found:%d\n", track_confidence);
} else
if (track_orange() > 4) {
printf("orange found:%d\n", track_confidence);
} else {
beep();
printf("nothing...\n"); }
} // end while()
} // end main()

36. CMUcamGUI