Common Sensing Techniques for Reactive Robots ( ) Sungmin Lee ( 이성민 ) Division of Electronic Engineering, Chonbuk National University Intelligent Systems & Robotics Lab.

Chapter objectives Describe difference between active and passive sensors Describe the types of behavioral sensor fusion Define each of the following terms in one or two sentences: proprioception, exteroception, exproprioception,proximity sensor, logical sensor, false positive, false negative, hue, saturation, computer vision Describe the problems of specular reflection, cross talk, foreshortening, and if given a 2D line drawing of surfaces, illustrate where each of these problems would be likely to occur If given a small interleaved RGB image and a range of color values for a region, be able to 1) threshold on color and 2) construct a color histogram

Contents 1.Logical sensors 2.Behavioral Sensor Fusion 3.Attributes of a sensor 4.Sensor Categories 5.Computer vision 6.Case study 7.Summary

Motivation Sensing is tightly coupled with acting in reactive systems, so need to know about sensors What sensors are out there? –Ultrasonics, cameras are traditional favorites –Sick laser ranger is gaining fast in popularity How would you describe them (attributes)? How would you decide which one to pick and use for an application?

Logical Sensors A unit of sensing or module (supplies a particular percept). It consists of the signal processing and the software processing. Can be easily implemented as a perceptual schema. Different sensors/perceptual schemas can produce the same percept - motor schema doesn’t care! –Behavior can pick what’s available Example: ring of IRs, ring of sonars –If sensor fails, then another can be substituted without deliberation or explicit modeling –Conflicts in allocation can be solved by using logical sensors (deliberation is required to assign)

Active vs. Passive (Example) Active sensors - Sensor emits some form of energy and then measures the impact as a way of understanding the environment - Ex. Ultrasonics, laser Passive sensors - Sensor receives energy already in the environment - Ex. Camera Passive consume less energy, but often signal-noise problems Active often have restricted environments Stereo Camera pair Thermal sensor Laser ranger Sonars Bump sensor

Behavioral Sensor Fusion Sensor fusion is a broad term used for any process that combines information from multiple sensors into a single percept. In some cases multiple sensors are used when a particular sensor is too imprecise or noisy to give reliable data. Adding a second sensor can give another “vote” for the percept. When a sensor leads the robot to believe that a percept is present, but it is not, the error is called a false positive. The robot has made a positive identification of percept, but it was false. Likewise, an error where the robot misses a percept is known as a false negative. False positive False negative

Sensing Model 11 Sensor/TransducerBehaviorAction

Sensing in Reactive Paradigm Each behavior has its own dedicated sensing. One behavior literally does not know what another behavior is doing or perceiving. Behavior

Perceptual Schemas Motor Schemas Behavioral Sensor Fusion: -sensor fission This sensor fission in part as a take off on the connotations of the word “fusion” in nuclear physics. In nuclear fusion, energy is created by forcing atoms and particles together, while in fission, energy is creating by separating atoms and particles.

Perceptual Schema Motor Schema Behavioral Sensor Fusion: -action-oriented sensor fusion This type of sensor fusion is called action-oriented sensor fusion to emphasize that the sensor data is being transformed into a behavior-specific representation in order to support a particular action, not for constructing a world model.

Perceptual Schema Motor Schema Behavioral Sensor Fusion: -sensor fashion Sensor fashion, an alliterative name intended to imply the robot was changing sensors with changing circumstances just as people change styles of clothes with the seasons.

Designing a Sensor Suite -Attributes of a sensor Field of view, range : does it cover the “right” area Accuracy & repeatability : how well does it work? Responsiveness in target domain : how well does it work for this domain? Power consumption : may suck the batteries dry too fast Reliability : can be a bit flakey, vulnerable Size : always a concern! Computational Complexity : can you process it fast enough? Interpretation Reliability : do you believe what it’s saying?

Should be considered for the entire sensing suite : Simplicity Modularity Redundancy - physical redundancy ( there are several instances of physically identical sensors on the robot.) -logical redundancy (another sensor using a different sensing modality can produce the same percept or releaser.) - fault tolerance Designing a Sensor Suite -Attributes of a sensor suite

Sensor Categories Proprioceptive –Inertial Navigation System(INS) –Global Positioning System(GPS) Exteroceptive –Proximity Range Contact –Computer Vision

Proprioceptive Sensors(1) -Inertial navigation system (INS) Measure movements electronically through miniature accelerometers INS can provide accurate dead reckoning to 0.1 percent of the distance traveled. However, this technology is unsuitable for mobile robots for several reasons.(cost, Size, etc) MQ-9 Reaper

GPS systems work by receiving signals from satellites orbiting the Earth. GPS is not complete solutions to the dead reckoning problem in mobile robots. GPS does not work indoors(environmental limit) Proprioceptive Sensors(2) -Global Positioning System (GPS)

Proximity Sensors(1) -Sonar or ultrasonic Sonar refers to any system for using sound to measure range. (use a sonar for underwater vehicles ). Ground vehicles commonly use sonar with an ultrasonic frequency. Ultrasonic sensors generate high frequency sound waves and evaluate the echo which is received back by the sensor. Sensors calculate the time interval between sending the signal and receiving the echo to determine the distance to an object. Ultrasonic is possibly the most common sensor on commercial robots operating Polaroid ultrasonic transducer

chairs, tables– legs, edges too thin for resolution Proximity Sensors(1) - Three problems with sonar range readings foreshortenin g cross-talk specular reflection

Maps produced by a mobile robot using sonars in: a.) a lab and b.) a hallway. (The black line is the path of the robot.) Proximity Sensors(1) - sonar maps labhallway

Power consumption –High Reliability –Lots of problems Size –Size of a Half dollar, board is similar size and can be creatively packaged Computational Complexity –Low; doesn’t give much information Interpretation Reliability –poor Proximity Sensors(1) - Attributes of ultrasonic

Physics : active sensor, works on time of flight Advantages : range, inexpensive ($30 US), small Disadvantages : specular reflection, crosstalk, foreshortening, high power consumption, low resolution Proximity Sensors(1) - Ultrasonic Summary

They emit near-infrared energy and measure whether any significant amount of the IR light is returned. These often fail in practice because the light emitted is often “washed out” by bright ambient lighting or is absorbed by dark materials (i.e., the environment has too much noise). Proximity Sensors(2) - Infrared ray (IR) Sharp GP2Y0A21YK

Popular class of robotic sensing is tactile, or touch, done with bump and feeler sensors. The sensitivity of a bump sensor can be adjusted for different contact pressures Proximity Sensors(3) - Bump and feeler sensors Roomba 500 Bump

Computer Vision - Definition Computer vision refers to processing data from any modality which uses the electromagnetic spectrum which produces an image. face recognition

Computer Vision - Attributes Physics : light reflecting off of surfaces, respond to wavelenght Field of view, range : depends on lens; lens typical have a different VFOV and HFOV (vertical, horizontal) Accuracy & repeatability : good Responsiveness in target domain : depends on lighting source, inherent constrast between objects of interest Power consumption : low Reliability : good Size : can be miniaturized Interpretation Reliability : good

A charge-coupled device (CCD) is a device for the movement of electrical charge, usually from within the device to an area where the charge can be manipulated, for example conversion into a digital value. Computer Vision - CCD cameras CCD sensors typically produce less NOISE. CCD sensors typically are more light-sensitive. CMOS sensors use far less power. CMOS sensor cost less to produce.

RGB (red, green, blue) is the NTSC output –Poor color constancy in “real world” H,S,I (hue, saturation, intensity) has theoretical color constancy –But not with conversion from RGB to HSI Alternatives SCT (Spherical Coordinate Transform) That color space was designed to transform RGB data to a color space that more closely duplicates the response of the human eye. Computer Vision - Color planes Original imageRG B HS I SC T

For reactive applications: –Color segmentation Imprint on a color region, then follow it (or remember it) –Color histogramming Imprint on a region with a distribution of color, then follow it (or remember it) Computer Vision - Common Vision Algorithms

Range from Vision -Stereo camera pairs Using two cameras to extract range data is often referred to as range from stereo, stereo disparity, binocular vision, or just plain “stereo.” One way to extract depth is to try to superimpose a camera over each eye. Each camera finds the same point in each image, turns itself to center that point in the image, then measures the relative angle. The cameras are known as the stereo pair. Ways of extracting depth from a pair of cameras

Light striping, light stripers or structured light detectors work by projecting a colored line (or stripe), grid, or pattern of dots on the environment. Then a regular vision camera observes how the pattern is distorted in the image. Range from Vision -Light stripers

Range from Vision -Laser ranging(Sick) Accuracy & repeatability - Excellent results Responsiveness in target domain Power consumption - High; reduce battery run time by half Reliability - good Size - A bit large Computational Complexity –Not bad until try to “stack up” Interpretation Reliability –Much better than any other ranger flat surface an obstacle a negative obstacle SICK PLS100

180 o plane Advantages: high accuracy, coverage Disadvantages: 2D, resistant to miniaturization, cost ($13,000 US) Range from Vision -Laser Ranger Summary NASA/CMU Nomad robot (Carnegie Mellon University )

Case Study : Hors d’Oeuvres, Anyone? (Borg Shark and Puffer Fish) Camera pair (redundant): Face color Laser range: Count treat removal Sonars: Avoid obstacles, If blocked, Puffed up Digital thermometer: “Face” temperature check Sensor fusion: Reduced False positives, False negatives From 27.5% to 0%

State diagrams for the Borg Shark sonar shaft encoders map: evidence grid waypoint navigation: move to goal, avoid serving food: finding faces, counting treat removal thermal vision laser range sonar awaiting refill: finding faces visionthermal at waypoint OR food removed time limit serving exceeded food depleted full tray

Summary Design of a sensor suite requires careful consideration –Almost all robots will have proprioception, but exteroception needs to be closely matched to the task and the environment Most common exteroceptive sensors on mobile robots are: –Ultrasonics –Computer vision –Laser range Color vision can be hard, almost all vision is computationally expensive unless focus on affordances –Borg shark and Puffer fish with color plus heat –Polly and texture

Thank you