1. 1 CMPUT 412 Sensing Csaba Szepesvári University of Alberta TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A A A A

2. 2 Defining sensors and actuators Environment actions Sensations (and reward) Controller = agent Sensors Actuators

3. 3 Perception Sensors Uncertainty Features

4. 4 How are sensors used?

5. 5 HelpMate, Transition Research Corp.

6. 6 B21, Real World Interface

7. 7 Robart II, H.R. Everett

8. 8 Savannah, River Site Nuclear Surveillance Robot

9. 9 BibaBot, BlueBotics SA, Switzerland Pan-Tilt Camera Omnidirectional Camera IMU Inertial Measurement Unit Sonar Sensors Laser Range Scanner Bumper Emergency Stop Button Wheel Encoders

10. 10 Taxonomy of sensors

11. 11 Classification of Sensors  Where is the information coming from? Inside: Proprioceptive sensors  motor speed, wheel load, heading of the robot, battery status Outside: Exteroceptive sensors  distances to objects, intensity of the ambient light, unique features  How does it work? Requires energy emission? No: Passive sensors  temperature probes, microphones, CCD Yes: Active sensors  Controlled interaction -> better performance  Interference  Simple vs. composite (sonar vs. wheel sensor)

12. 12 General Classification (1)

13. 13 General Classification (2)

14. 14 Sensor performance

15. 15 How Do (Simple) Sensors Work? Analog signalsDigital signals Physical process Environment Electrical current Analog to digital conversion 00101011010100 11010111010101 inputoutput

16. 16 Mathematical Models  Signal in => signal out: response Memoryless: V out = S( E in, Noise t ) With memory: V out = f( V out, E in, Noise t ) Physical process Environment Electrical current Analog to digital conversion 00101011010100 11010111010101 inputoutput Sampling rate, aliasing, dithering

17. 17 Nominal Sensor Performance  Valid inputs E min : Minimum detectable energy E max : Maximum detectable energy Dynamic range = E max /E min, or 10 log(E max /E min ) [dB]  power measurement or volt? (V 2 ~ power) Operating range (N min, N max ): E min · N min · N max · E max  No aliasing in the operating range (e.g., distance sens.)  Response Sensor response: S(E in )=?  Linear? (or non-linear)  Hysteresis Resolution ( ¢ ):  E 1 -E 2 · ¢ ) S(E 1 ) ¼ S(E 2 ); often ¢ =min(E min, ¢ A/D ) Timing  Response time ( range ): delay between input and output [ms]  Bandwidth: number of measurements per second [Hz]

18. 18 In Situ Sensor Performance: Sensitivity Characteristics.. especially relevant for real world environments  Sensitivity: How much does the output change with the input? Memoryless sensors: min{ [d/dE S] (E in ) | E in } Sensors with memory: min{ f(V,E in )/E in | V, E in }  Cross-sensitivity sensitivity to environmental parameters that are orthogonal to the target parameters e.g. flux-gate compass responds to ferrous buildings, orthogonal to magnetic north  Error: ² (t) = S(t) - S(E in (t)) Systematic: ² (t) = D(E in (t)) Random: ² (t) is random, e.g., ² (t) ~ N( ¹, ¾ 2 )  Accuracy (systemacity): 1-|D(E in )|/E in, e.g., 97.5% accuracy  Precision (reproducability): Range out / Var( ² (t)) 1/2

19. 19 In Situ Sensor Performance: Errors Characteristics.. especially relevant for real world environments  Error: ² (t) = S(t) - S(E in (t)) Systematic: ² (t) = D(E in (t))  Predictable, deterministic  Examples: Calibration errors of range finders Unmodeled slope of a hallway floor Bent stereo camera head due to an earlier collision Random: ² (t) is random, e.g., ² (t) ~ N( ¹, ¾ 2 )  Unpredictable, stochastic  Example: Thermal noise ~ hue calibration, black level noise in a camera  Accuracy – accounts for systemic errors 1-|D(E in )|/E in, e.g., 97.5% accuracy  Precision – high precision ~ low noise Range out / Var( ² (t)) 1/2

20. 20 Challenges in Mobile Robotics Systematic vs. random errors Error distributions

21. 21 Systematic vs. Random?  Sonar sensor: Sensitivity to: material, relative positions of sensor and target (cross-sensitivity) Specular reflections (smooth sheetrock wall; in general material, angle) Systematic or random? What if the robot moves?  CCD camera: changing illuminations light or sound absorbing surfaces  Cross-sensitivity of robot sensor to robot pose and robot-environment dynamics rarely possible to model -> appear as random errors systematic errors and random errors might be well defined in controlled environment. This is not the case for mobile robots !!

22. 22 Error Distributions  A convenient assumption: ² (t) ~ N(0, § )  WRONG! Sonar (ultrasonic) sensor  Sometimes accurate, sometimes overestimating  Systematic or random? “Operation modes”  Random => Bimodal: - mode for the case that the signal returns directly - mode for the case that the signals returns after multi- path reflections. Errors in the output of a stereo vision system (distances)  Characteristics of error distributions Uni- vs. Multi-modal, Symmetric vs. asymmetric Independent vs. dependent (decorrelated vs. correlated)

23. 23 About Some Sensors Wheel Encoders Active Ranging

24. 24 Wheel Encoders

25. 25 Wheel/Motor Encoders (1)  Principle: Photo detection + optical grid  Direction of motion: Quadrature encoder  Output: Read values with polling or use interrupts  Resolution: 2000 (->10K) cycles per revolution (CPR). for higher resolution: interpolation, sine waves  Accuracy: no systematic error (accuracy~100%) time Rotating optical grid

26. 26 Wheel/Motor Encoders (2)  Measures position or speed of the wheels or steering  Use: odometry, position estimation, detect sliding of motors scanning reticle fields scale slits Direction change:

27. 27 Active Range Sensors

28. 28 Range Sensors  Large range distance measurement -> “range sensors”  Why? Range information is key for localization and environment modeling Cheap Relatively accurate  How? Time of flight Active sensing (sound, light)

29. 29 Time of flight - principles  Time delay of arrival (TDOA) TDOA – impulses  Sound, light TDOA – phase shift  Light  Geometry Triangulation – single light beam  Light Triangulation – structured light  Light  Light sensor; 1D or 2D camera

30. 30 Time Delay of Arrival  d = v t  d – distance travelled (computed)  v – speed of propagation (known)  t – time of flight (measured) 2D = v t D TargetSource & sensor

31. 31 TDOA: Limitations  What distances can we measure? Must wait for the last package to arrive before sending out the next one => Speed of propagation determines maximum range! Speeds  Sound: 0.3 m/ms  Electromagnetic signals (light=laser): 0.3 m/ns, 1M times faster! 3 meters takes..  Sound: 10 ms  Light: 10 ns.. But technical difficulties => expensive and delicate sensors

32. 32 TDOA: Errors Time measurement  Exact time of arrival of the reflected signal  Time of flight measure (laser range sensors) Opening angle of transmitted beam (ultrasonic range sensors) Interaction with the target (surface, specular reflections) Variation of propagation speed Speed of mobile robot and target (if not at stand still)

33. 33 Ultrasonic Sensor

34. 34 Ultrasonic (US) Sensor  transmit a packet of US pressure waves  The speed of sound v (340 m/s) in air is  ° : adiabatic index (sound wave->compression->heat)  R: moral gas constant [J/(mol K)]  M: molar mass [kg/mol]  T: temperature [K]

35. 35 Operation Transmitted sound Analog echo signal & threshold Digital echo signal Integrated time & output signal integrator Time of flight (output) threshold Wave packet Blanking time

36. 36 Ultrasonic Sensor  Frequencies: 40 - 180 kHz  Sound source: piezo/electrostatic transducer transmitter and receiver separated or not separated  Propagation: cone opening angles around 20 to 40 degrees regions of constant depth segments of an arc (sphere for 3D) Typical intensity distribution of an ultrasonic sensor Piezo transducer Electrostatic transducer

37. 37 Example

38. 38 Imaging with an US Issues:  Soft surfaces  Sound surfaces that are far from being perpendicular to the direction of the sound -> specular reflection a) 360° scanb) results from different geometric primitives

39. 39 Characteristics  Range: 12cm – 5 m  Accuracy: 98%-99.1%  Single sensor operating speed: 50Hz 3m -> 20ms ->50 measurements per sec  Multiple sensors: Cycle time->0.4sec -> 2.5Hz ->limits speed of motion (collisions)

40. 40 Laser Range Sensor

41. 41 Laser Range Sensor: Physics Laser= Low divergence Well-defined wavelength

42. 42 Time of flight measurement methods  Pulsed laser Direct measurement of elapsed time Receiver: Picoseconds accuracy Accuracy: centimeters  Beat frequency between a frequency modulated continuous wave and its received reflection  Phase shift measurement Technically easier than the above two methods

43. 43 Distance from phase-shift Phase [m] Transmitted beam (s(x)) Reflected beam (r(x)) Target Amplitude [V] Ambiguity! d and d+ ¸ /2 give the same µ

44. 44 Laser Range Sensor  Phase-Shift Measurement c: speed of light (0.3 m/ns) f: the modulating frequency D’: distance covered by the emitted light for f = 5 Mhz (as in the AT&T sensor), = 60 meters = c/f

45. 45 Laser Range Sensor  Confidence in the range (phase estimate) is inversely proportional to the square of the received signal amplitude. Hence dark, distant objects will not produce such good range estimated as closer brighter objects …

46. 46 Laser Range Sensor  Typical range image of a 2D laser range sensor with a rotating mirror. The length of the lines through the measurement points indicate the uncertainties.

47. 47 Triangulation Ranging  Geometry -> distance Unknown object size: project a known light pattern onto the environment and use triangulation Known object size: triangulation without light projecting

48. 48 Laser Triangulation (1D)

49. 49 Sharp IR Rangers

50. 50 Conclusions  Why & how? Sensing: Essential to deal with contingencies in the world Sensors: Make sensing possible  Anatomy of sensors: Physics, A/D, characteristics  Wheel encoders  Distance sensors Time of flight Triangulation