Sensor Placement Agile Robotics Program Review August 8, 2008 Matthew Walter, Michael Boulet, Luke Fletcher, Matthew Antone, Nick Roy, Seth Teller

Outline Perception requirements Sensor types Candidate sensor placements Perception simulations Data collection Summary

Perception Tasks Several roles expected of forklift sensors Situational Awareness: supervisor “bot’s-eye” view Navigation: path planning, terrain mapping, obstacle avoidance Object Detection: finding and recognizing trucks, pallets, slots; estimating pose Object Manipulation: pallet geometry estimation, tine insertion, load balancing Safety: Shouted command recognition, seeing beyond load, detecting proximity of humans Requires array of sensor types and placements

Perception Constraints System complexity Simplify hardware and perception algorithms Physical constraints Maximize visibility, protect sensors Practical considerations Subject to acceptable limits on size, weight, power, cost

Sensor Types Exteroceptive (perceiving the world) Array Microphone FOV: 180 deg Color Digital Camera FOV: 90 deg Res: 752x480 pix Rate: 60 Hz Proprioceptive (self movement) GPS/IMU Odometry Mast and tine states Wheel encoders Interoceptive (internal state) Motor, brake status Strain gauges SICK Laser Range Scanner Range: up to 80m FOV: 180 deg Res: 1 deg Rate: 75 Hz Hokuyo Laser Range Scanner Range: up to 4m FOV: 240 deg Res: 0.36 deg Rate: 10 Hz

Cameras provide 360o situational awareness Camera Placement Cameras provide 360o situational awareness Right Front Rear Left

Supervisor Camera Views Left Front Right

Skirt Laser Placement Long-range skirt lasers assist in navigation and obstacle avoidance Scan in plane approximately parallel to the ground

Pushbroom Laser Placement Long-range pushbroom lasers assist in navigation and terrain mapping Rear: 10-20m out Front Top: 10-20m out Front Bottom: Close range, beneath load

Temporal Scan Persistence Single scan sees only narrow “slice” of world Each scan placed in local 3D coordinate frame Requires knowledge of forklift, mast, and tine pose Allows aggregation of multiple scans over time Drive forward Raise mast

Lasers for Navigation

Skirt lasers (“virtual tines”) Pallet Sensor Layout Short-range lasers move with mast and tines to perceive pallets and slots Pushbroom (2-5m out) Skirt lasers (“virtual tines”)

Pallet Sensing Simulations Goal: study effects of sensor placement on pallet and slot perception Use system infrastructure for simulation Virtual 3D environment containing objects of interest Simulated returns from lasers in various configurations Data acquisition with different pallet types, pallet poses, ranges, approach maneuvers

Pallet Sensing Simulations

Simulated Data: Ground Pallet No mast or tine motion Approaching Pallet on ground Tilt mast Raise/lower mast

Simulated Data: Truck Pallet Far-field, ~18m Approaching Pallet on truck Near-field, ~2m Mid-field, ~4m

Real Data Collection Goal: acquire representative sensor data Pallet approach and slot detection Mast and tine movement to ‘scan’ objects Realistic non-level, non-smooth surfaces

Data Collection Sensor Layout Camera Pushbroom Vertical Skirt

Short Range Laser Scans

Longer Range Laser Scans

Summary Sensor types determined to meet operational needs Support situational awareness, navigation, object detection, object manipulation, safety tasks Studies performed using simulated and real data sets 3D perception simulator with different configurations Data collected from real forklift on realistic terrain Objects ‘scanned’ using mast motion Additional studies to be performed in coming weeks

Timeline (Year 1) start of year 1 end of year 1 today sep 08 nov 08 perception simulator preliminary placement studies sep 08 navigation sensors on forklift experimental configurations nov 08 all sensors on forklift final sensor configuration jan 09 precise body/mast calibration experimental validation mar 09 additional simulation investigation of year 2 sensors apr 08