Luis Mejias, Srikanth Saripalli, Pascual Campoy and Gaurav Sukhatme

 Introduction  Related work  Testbed  Visual preprocessing  Control Architectures  Experiments  Conclusion

 Introduction  Related work  Testbed  Visual preprocessing  Control Architectures  Experiments  Conclusion

 Goal:  vision-guided autonomous flying robots  Application:  Law enforcement, search and rescue, inspection and surveillance  Technique:  Object detection, tracking, inertial navigation, GPS and nonlinear system modeling

 In this paper:  Two UAVs – Avatar and COLIBRI  Visual tracking => control commands

 Introduction  Related work  Testbed  Visual preprocessing  Control Architectures  Experiments  Conclusion

 Hummingbird (A. Conway, 1995)  Model-scale  Use GPS only  4 GPS antennas  Precisions: position 1cm attitude 1 degree

 AVATAR (Jun, 1999)  Onboard INS & GPS  Kalman Filter for State Estimation  Simulation

 Vision-guided Helicopter (Amidi, 1996, 1997)  Onboard DSP-based vision processor  Combine GPS and IMU data

 Vision-augmented navigation system (Bosse, 1997)  Uses vision in-the-loop to control a helicopter  Visual odometer (Amidi, 1998)  A notable vision-based technique used in autonomous helicopter  (Wu, et al, 2005)  Vision is used as additional sensor and fused with inertial and heading measurements for control

 Introduction  Related work  Testbed  Visual preprocessing  Control Architectures  Experiments  Conclusion

 AVATAR  Gas-powered radio-controlled model helicopter  RT-2 DGPS system provides positional accuracy of 2 cm  ISIS-IMU provides rate information to onboard computer, which is fused using a 16 state Kalman filter  Ground station: a laptop to send high-level control commands and differential GPS corrections  Autonomous flight is achieved using a behavior-based control architecture

 COLIBRI  Gas powered model helicopter  Fitted with a Xscale based flight computer augmented with GPS, IMU, Magnetometer, fused with a Kalman filter  VIA mini-ITX 1.25 GHz computer onboard with 512 Mb RAM, wireless interface and a firewire color camera  Ground station: a laptop to send high-level control commands, and for visualization

 Introduction  Related work  Testbed  Visual preprocessing  Control Architectures  Experiments  Conclusion

 Image segmentation and thresholding  Convert the image to grayscale  Use the value of “target color” as threshold  Segment the image to binary image where the object of interest is represented by 1’s and background with 0’s

 Square Finding  Find contours (represented by polylines) from the binary image  Use an algorithm to reduce the points in polylines  Result: simplified squares

 Template Matching  User selects a detected window (a target)from the GUI  A patch is selected around the location of the target  Use local search window to find best match between the target and the detected contours, deciding which window to track

 Kalman Filter  Once a suitable match is found, a Kalman filter is used to track the feature positions  Input: x and y coordinates of the features  Output: estimates of these coordinates in the next frame

 The user selects the object of interest from the GUI  The location of the object is used to generate visual reference

 Lateral visual reference

 Vertical Visual Reference

 Introduction  Related work  Testbed  Visual preprocessing  Control Architectures  Experiments  Conclusion

 A hierarchical behavior based control architecture  Output of Kalman filter is compared with desired values to give an error signal to controller

 Controller is based on a decoupled PID control

 Introduction  Related work  Testbed  Visual preprocessing  Control Architectures  Experiments  Conclusion

 At Del Valle Urban Search and Rescue Training site in Santa Clarita, California  AVATAR, four trials  First, the helicopter is commanded to fly autonomously to a given GPS waypoint  As soon as it detects the featured window, the controller switches from GPS-based to vision-based control

Location of the features in the image

Helicopter position in meters. (left figure) vertical axis– easting (right figure) vertical axis – northing

 At ETSII Campus in Madrid, Spain  COLIBRI  Seven experimental trials on two different days

Velocity references (vy r ) with the helicopter velocity (vy) Lateral displacement (east)

Velocity references (vz r ) with the helicopter velocity (vz) altitude displacement (down)

Helicopter displacements during the entire flight trial

 colibrivideoWeb.wmv colibrivideoWeb.wmv

 Introduction  Related work  Testbed  Visual preprocessing  Control Architectures  Experiments  Conclusion

 Demonstrated an approach to visually control an autonomous helicopter: use visual algorithm to command UAV when GPS has dropouts  Experimentally demonstrated by performing vision-based window tracking tasks on two different platforms at different locations and different conditions

 The topic is interesting  Visual algorithm is demonstrated effective in the experiments  But… the writing is so ugly.  Poor explanation ▪ features, template and matching  Incomplete explanation of figures