Optic Flow QuadCopter Control Update Presentation 1 Weeks 1-4 Optic Flow QuadCopter Control Oscar Merry

Contents Introduction OpenCV Honeybee Vision Research Paper 1 – AR Drone LabVIEW Toolkit Research Paper 2 – Optical Flow Quadrotor controller Other Research Papers General Problems Aims of Project Discussion

Introduction Optical Flow: Last 4 weeks: The estimation of the motion field created by a moving camera with respect to a rigid scene. Last 4 weeks: Literature Analysis Research into Optic Flow Research into honeybee navigation Research into OpenCV

OpenCV Open-source C / C++ library for advanced computer vision. Built in functions for many optical flow processing techniques: Canny edge detector Feature tracking Lucas–Kanade Horn-Schunck Shi-Tomasi

Honeybee Vision Each compound eye in the honeybee contains ~ 5000 ommatidia Each ommatidium has 9 photoreceptor cells grouped into 3 classes Ultraviolet sensitive Blue sensitive Green sensitive Bees control their flight speed by keeping optical flow constant. (The same process is used for landing) This is done via the Green photoreceptors. Bees recognise objects both through strong contrasts in luminance or colour and through optical flow.

Research Paper 1 - AR Drone LabVIEW Toolkit Michael Mogenson Masters Thesis: “A software framework for the control of low-cost quadrotor aerial robots.” Created a LabVIEW toolkit for the control of Parrot AR Drone. http://dl.dropboxusercontent.com/u/22965474/Michael_Mogenson_Thesis.pdf

AR Drone LabVIEW Toolkit Toolkit has multiple virtual instruments. (VIs) Regular Comms VIs: Main VI for control commands and comms management Video VI for video reading and decoding Nav Data VI for reading navigation data ‘Thinking’ Vis: State VI to estimate position in X,Y,Z Cartesian coordinates Various image processing Vis (Fast blob detection, dense optical flow, image space conversion, ROI VI) Toolkit has wrapper for OpenCV Library. http://dl.dropboxusercontent.com/u/22965474/Michael_Mogenson_Thesis.pdf

AR Drone LabVIEW Toolkit – State VI Populates a rotation matrix between the drones coordinate system and the ground using Euler orientation angles from navigation data. Applies rotation matrix to the velocities measured from optical flow from the bottom camera. Rotated velocities integrated into a position with the timestamp data. Suffers from problem of drift.

AR Drone LabVIEW Toolkit – Achievements Benefit of LabVIEW – modifications without recompiling Demonstrations: Face tracking Indoor hallway navigation (Via vanishing point) Fly through hoop Problems: Indoor hallway navigation fails if 90 degree turn or if facing wall State Estimation VI suffers from drift

Research Paper 1 - Optical Flow Quadrotor controller “Optical Flow-Based Controller for Reactive and Relative Navigation dedicated to a Four Rotor Rotorcraft” Eduardo Rondon, Isabelle Fantoni-Coichot, Anand Sanchez, Guillaume Sanahuja Produced a controller for a Quadrotor based on the optical flow from 2 cameras. (1 for velocity regulation, 1 for obstacle avoidance) Implemented obstacle avoidance for indoor navigation. http://ieeexplore.ieee.org.eresources.shef.ac.uk/stamp/stamp.jsp?tp=&arnumber=5354483

Optical Flow Quadrotor controller http://ieeexplore.ieee.org.eresources.shef.ac.uk/stamp/stamp.jsp?tp=&arnumber=5354483

Optical Flow Quadrotor controller - Achievements Velocity regulation via optical flow controller. Sets the inverse time-to-contact to a threshold to stop the vehicle if obstacle detected. Has lateral and altitude avoidance. http://ieeexplore.ieee.org.eresources.shef.ac.uk/stamp/stamp.jsp?tp=&arnumber=5354483

Other Research Papers “Optic flow based slope estimation for autonomous landing” de Croon et al. Achieved slope following and autonomous landing. “Combined Optic-Flow and Stereo-Based Navigation of Urban Canyons for a UAV” Hrabar et al. Used optical flow to balance UAV in canyon. Used stero vision to navigate T and L junctions. “An adaptive vision-based autopilot for mini flying machines guidance, navigation and control.” used optic flow and IMU data for guidance navigation and control, specifically automatic hovering, landing, and target tracking. Use feature tracking to reduce optic flow computation.

General Problems Camera Calibration – In order for the depth of a scene to be known the camera must be calibrated on a known object. (Or height above ground must be known) Video Delay – Video encoding, decoding, and transmission must be fast enough.

Project Aims Hovering Stabilisation Position and velocity control Smooth Landing Execution Obstacle recognition and avoidance (with a focus on methods that have similarities to honeybees e.g. flower recognition)

Discussion Communication: Formalize state commands. Hardware State commands (roll angle, pitch angle, yaw rate, climb rate, ??) Video frames Formalize state commands.