1. Optic Flow QuadCopter Control
Update Presentation 1
Weeks 1-4
Optic Flow QuadCopter Control
Oscar Merry

2. Contents Introduction OpenCV Honeybee Vision
Research Paper 1 – AR Drone LabVIEW Toolkit
Research Paper 2 – Optical Flow Quadrotor controller
Other Research Papers
General Problems
Aims of Project
Discussion

3. Introduction Optical Flow: Last 4 weeks:
The estimation of the motion field created by a moving camera with respect to a rigid scene.
Last 4 weeks:
Literature Analysis
Research into Optic Flow
Research into honeybee navigation
Research into OpenCV

4. OpenCV Open-source C / C++ library for advanced computer vision.
Built in functions for many optical flow processing techniques:
Canny edge detector
Feature tracking
Lucas–Kanade
Horn-Schunck
Shi-Tomasi

5. Honeybee Vision
Each compound eye in the honeybee contains ~ 5000 ommatidia
Each ommatidium has 9 photoreceptor cells grouped into 3 classes
Ultraviolet sensitive
Blue sensitive
Green sensitive
Bees control their flight speed by keeping optical flow constant. (The same process is used for landing) This is done via the Green photoreceptors.
Bees recognise objects both through strong contrasts in luminance or colour and through optical flow.

6. Research Paper 1 - AR Drone LabVIEW Toolkit
Michael Mogenson Masters Thesis:
“A software framework for the control of low-cost quadrotor aerial robots.”
Created a LabVIEW toolkit for the control of Parrot AR Drone.

7. AR Drone LabVIEW Toolkit
Toolkit has multiple virtual instruments. (VIs)
Regular Comms VIs:
Main VI for control commands and comms management
Video VI for video reading and decoding
Nav Data VI for reading navigation data
‘Thinking’ Vis:
State VI to estimate position in X,Y,Z Cartesian coordinates
Various image processing Vis (Fast blob detection, dense optical flow, image space conversion, ROI VI)
Toolkit has wrapper for OpenCV Library.

8. AR Drone LabVIEW Toolkit – State VI
Populates a rotation matrix between the drones coordinate system and the ground using Euler orientation angles from navigation data.
Applies rotation matrix to the velocities measured from optical flow from the bottom camera.
Rotated velocities integrated into a position with the timestamp data.
Suffers from problem of drift.

9. AR Drone LabVIEW Toolkit – Achievements
Benefit of LabVIEW – modifications without recompiling
Demonstrations:
Face tracking
Indoor hallway navigation (Via vanishing point)
Fly through hoop
Problems:
Indoor hallway navigation fails if 90 degree turn or if facing wall
State Estimation VI suffers from drift

10. Research Paper 1 - Optical Flow Quadrotor controller
“Optical Flow-Based Controller for Reactive and Relative Navigation dedicated to a Four Rotor Rotorcraft”
Eduardo Rondon, Isabelle Fantoni-Coichot, Anand Sanchez, Guillaume Sanahuja
Produced a controller for a Quadrotor based on the optical flow from 2 cameras. (1 for velocity regulation, 1 for obstacle avoidance)
Implemented obstacle avoidance for indoor navigation.

11. Optical Flow Quadrotor controller

12. Optical Flow Quadrotor controller - Achievements
Velocity regulation via optical flow controller.
Sets the inverse time-to-contact to a threshold to stop the vehicle if obstacle detected.
Has lateral and altitude avoidance.

13. Other Research Papers
“Optic flow based slope estimation for autonomous landing” de Croon et al.
Achieved slope following and autonomous landing.
“Combined Optic-Flow and Stereo-Based Navigation of Urban Canyons for a UAV” Hrabar et al.
Used optical flow to balance UAV in canyon.
Used stero vision to navigate T and L junctions.
“An adaptive vision-based autopilot for mini flying machines guidance, navigation and control.”
used optic flow and IMU data for guidance navigation and control, specifically automatic hovering, landing, and target tracking.
Use feature tracking to reduce optic flow computation.

14. General Problems
Camera Calibration – In order for the depth of a scene to be known the camera must be calibrated on a known object. (Or height above ground must be known)
Video Delay – Video encoding, decoding, and transmission must be fast enough.

15. Project Aims Hovering Stabilisation Position and velocity control
Smooth Landing Execution
Obstacle recognition and avoidance (with a focus on methods that have similarities to honeybees e.g. flower recognition)

16. Discussion Communication: Formalize state commands. Hardware
State commands (roll angle, pitch angle, yaw rate, climb rate, ??)
Video frames
Formalize state commands.