A Vision System for Landing an Unmanned Aerial Vehicle Omid Shakernia, Cory Sharp Department of EECS University of California at Berkeley

Outline Motivation Vision system hardware/software Landing target design/tracking Active camera control Flight videos

Goal: Autonomous UAV landing on a ship’s flight deck Motivation Goal: Autonomous UAV landing on a ship’s flight deck Challenges Hostile operating environments high winds, pitching flight deck, ground effect UAV undergoing changing nonlinear dynamics Why the vision sensor? Passive sensor (for stealth) Gives relative UAV motion to flight deck U.S. Navy photo

Objective for Vision Based Landing

Camera Model Calibrated pinhole camera Perspective Projection: The image of a point is denoted by Notice: where: Important identity:

Planar Essential Constraint All points on landing pad satisfy Image correspondences satisfy the planar essential constraint Current camera position planar constraint Desired camera position Feature points on landing pad

Vehicle Control Language Vision in Control Loop Camera Pan/Tilt Control Feature Point Correspondence Motion Estimation Image Processing, Corner Finding Helicopter State RS-232 Control Strategy Vehicle Control Language Navigation Computer Vision Computer

UAV Testbed

Vision System Hardware Ampro embedded PC Little Board P5/x Low power Pentium 233MHz, running LINUX 440 MB flashdisk HD, robust to body vibration Runs motion estimation algorithm Controls PTZ camera Motion estimation algorithms Written and optimized in C++ using LAPACK Give motion estimates at 30 Hz

Vision System Software

Nonlinear Motion Estimation Minimize reprojection error using Newton-Raphson Gaussian elimination to solve for dlamda Iteratively, this drives dbeta to 0

Pan/Tilt Camera Control Feature tracking issues: Leave the field of view Pan/tilt increases the range of motion of the UAV Pan/tilt control drive all feature points to the center of the image

Coordinate Frames

Flight Test Results

Vision Based State Estimate, RMS Error Position error to within 5cm Rotation error to within 5deg

Vision Ground Station

Flight Video

Pitching Landing Deck Landing deck to simulate motion of a ship at sea 6 electrically actuated cylindrical shafts Motion Parameters: sea state (freq, amp of waves) ship speed direction into waves Stiffened Aluminum construction Dimensions: 8’ x 6’

Moving Landing Pad

Hovering Above Deck

Landing on Deck

Papers Published A Vision System for Landing an Unmanned Aerial Vehicle Cory Sharp, Omid Shakernia, Shankar Sastry Submitted: ICRA 2001 Landing an Unmanned Air Vehicle: Vision based motion estimation and nonlinear control Omid Shakernia, Yi Ma, T. John Koo, Shankar Sastry, Asian Journal of Control, Vol. 1, No. 3, Sept. 1999 Vision guided landing of an Unmanned Air Vehicle, Omid Shakernia, Yi Ma, Joao Hespanha, Shankar Sastry, IEEE Conf. on Decision and Control, Phoenix, Arizona, Dec. 1999