1. A Vision System for Landing an Unmanned Aerial Vehicle
Omid Shakernia, Cory Sharp
Department of EECS
University of California at Berkeley

2. Outline Motivation Vision system hardware/software
Landing target design/tracking
Active camera control
Flight videos

3. Goal: Autonomous UAV landing on a ship’s flight deck
Motivation
Goal: Autonomous UAV landing on a ship’s flight deck
Challenges
Hostile operating environments
high winds, pitching flight deck, ground effect
UAV undergoing changing nonlinear dynamics
Why the vision sensor?
Passive sensor (for stealth)
Gives relative UAV motion to flight deck
U.S. Navy photo

4. Objective for Vision Based Landing

5. Camera Model Calibrated pinhole camera Perspective Projection:
The image of a point is
denoted by
Notice: where:
Important identity:

6. Planar Essential Constraint
All points on landing pad satisfy
Image correspondences satisfy
the planar essential constraint
Current camera
position
planar constraint
Desired camera
position
Feature points on landing pad

7. Vehicle Control Language
Vision in Control Loop
Camera
Pan/Tilt Control
Feature Point
Correspondence
Motion
Estimation
Image Processing,
Corner Finding
Helicopter State
RS-232
Control Strategy
Vehicle Control Language
Navigation Computer
Vision Computer

8. UAV Testbed

9. Vision System Hardware
Ampro embedded PC Little Board P5/x
Low power Pentium 233MHz, running LINUX
440 MB flashdisk HD, robust to body vibration
Runs motion estimation algorithm
Controls PTZ camera
Motion estimation algorithms
Written and optimized in C++ using LAPACK
Give motion estimates at 30 Hz

10. Vision System Software

11. Nonlinear Motion Estimation
Minimize reprojection error using Newton-Raphson
Gaussian elimination to solve for dlamda
Iteratively, this drives dbeta to 0

12. Pan/Tilt Camera Control
Feature tracking issues:
Leave the field of view
Pan/tilt increases the
range of motion of the UAV
Pan/tilt control
drive all feature points to the
center of the image

13. Coordinate Frames

14. Flight Test Results

15. Vision Based State Estimate, RMS Error
Position error to within 5cm
Rotation error to within 5deg

16. Vision Ground Station

17. Flight Video

18. Pitching Landing Deck Landing deck to simulate motion of a ship at sea
6 electrically actuated cylindrical shafts
Motion Parameters:
sea state (freq, amp of waves)
ship speed
direction into waves
Stiffened Aluminum construction
Dimensions: 8’ x 6’

19. Moving Landing Pad

20. Hovering Above Deck

21. Landing on Deck

22. Papers Published
A Vision System for Landing an Unmanned Aerial Vehicle
Cory Sharp, Omid Shakernia, Shankar Sastry
Submitted: ICRA 2001
Landing an Unmanned Air Vehicle: Vision based motion estimation and nonlinear control
Omid Shakernia, Yi Ma, T. John Koo, Shankar Sastry,
Asian Journal of Control, Vol. 1, No. 3, Sept. 1999
Vision guided landing of an Unmanned Air Vehicle,
Omid Shakernia, Yi Ma, Joao Hespanha, Shankar Sastry,
IEEE Conf. on Decision and Control, Phoenix, Arizona, Dec. 1999