A UTONOMOUS GUIDANCE FOR A UAS ALONG A STAIRCASE 14/12/2015 Olivier De Meyst, Thijs Goethals, Haris Balta, Geert De Cubber, Rob Haelterman Belgian Royal Military Academy (RMA)

P ROJECT C ONTEXT : ICARUS

P ROBLEM S TATEMENT Indoor flight is becoming possible However, staircases remain a problem ?

P ROBLEM S TATEMENT & C ONSTRAINTS Design a UAS control architecture able to guide a UAS over a staircase Use a low-cost UAV: Parrot AR-Drone Use as much as possible existing (ROS) components  Focus on system integration

S YSTEM A RCHITECTURE Components: Perception 2D Detection 3D Detection 2D/3D Data Fusion Navigation

2D D ETECTION P IPELINE : 1) L INE D ETECTION 1.Line Segment Detector (LSD) 2.Filter out parallel lines (RANSAC) 3.Detect vanishing points

2D D ETECTION P IPELINE : 2) O BJECT D ETECTION 1.ADABOOST- based object detector, trained for staircase models

2D D ETECTION P IPELINE : 3) C OMBINING LINES AND OBJECTS Retain only detections where both detector agree  Verify if detected lines fall in object detection rectangles  Cohen-Sutherland clipping algorithm

3D D ETECTION P IPELINE Sparse 3D Data collected via LSD-SLAM Projected to depth image for further processing

2D-3D C OMBINATION 1.Back-project 2D lines in 3D image (PCL SACMODEL_LINE ) 2.Fit 3D Planes 3.Planes are agglomerated to create a model of a staircase 4.Apply hierarchical clustering to reduce the number of hits and improve the accuracy of detection

S TAIRCASE M ODELING Projection Model:

The staircase can be characterized by expressing the ratio between the observed distance d between the projected lines P i of the staircase in the 2D image model: f(c x, c y, s d, s h, i,  )  f(c x,i) S TAIRCASE M ODELING

A UTONOMOUS N AVIGATION Based on ROS tum_ardrone: 1. When the developed staircase algorithm has enough detected points, it calculates the central point using an agglomerative clustering algorithm 2. The top step of the staircase is identified 3. A point 1.5 meters above the top step is calculated and chosen as a goal position 4. A goto x y z w command is issued to tum ardrone (w, the yaw angle, in degrees and the coordinates (x, y, z ) are in m) 5. The tum ardrone navigation controller ensures the autonomous navigation of the UAS from the start to the goal position by using four built-in PID controllers.

R ESULTS Limitation of the system: off-board processing Framerate: 5-10 FPS (i7) Initialization of the tum_ardrone package has proven to be the most diffcult part of this methodology. Planar scenes with a lot of different texture worked fine with a success rate of over 80 percent. A scene with a lot of depth or similar texture (e.g. concrete wall) did not work at all.

R ESULTS Outdoor staircases had a higher success rate. Mainly because of better light conditions which result in a superior performance of lsd_slam. tum_ardrone is very sensitive to yaw movements. Sudden movements can result in losing the tracking and this can only be solved by a re-initialization. Once tum ardrone and lsd slam were successful initialized, a 100 percent success rate of detection was obtained and 75 percent of the staircases were successfully climbed.

R ESULTS : I NDOOR N AVIGATION

R ESULTS : O UTDOOR N AVIGATION Outdoor Navigation

R ESULTS : O UTDOOR N AVIGATION

C ONCLUSIONS & F UTURE W ORK The methodology integrating multiple SoA approaches was presented, enabling a UAS to climb stairs This is a first step; multiple improvements are required: Better UAS platform and sensors (e.g dedicated depth sensors) On-line processing Higher robustness Dealing with narrower staircases Faster initialisation Future evolution: hybrid system (UAV-UGV)

14/12/2015 A UTONOMOUS GUIDANCE FOR A UAS ALONG A STAIRCASE Olivier De Meyst, Thijs Goethals, Haris Balta, Geert De Cubber, Rob Haelterman Belgian Royal Military Academy (RMA)