Rotational Rectification Network for Robust Pedestrian Detection Xinshuo Weng Shangxuan Wu Wentao Han RI MSCV May 3, 2017 This work has been submitted to BMVC2017

Motivations “Upright” assumption in benchmark datasets (Caltech, ETH and INRIA): Pedestrians stand upright on the ground. Camera image plane is roughly orthogonal to the ground plane. This assumption is easily invalidated in real world Mobile phone camera UAV Construction vehicles on rugged terrain This lead to poor performance of general pedestrian detector Visualization 先讲Main Task再讲Motivation

Detection results from general detector without rotational robustness Visualization Rotated Caltech dataset Frames from YouTube videos 先讲Main Task再讲Motivation Detection results from general detector without rotational robustness

Overall Architecture

Approach Global Polar Pooling (GP-Pooling) Rotation Estimation Module Rotational Rectification Network (R2N)

Global Polar Pooling

Visualization: Global Polar Pooling Rotational changes on input image Translational shifts on responses

Rotational Estimation Module Layers: Red: Convolution Yellow: Max Pooling Gray: Normalization Green: Concatenation Magenta: Flatten Cyan: Fully-Connected Module Input: Image features Module Output: Rotation angle present in the image features

Rotational Rectification Network Plug-in property: R2N could be flexibly inserted between any intermediate layers prior to detection

Evaluation Results the GP-Pooling operator Rotated MNIST Dataset the GP-Pooling operator On Rotated MNIST Dataset On Rotated Caltech Dataset Rotation Invariant Pedestrian Detection Rotated Caltech Dataset So Let’s talk about the evaluation results. First, we did some evaluation on our proposed GP-Pooling operator, both on rotated MNIST digit dataset and rotated Caltech dataset. On both of these datasets, we achieve state-of-the-art results on rotation angle estimation task. Secondly, we tested on our main task, pedestrian detection. As you can see from the lower figures, we got much less miss rate in the rotated images than state-of-the-art pedestrian detector, while the performance on upright images only drop 1%. Original Caltech Dataset Rotated Caltech Dataset Camera Naturally Rotated Dataset (from YouTube videos)

Qualitative Results Rotation-Invariant Pedestrian Detection Here are some qualitative results of our rotated pedestrian detection experiment. The yellow bounding box is the output of state-of-the-art pedestrian detector, and the red bounding box is the output of our rotation-invariant pedestrian detector. As you can see, traditional pedestrian detectors fail when the the rotation angle is larger than 45 degree, while our detector could always detect people. We can even detect heavily-rotated pedestrians that are almost in 90 degrees of rotation. What’s more, we can detect small people in the scene while general detectors fail.

Main Contributions Proposed a Rotation-Invariant GP-Pooling operator, which can be used to encode radial distribution of features Proposed a Rotational Rectification Network (R2N) that can be inserted into a wide range of CNN-based detectors to achieve rotational invariance In conclusion, our main contribution in this project is two fold. First, we proposed a rotati………. Secondly, …… Evaluation results show that our rotational rectification network is capable of detecting heavily rotated pedestrian while state-of-the-art detectors fail.