1. Rotational Rectification Network for Robust Pedestrian Detection
Xinshuo Weng Shangxuan Wu Wentao Han
RI MSCV
May 3, 2017
This work has been submitted to BMVC2017

2. Motivations
“Upright” assumption in benchmark datasets (Caltech, ETH and INRIA):
Pedestrians stand upright on the ground.
Camera image plane is roughly orthogonal to the ground plane.
This assumption is easily invalidated in real world
Mobile phone camera
UAV
Construction vehicles on rugged terrain
This lead to poor performance of general pedestrian detector
Visualization
先讲Main Task再讲Motivation

3. Detection results from general detector without rotational robustness
Visualization
Rotated Caltech
dataset
Frames from
YouTube videos
先讲Main Task再讲Motivation
Detection results from general detector without rotational robustness

4. Overall Architecture

5. Approach Global Polar Pooling (GP-Pooling) Rotation Estimation Module
Rotational Rectification Network (R2N)

6. Global Polar Pooling

7. Visualization: Global Polar Pooling
Rotational changes on input image
Translational shifts on responses

8. Rotational Estimation Module
Layers:
Red: Convolution
Yellow: Max Pooling
Gray: Normalization
Green: Concatenation
Magenta: Flatten
Cyan: Fully-Connected
Module Input:
Image features
Module Output:
Rotation angle present in the image features

9. Rotational Rectification Network
Plug-in property:
R2N could be flexibly inserted between any intermediate layers prior to detection

10. Evaluation Results the GP-Pooling operator
Rotated MNIST Dataset
the GP-Pooling operator
On Rotated MNIST Dataset
On Rotated Caltech Dataset
Rotation Invariant Pedestrian Detection
Rotated Caltech Dataset
So Let’s talk about the evaluation results. First, we did some evaluation on our proposed GP-Pooling operator, both on rotated MNIST digit dataset and rotated Caltech dataset. On both of these datasets, we achieve state-of-the-art results on rotation angle estimation task.
Secondly, we tested on our main task, pedestrian detection. As you can see from the lower figures, we got much less miss rate in the rotated images than state-of-the-art pedestrian detector, while the performance on upright images only drop 1%.
Original Caltech Dataset
Rotated Caltech Dataset
Camera Naturally Rotated Dataset
(from YouTube videos)

11. Qualitative Results Rotation-Invariant Pedestrian Detection
Here are some qualitative results of our rotated pedestrian detection experiment. The yellow bounding box is the output of state-of-the-art pedestrian detector, and the red bounding box is the output of our rotation-invariant pedestrian detector. As you can see, traditional pedestrian detectors fail when the the rotation angle is larger than 45 degree, while our detector could always detect people. We can even detect heavily-rotated pedestrians that are almost in 90 degrees of rotation. What’s more, we can detect small people in the scene while general detectors fail.

12. Main Contributions
Proposed a Rotation-Invariant GP-Pooling operator, which can be used to encode radial distribution of features
Proposed a Rotational Rectification Network (R2N) that can be inserted into a wide range of CNN-based detectors to achieve rotational invariance
In conclusion, our main contribution in this project is two fold. First, we proposed a rotati………. Secondly, …… Evaluation results show that our rotational rectification network is capable of detecting heavily rotated pedestrian while state-of-the-art detectors fail.