1. Object Detection based on Segment Masks
Facebook AI Research
Wenchi Ma
Data: 11/04/2016

2. More information from object detection

3. More information from object detection

4. More information from object detection

5. Object Detection for now with Deep Learning

6. New technology: A MultiPath Network for Object Detection
Data:
COCO
Data
Proposals
Convolutional Layers
Classification
Loss
arXiv: v2[cs.CV] 8 Aug 2016
Proposals: Deep Mask
One
Single
Network
Convolutional Layers: VGG16 Trunk
RCNN
Classification:
VGG16 Classifier
Loss:
Integral Loss
Proposal: Find and locate all possible objects and then scale down its range(most possible)

7. A MultiPath Network for Object Detection
Dataset:
COCO
Microsoft COCO: Common Objects in Context, ECCV 2014
Facebook: Learning to Segment Object Candidates, NIPS 2015
Proposals: Deep Mask
Facebook: Learning to Refine Object Segmentation, EECV 2016
Cornell University and Microsoft Research: Skip-layer connections and RoI pooling, CVPR 2015
Convolutional Layers: VGG16 Trunk
University Paris Est: Object detection via a multi-region & semantic segmentation-aware CNN model, ICCV 2015
Classification:
VGG16 Classifier
Loss:
Integral Loss
Proposed

8. Microsoft COCO DataSet
Objects are labeled using per-instance segmentations to aid in precise object localization
Contain 91 objects types
A total of 2.5 million labeled instances in 328k images
Contains objects at a broad range of scales, including a high percentage of small objects
Objects are less iconic, often in non-standard configurations and amid clutter or heavy occlusion
Requires more precise localization

9. Deep Mask Proposal(a single convolutional network)
Predicts a segmentation mask given an input patch, and assign a score corresponding to how likely the patch is to contain an object
(Top): The top branch predicts a segmentation mask for the object located at the center while the bottom branch predicts an object score for the input patch
(Bottom): Green patches contain objects and are assigned the label y=1 while the red ones are negative examples are not used
The patch contains an object roughly centered in the input patch
The object is fully contained in the patch and in a given scale range

10. Proposed Multipath architecture
“Multi-stage”(skip connection): capture multi-resolution feature maps: detecting objects at multiple scales(small objects)
Integral Loss
concatenated
RoI-Pooling:
Generate feature maps with different sizes of contexts: improve localization accuracy
Each RoI is pooled into a fixed-size feature map and then mapped to a feature vector by fully connected layers(FCs)

11. Integral Loss: more reliable clssificaiton
PASCAL and ImageNet
COCO
Fast R-CNN
Proposed
If the proposal overlaps a ground truth box with IoU greater than 50, the true class k* is given the class of the ground truth, otherwise k*=0
Ideally, proposals with higher IoU to the ground truth should be scored more highly
The combined loss is computed for every object proposal
n=6
The output softmax probabilities pu of each of the n classifiers are averaged to compute the final class probability p

12. Detection Results

13. Detection Results

14. Evaluation
Use AP to denote AP averaged across IoU values from 50 to 95, and APu to denote AP at IoU threshold u
Each contributes roughly equally to final accuracy, and in total AP increases 2.7 points to 27.9
4-region foveal setup versus the 10 regions. Surprisingly, the former outperforms despite using fewer regions.
The integral loss yields good results at all settings
Integral loss achieves best at AP with 6 heads

15. Evaluation
This method placed second in both the bounding box and segmentation tracks
Only the deeper ResNet classifier outputformed this approach (potentially ResNet could be integrated as the feature extractor in this MultiPath network)
Compared to the baseline Fast R-CNN, this method showed the largest gains on small objects and localization

16. Evaluation

17. Thank you !