Object Detection based on Segment Masks Facebook AI Research Wenchi Ma Data: 11/04/2016

More information from object detection

More information from object detection

More information from object detection

Object Detection for now with Deep Learning

New technology: A MultiPath Network for Object Detection Data: COCO Data Proposals Convolutional Layers Classification Loss arXiv:1604.02135v2[cs.CV] 8 Aug 2016 Proposals: Deep Mask One Single Network Convolutional Layers: VGG16 Trunk RCNN Classification: VGG16 Classifier Loss: Integral Loss Proposal: Find and locate all possible objects and then scale down its range(most possible)

A MultiPath Network for Object Detection Dataset: COCO Microsoft COCO: Common Objects in Context, ECCV 2014 Facebook: Learning to Segment Object Candidates, NIPS 2015 Proposals: Deep Mask Facebook: Learning to Refine Object Segmentation, EECV 2016 Cornell University and Microsoft Research: Skip-layer connections and RoI pooling, CVPR 2015 Convolutional Layers: VGG16 Trunk University Paris Est: Object detection via a multi-region & semantic segmentation-aware CNN model, ICCV 2015 Classification: VGG16 Classifier Loss: Integral Loss Proposed

Microsoft COCO DataSet Objects are labeled using per-instance segmentations to aid in precise object localization Contain 91 objects types A total of 2.5 million labeled instances in 328k images Contains objects at a broad range of scales, including a high percentage of small objects Objects are less iconic, often in non-standard configurations and amid clutter or heavy occlusion Requires more precise localization

Deep Mask Proposal(a single convolutional network) Predicts a segmentation mask given an input patch, and assign a score corresponding to how likely the patch is to contain an object (Top): The top branch predicts a segmentation mask for the object located at the center while the bottom branch predicts an object score for the input patch (Bottom): Green patches contain objects and are assigned the label y=1 while the red ones are negative examples are not used The patch contains an object roughly centered in the input patch The object is fully contained in the patch and in a given scale range

Proposed Multipath architecture “Multi-stage”(skip connection): capture multi-resolution feature maps: detecting objects at multiple scales(small objects) Integral Loss concatenated RoI-Pooling: Generate feature maps with different sizes of contexts: improve localization accuracy Each RoI is pooled into a fixed-size feature map and then mapped to a feature vector by fully connected layers(FCs)

Integral Loss: more reliable clssificaiton PASCAL and ImageNet COCO Fast R-CNN Proposed If the proposal overlaps a ground truth box with IoU greater than 50, the true class k* is given the class of the ground truth, otherwise k*=0 Ideally, proposals with higher IoU to the ground truth should be scored more highly The combined loss is computed for every object proposal n=6 The output softmax probabilities pu of each of the n classifiers are averaged to compute the final class probability p

Detection Results

Detection Results

Evaluation Use AP to denote AP averaged across IoU values from 50 to 95, and APu to denote AP at IoU threshold u Each contributes roughly equally to final accuracy, and in total AP increases 2.7 points to 27.9 4-region foveal setup versus the 10 regions. Surprisingly, the former outperforms despite using fewer regions. The integral loss yields good results at all settings Integral loss achieves best at AP with 6 heads

Evaluation This method placed second in both the bounding box and segmentation tracks Only the deeper ResNet classifier outputformed this approach (potentially ResNet could be integrated as the feature extractor in this MultiPath network) Compared to the baseline Fast R-CNN, this method showed the largest gains on small objects and localization

Evaluation

Thank you !