1. Neural Network Pipeline CONTACT & ACKNOWLEDGEMENTS
Reconstruction-free deep convolutional neural networks for partially observed images
Arun Asokan Nair*, Luoluo Liu*, Akshay Rangamani*, Peter Chin*^, Muyinatu A. Lediju Bell *+#, Trac D. Tran*
*Department of Electrical and Computer Engineering, +Department of Computer Science, #Department of Biomedical Engineering - Johns Hopkins University
^Department of Computer Science - Boston University
Introduction
Neural Network Pipeline
We propose a framework to extract information from visual data with an unknown fraction of pixels missing using CNNs, without performing reconstruction or re-training the neural network fon every possible partial observation ratio. We prove that by training a neural network on a few observation ratios, it generalizes to unseen observation ratios for both classification and object detection tasks.
Experiments
Image Classification on CIFAR-10 using VGG-16
Object Detection on Pascal VOC 2007 using Faster R-CNN [3]
Fig. Image classification on partially observed images. During training, we input fully sampled images and images with missing pixel ratios of 0.5, 0.25, to a VGG-16[2] network. The test data to the network has observation ratios randomly generated between 0 and 1.
Networks were trained using SGD with momentum=0.9, learning rate=0.1, learning rate decay=10-6 and batch size=128 for 250 epochs
Conclusions
Benefits of our approach:
Faster than reconstruct-then-classify
Higher accuracies for challenging partial observation ratios (0.2 and 0.1).
Generalizes to arbitrary unseen observation ratios.
Motivation
Pixel-wise Coded Exposure (PCE) Imaging [1] is a technique used to sense and record video sequences under power and sensor size constraints. The PCE measurements are close to partially observed frames, which we study in this work through the classification and object detection tasks.
References
Results
[1] Zhang, J., Xiong, T., Tran, T., Chin, S., & Etienne-Cummings, R. (2016). Compact all-CMOS spatiotemporal compressive sensing video camera with pixel-wise coded exposure. Optics express, 24(8),
[2] Karen Simonyan and Andrew Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv: , 2014.
[3] Shaoqing Ren et al., “Faster R-CNN: Towards real-time object detection with region proposal networks,” in Advances in Neural Information Processing Systems, 2015
Exemplars
Object Detection Results from a Faster-RCNN [3] network trained on different partial observation ratios
Observation ratio: 50%
Observation ratio: 25%
Fig. Testing times
Fig. Averaged classification accuracies
Fig. Mean Average Precision(mAP)
CONTACT & ACKNOWLEDGEMENTS
Contact author – Arun Asokan Nair & Akshay Rangamani –
This work is partially supported by the National Science Foundation (NSF) under Grant CCF
Table: Averaged classification accuracies with various partial observation ratios
-> DICE score!!
Text and figures could be more integrated.