Neural Network Pipeline CONTACT & ACKNOWLEDGEMENTS Reconstruction-free deep convolutional neural networks for partially observed images Arun Asokan Nair*, Luoluo Liu*, Akshay Rangamani*, Peter Chin*^, Muyinatu A. Lediju Bell *+#, Trac D. Tran* *Department of Electrical and Computer Engineering, +Department of Computer Science, #Department of Biomedical Engineering - Johns Hopkins University ^Department of Computer Science - Boston University Introduction Neural Network Pipeline We propose a framework to extract information from visual data with an unknown fraction of pixels missing using CNNs, without performing reconstruction or re-training the neural network fon every possible partial observation ratio. We prove that by training a neural network on a few observation ratios, it generalizes to unseen observation ratios for both classification and object detection tasks. Experiments Image Classification on CIFAR-10 using VGG-16 Object Detection on Pascal VOC 2007 using Faster R-CNN [3] Fig. Image classification on partially observed images. During training, we input fully sampled images and images with missing pixel ratios of 0.5, 0.25, 0.125 to a VGG-16[2] network. The test data to the network has observation ratios randomly generated between 0 and 1. Networks were trained using SGD with momentum=0.9, learning rate=0.1, learning rate decay=10-6 and batch size=128 for 250 epochs Conclusions Benefits of our approach: Faster than reconstruct-then-classify Higher accuracies for challenging partial observation ratios (0.2 and 0.1). Generalizes to arbitrary unseen observation ratios. Motivation Pixel-wise Coded Exposure (PCE) Imaging [1] is a technique used to sense and record video sequences under power and sensor size constraints. The PCE measurements are close to partially observed frames, which we study in this work through the classification and object detection tasks. References Results [1] Zhang, J., Xiong, T., Tran, T., Chin, S., & Etienne-Cummings, R. (2016). Compact all-CMOS spatiotemporal compressive sensing video camera with pixel-wise coded exposure. Optics express, 24(8), 9013-9024. [2] Karen Simonyan and Andrew Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv:1409.1556, 2014. [3] Shaoqing Ren et al., “Faster R-CNN: Towards real-time object detection with region proposal networks,” in Advances in Neural Information Processing Systems, 2015 Exemplars Object Detection Results from a Faster-RCNN [3] network trained on different partial observation ratios Observation ratio: 50% Observation ratio: 25% Fig. Testing times Fig. Averaged classification accuracies Fig. Mean Average Precision(mAP) CONTACT & ACKNOWLEDGEMENTS Contact author – Arun Asokan Nair & Akshay Rangamani E-mail – anair8@jhu.edu, rangamani.akshay@jhu.edu This work is partially supported by the National Science Foundation (NSF) under Grant CCF-1422995. Table: Averaged classification accuracies with various partial observation ratios -> DICE score!! ------------------------ Text and figures could be more integrated.