1. Deep screen image crop and enhance
Week 1
(Aaron Ott, Amir Mazaheri)

2. Problem Image Detector/Cropper Image Enhancer
We have taken a photo of an image, and we want the original image.
This can be broken into 2 parts:
Image Detector/Cropper
Image Enhancer
and

3. Spatial Transformer Networks
Jaderberg, Simonyan, Zisserman, Korayk. “Spatial Transformer Networks”, 4 Feb 2016
Localization Network – generate Theta
Grid Generator – Maps pixels for transformation
Sampler – Creates transformed image

4. Spatial Transformer Networks (cont.)
Benefits
Localization network is fully trainable
Module can be included in any existing model
Uses:
reshaping input image to better focus on important regions
attention mechanism
cropping images?

5. Dataset
10 Images (100 Samples, 90 training – 10 validation) - (Caltech-UCSD Birds 200)

6. Loss Function

7. Results Validation Loss: 0.1195 => 0.05349 over 200 epochs
(less than 5 minutes of training)

8. Output
Input
Output
Ground Truth

9. Image Enhancement: Existing Research
Photo Enhancement
Super Resolution
SRGAN
EDSR
WDSR
Chen, Wang, Kao, Chuang. “Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs”, CVPR 2018
Ledig, Theis, Huszar, Caballero, Cunningham, Acosta, Aitken, Tejani, Totz, Wang, Shi. “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”. 25 May 2017
Lim, Son, Kim, Nah, Lee. “Enhanced Deep Residual Networks for Single Image Super-Resolution”. 10 July 2017
Yu, Fan, Yang, Ju, Wang, Wang, Huang. “Wide Activation for Efficient and Accurate Image Super-Resolution”. 21 December 2018

10. Next Steps Fine-tune the photo cropper
Test different localization network hyperparameters
Loss function and optimizer
Build a more difficult data set to test the cropper on
Validate on new images
Further distance away
More background noise
Change location within image
Begin working on Image Enhancement
Connect Cropper to existing image enhancement network
Fine tune image enhancement networks