Deep screen image crop and enhance Week 2 (Aaron Ott, Amir Mazaheri)

Problem We have taken a photo of an image, and we want the original image. This can be broken into 2 parts: Image Detector/Cropper Image Enhancer

Enhancer - Pretrained EDSR (trained on DIV2K) https://github.com/krasserm/super-resolution - Pretrained EDSR (trained on DIV2K) - Modified form of Resnet - Actually scales up the image 4x Lim, Son, Kim, Nah, Lee. “Enhanced Deep Residual Networks for Single Image Super-Resolution”. 10 July 2017

Combining the Cropper and Enhancer Had to use lambda functions and AveragePooling2D to get the Enhancer to properly work with the Cropper Loss Functions: - Trained Cropper on VGG+CosineProximity - Trained Enhancer on VGG+MSE

Last Week (Cropper) Metrics: PSNR: 11.1903 SSIM: 0.4254 MSE: 0.0796 Last Week (Cropper) Cropper using a Spatial Transformation Network Input Output Actual

This Week (Cropper + Enhancer) Metrics: PSNR: 15.8368 SSIM: 0.4807 MSE: 0.0316 This Week (Cropper + Enhancer) Input Output Actual

Metrics Compared Model \ Metric PSNR SSIM MSE Cropper 11.1903 0.4254 0.0796 Cropper + Enhancer 15.8368 0.4807 0.0316

Shortcomings of PSNR, SSIM, and MSE Model \ Metric PSNR SSIM MSE Cropper 11.1903 0.4254 0.0796 Cropper + Enhancer 15.8368 0.4807 0.0316 Cropper + Enhancer w/ MSE Loss 18.2626 0.5314 0.0166

Shortcomings of PSNR, SSIM, and MSE Metrics: PSNR: 18.2626 SSIM: 0.5314 MSE: 0.0166 Input Output Loss Functions VGG + Cosine Proximity MSE Actual

Next Week Create and refine GAN architecture Refine EDSR or try a different SISR (WDSR)