1. Single Image Super-Resolution
Adam Vest

2. Problem Overview
Challenging problem → difficult to infer the values for multiple pixels from a single pixel
Significant advances in the last two years due to deep learning methods
First deep learning systems used ConvNets, recent state-of-the-art uses GANs
Goal is to find some mapping f(ILR) = ISR where ISR is as close as possible to IHR

3. The First Wave: ConvNets
Convolutional networks made great progress in SISR
Several variants over the years:
SRCNN
DRCN
ESPCN
General take-aways: learning upscaling leads to dramatic increases in performance, deeper networks performed better
Suffers from blurry image outputs/cannot capture image textures well due to reliance on L1/L2 losses
SRResNet is the current state-of-the-art ConvNet for SISR

4. SRGAN First model to use adversarial learning for super-resolution
16 block deep ResNet structure + 2 PixelShuffle blocks (Transpose Conv.)
Utilized a content loss (MSE or VGG) and an adversarial loss (BCE) to produce realistic images
Pretty good PSNR, very good Mean-Opinion-Score

5. Project Goal
To use adversarial learning to improve on the current state-of-the-art in SISR
Build on the SRGAN baseline by exploring different choices for network architectures, the merits of different loss functions, etc.
Implement an existing/design a new adversarial model that has not yet been used for SISR

6. Where Are We Now? Downloaded ImageNet and other datasets
Setup on the cluster
Modular implementations of all SRResNet and SRGAN variants
Trained SRResNet on ImageNet, currently training SRGAN
Implemented evaluation suite for BSD100, Urban100, Set5, and Set14 datasets
Tested SRResNet implementation on evaluation suite

7. SRResNet Implementation ResultsLR SR HR

8. SRResNet Implementation Results Cont.
PSNR Results for 4x Upscaling
Dataset
Ours (RGB)
Theirs (Y-Channel)
Set5
29.90
32.05
Set14
26.42
28.49
BSD100
26.15
27.58
Urban100
24.31
Not Reported

9. Where Are We Going? SRGAN + Wasserstein discriminator
SRGAN + PatchGAN discriminator?
Do super-resolution on Y-channel of the image only
Investigate other adversarial models/network structures
Investigate different evaluation metrics → SSIM
Could lead to different choices for loss functions

10. Thank you! Questions?