Fast Direct Super-Resolution by Simple Functions Chih-Yuan Yang and Ming-Hsuan Yang 12/14/13

Outline Introduction Related work Proposed method Experimental results Conclusions

Introduction Super-Resolution = Predicting pixel intensities From single or multiple image(s) For spatial or temporal intestines 480 120 320 80

Challenges Effectiveness Stability Computational load Numerous pixel intensities Stability Various image content Computational load

Related Work - Bicubic Interpolation Predicting intensities by interpolating nearby pixels Simple and fast But over-smooth

Self-Similar Exemplars [Glasner 11] Predict HR patches through example patches found in a self-generated image pyramid Sharp and clear edges Blurred textures Slow I1 I0 I-1 I-3 I-2 I2 I3 I4 ICCV

Gradient Profile Prior [Sun08] Predict intensities by a edge model Sharp edges, but jaggy Only effective for edges

Sparse Representation [Yang08] Predicting HR patch features by learned sparse dictionaries Rich details Noise artifacts along edges

Proposed Method Split LR feature space for effectiveness Use simple features and simple functions for speed Exploit a large image set to collect training samples Asymmetric computational load slow for training fast for test

Training Phase (1) Generate LR images Generate LR images from HR training images 𝐼 𝑙 = 𝐼 ℎ ⊗𝐺 ↓ 𝑠 Extract patch pairs in LR and HR Since there is a convolution for HR images, we crop LR patches affected by predicted HR pixels (blue region)

Training Phase (2) Extract LR/HR features LR: LR intensity minus the mean of the LR patches HR: HR intensity minus the mean of the LR patches High-frequency information Cluster the LR training features by K-mean split the LR feature space into K subspaces

Cluster Centers and Patch Numbers

Training Phase (3) Collect training instances Randomly select sufficeint LR/HR feature pairs for each cluster For some cluster centers, use bicubic interpolation if no sufficient instances available

Training Phase (4) Learn regression coefficients C: coefficients W: HR features (1000 instances) V: LR features (1000 instances)

Test Phase Crop LR patches Compute patch mean, and extract LR features Find the closest cluster center Compute HR features HR patch intensity = HR feature + LR patch mean Average overlapped HR patches as output

Experimental Results - Child

Experimental Results - Lena

Experimental Results - IC No groundtruth image

Experimental Results - Mansion

Experimental Results - Mermaid

Experimental Results - Shore

Experiments on BSD200 dataset

Performance (averaged) and Execution Time for BSD200 Dataset

Conclusions Clear and sharp edges Rich texture details Easy for implementation Fast to generate SR images

Insight Split of the LR feature space K has to be large enough Exploit a large image set to collect sufficient examples for high-frequency patches Regression methods make little difference

Code and Dataset Available https://eng.ucmerced.edu/people/cyang35 Including code for training phase