Video Enhancement with Super-resolution 陳彥雄

Outline Introduction Bayesian MAP-based SR Exampled-based SR Ending

Introduction What is Super-Resolution (SR)? SR is an image-processing technology that enhance the resolution of an image system. SR fuses several low-resolution (LR) images together into one enhanced- resolution image.

SR vs. Interpolation & Filter Traditional interpolation methods, like bilinear, cubic splines, are applied to a single picture. But they add no additional information to high-frequency ranges. We can use filters sharpening up image details, but they also amplify noise. SR combines information form multiple sources.

SR in Video We can divide Video into several groups of picture, each GOP contains lots of similar content with block (object) motion. Since GOP contains lots of similar content, SR enhancement is achievable.

SR example From Wikipedia:

SR example Following videos source from: Analysis/Demos/SpaceTimeSR/SuperRes_demo s.html Analysis/Demos/SpaceTimeSR/SuperRes_demo s.html

SR example

Bayesian MAP-based SR

Maximum a Posteriori The following MAP example sources from: >

Maximum a Posteriori You have a bag of candy, which is one of follows: H1: 100%cherry H2: 75%cherry + 25%lime H3: 50%cherry + 50%lime H4: 25%cherry + 75%lime H5: 100%lime Which bag is at most possible if you get 2 lime candy from it? And what if the possibility of each bag is {H1,H2,H3,H4,H5} = {0.1, 0.2, 0.4, 0.2, 0.1}

Maximum a Posteriori

Bayesian MAP SR Notation

The Video Observation Model

Bayesian MAP SR Notation

Bayesian Maximum a posteriori

Exampled-based SR

Also called single-frame super resolution. Use data learning technology. It contains a training phase. Effectiveness depend by data.

Basic Idea Image can be decomposed by frequency into low, median and high. The low part of an Image is independent from the high ones. When an image is up scaled, it loses high frequency information. We can patch the high part of an upscale image from trained patch dictionary.

Training

SR Algorithm

SR result

Ending

Further Discussion Motion Vector matters! Image sequences or compressed video? Video on example-based SR? Other SR method?