1. Feng Liu, JinjunWang,ShenghuoZhu (MM’08) University of Wisconsin-Madison, NEC Laboratories America, Inc. 第一組： 資訊四 B95902105 黃彥達 資訊碩一 R98922046 蔡旻光 網媒碩二 R97944012 鄒志鴻

2.  Introduction  Goal  File Format  Noise Reduced Image  Proposed Approach  Motion Estimation & Estimated Super- Resolution Result  Implementation  Result  Conclusion 2

3.  Low-quality videos often not only have limited resolution but also suffer from noise  In fact, the requirements of de-noising & super-resolution is quite similar  This paper present a unified framework which achieves simultaneous video de-noising and super- resolution algorithm by some measurements of visual quality 3

4.  Refine low-quality videos from YouTube, and make the video better effects, which has better quality by human eyes.  Input is low-quality and noise-included (block effects or somewhat noise) videos

5. .3gp file  Frame rate: 15(our video) or 25  Frame size: 176(w) * 144(h)  MPEG-4 Part 12  It is used on 3G mobile phones also can be played on 2G and 4G phones.  Our video: 867 KB/ 98 sec 5

6. mv-SADGaussian-spaceGaussian-time | p(I,j) – p(i’, j’) | > threshold

7. Frame t Pixel(I,j) Standard deviation Set Mean = 0

8. Frame t Pixel ( i, j, t) Frame t+1 Pixel ( i + mv_i, j + mv_j, t+1) (mv_i, mv_j)

9. Frame t - 2Frame t - 1Frame t Space Gaussian Time Gaussian Pixel(I,j) Frame t+1Frame t+2Frame t Shot Detection

10. BeforeAfter

11.  Consider the visual quality with respect to the following 3 aspects:  Fidelity Preserving ▪ To achieve similar high-resolution result  Detail Preserving ▪ Enhanced details (edge)  Spatial-Temporal Smoothness ▪ Remove undesirable high-frequency contents (e.g. jitter) 11

12.  Fidelity Preserving  Conventional metrics: ▪ Measure fidelity by the difference between I h & I l would be problematic & waste useful time-space information in video  Proposed metrics: ▪ Estimate an approximation of super-resolution results from space-time neighboring pixels ▪ The fidelity measurement: see next page for details noised 12

13.  Detail Preserving  Enhanced details (edge)  Contrast preserving ▪ Human visual system is more sensitive to contrast than pixel values ▪ Gradient fields of I h & should be close,where W k is one or zero if the patch k with/o edges (canny detector) 13

14.  (Spatial-Temporal) Smoothness  Smooth results are often favored by the human system  Encourage to minimize:  A 2-D Laplace filter may be Spatial-temporal Laplacian OR 14

15.  Proposed Measurements  A quadratic minimization problem to solve (AX = b): Contrast Similarity Detail Information(edge) Spatial-Temporal Smoothness 15

16. Input low = 6 -1 … -1 -1 6 -1 … -1 Laplacian Gradient -1 0 1 … 1 Edge Minimize Motion Estimation + + Result (X) Fidelity Gaussian filter 16

17. 17 = 6 -1 … -1 -1 6 -1 … -1 Laplacian Gradient -1 0 1 … 1 Edge Minimize Fidelity by sparse least square solver

18.  Adjustments for the weight terms  The measurement term is more emphasized if the weight is larger  By iteratively experiments for our test data, we took  However, we found that the best weight set may be different for different videos 18

19.  352 x 288 Result 19

20.  352 x 288 Result 20

21.  352 x 288 Result 21

22.  The proposed framework formulates noisy video super-resolution as an optimization problem, aiming to maximize the visual quality  By exploiting the space-temporal information, we can estimate a better baseline than conventional fidelity measurement  The properties include fidelity-preserving, detail- preserving and smoothness are considered to achieve the best visual quality results 22

23. 23