Robust Super-Resolution Presented By: Sina Farsiu

Project Goals Understanding & simulation of Dr. Assaf Zomet, et.al. paper : “Robust Super-Resolution” [1] Comparing the results obtained by this method to other methods Enhancing the method introduced in: [1]

Super-Resolution Objective Generation of a high-resolution image from multiple low-resolution moving frames of a scene.

Super-Resolution Formulation

Solutions for S-R Problem No Noise: With Noise

Problem with These Solutions In the presence of outliers (error in motion estimation, inaccurate blur model, pepper & salt noise, …), these methods do not work accurately. Robust S-R methods can help in these situations.

Robust S-R Formulation

Robustness

Why Median? Median is an estimate of mean. Unlike regularization method only one of low resolution frames contributes to reconstruct each pixel in the high-resolution frame. So outliers in other frames are discarded in the reconstruction process. Claim: In the absence of additive noise to all frames median method works better than regularization method.

What if noise is added to all frames? Claim: If considerable amount of additive noise is present in all frames regularization method works as good or even better than median method.

Median-Average Reconstruction Instead of using We can combine average and median operators to get better results.

Bias Detection Procedure It is useful to detect the outliers in the low resolution frames. We can omit those outlier pixels in our procedures. The difficulty is to differentiate between aliasing and outlier effects.

Bias Detection Formulation After thresholding non zero values are due to aliasing or outliers.

Bias Detection Result Due to outlierDue to aliasing

Bias detection Procedure 1: compute 2: Threshold 3: Filter the result with a LPF 4: Threshold 5: Omit corresponding pixels from super-resolution procedure

Bias Detection B-D method works only for uniform gray level outliers. In many situations median operation in robust super-resolution eliminates the bias of the estimator.

Original L-R Frame

Robust S-R Reconstructed H-R Frame mse=0.0017

Median Reconstruction after adding noise

Regularized S-R Reconstructed H-R Frame mse=0.0131`

Regularized Reconstruction after adding noise mse=.0125

Error Due to Outlier

Error Due to Aliasing

Conclusion Robust super-resolution method is quite effective in the presence of outliers, and produces better results in comparison with regularization method. In the presence of additive noise in all low –resolution frames this method loses its superiority to the regularization method.

Conclusion & Results Combination of mean and median operators can help us in this situation. Proposed bias detection algorithm is an effective method to detect outliers. If outliers are the only source of error in the L-R frames(no additive noise), more iterations we use smaller mse we will achieve.

Suggestions for Future Research Combining regularization and robust super-resolution methods. Using bias detection results in regularization method. Using robust super-resolution method in frequency domain.

Acknowledgment Thanks to Dr.Assaf Zomet, Dr.Michael Elad, Dirk Robinson and Dr. Peyman Milanfar for their valuable advices & suggestions.

references "Robust Super Resolution", A. Zomet, A. Rav-Acha and S. Peleg Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, December "Efficient Super-Resolution and Applications to Mosaics", A. Zomet and S. Peleg, Proceedings of the International Conference on Pattern Recognition (ICPR), Barcelona, September 2000.

“A Computationally effective Image super- resolution Algorithm”, Nguyen, N., P. Milanfar, G.H. Golub, IEEE Transactions on Image Processing, vol. 10, no. 4, pp , April 2001 “A Fast Super-Resolution Reconstruction Algorithm for Pure Transnational Motion and Common Space Invariant Blur”, M. Elad and Y. Hel-Or, the IEEE Trans. on Image Processing, Vol.10, no.8, pp , August 2001.

Thank You All

Additional Simulatins

Original H-R Frame

Blured median

Projected L-R frame

L-R Frame

Regularization with high noise mse=.0481

Median with high noise mse=.0693

Composite Median & Average MSE=0.0592

Original H.R. Frame

L.R. Frame Before Adding Noise

L.R. Frame After Adding Noise

Regularized Reconstruction MSE=.0216

Median Reconstruction MSE=.0077