Download presentation
Presentation is loading. Please wait.
Published byRandall Jones Modified over 8 years ago
1
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Amit Agrawal and Ramesh Raskar Mitsubishi Electric Research Labs (MERL) Cambridge, MA, USA Resolving Objects at Higher Resolution from a Single Motion-Blurred Image
2
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Can we recover more detail by letting objects blur? Low Res Camera: Static Object Object Moves Text Low Res Camera: Blurred Image Blurred Text: 30 pixels wide Our approach: Resolution enhanced deblurring Resolution enhancement factor = 3 Traditional Motion Deblurring: Recovers 10 pixels 10 pixels 30 pixels Text: 10 pixels wide
3
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & RaskarApplications Time = 0Time = T Aerial Imaging Goal: Automatic license plate number recognition Electronic Toll Booths
4
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & RaskarOverview Exploiting motion blur for super-res –Linear system –No hallucination, image priors or training data Motion deblurring is ill-posed –Practical Solution: Flutter Shutter Camera Resolution enhanced deblurring –Using Flutter Shutter Camera
5
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Sensor point spread function (PSF) Ground Truth Static Scene (High Resolution Camera) 20 pixel object … 0 0 0 1 1 0 0 0 … Decimation Matrix D Static Scene (Low Resolution Camera) 10 Decimation Matrix: High Resolution -> Low Resolution
6
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Exploiting motion blur for super-res Ground Truth Static Scene (High Resolution Camera) 20 pixel object Static Scene (Low Resolution Camera) 10 pixels Object Moves Traditional Motion Deblurring Blurred Image (High Resolution Camera) 40 Blurred Image (Low Resolution Camera) 20 pixels
7
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Blurring Matrix A Sharp Image x Blurred Image b Traditional Motion Deblurring 20 10 Blurred Image (Low Resolution Camera) 20 ? Static Scene (Low Resolution Camera) 10
8
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Blurring Matrix A Sharp Image x Blurred Image Low Resolution Traditional Motion Deblurring 10 Blurring Matrix A High Resolution … 0 0 0 1 1 0 0 0 … Decimation Matrix D Sharp Image x ? Resolution Enhanced Deblurring Blurred Image Low Resolution 20
9
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Deblurring is ill-posed K = D * A: Combined deblurring + resolution enhancement matrix Typical Case: Condition number of K ~ 10 18 Blurred Image + Noise K Estimated Ground Truth
10
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & RaskarOverview Exploiting motion blur for super-res –Linear system –No hallucination, image priors or training data Motion deblurring is ill-posed –Practical Solution: Flutter Shutter Camera [Raskar, Agrawal, Tumblin, SIGGRAPH 2006] Resolution enhancement deblurring –Using Flutter Shutter Camera
11
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Blurring == Convolution Traditional Camera: Shutter is OPEN: Box Filter Frequency Spectrum Magnitude Sinc Function ω
12
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Traditional Coded Exposure Image of Static Object Matlab Lucy Deblurred Image
13
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Implementation LCD Shutter in front of lens becomes opaque and transparent
14
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar License Plate Retrieval
15
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Related Work Super-Resolution & Motion Deblurring –Sondhi, Proc. IEEE 1972 –Irani & Peleg, CVGIP’91, CVPR’92 –Baker & Kanade, PAMI’02 –Ben-Ezra et al. PAMI 2005 –Bascle et al. ECCV’96 –Tappen et al. SCTV’03 –Shechtman et al. ECCV’02 Coding & Modulation in Cameras/Projectors –HDR, Nayar & Mitsunaga, CVPR’00 –Multiplexed Illumination, Schechner et al. ICCV’03 –Flutter shutter camera, Raskar et al. SIGGRAPH’06 –Coded aperture, Hiura & Matsuyama, CVPR’98 –Coded aperture with lens, light field modulation, SIGGRAPH’07 With Jack Tumblin, Ankit Mohan and Ashok Veeraraghavan (UMD)
16
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & RaskarOverview Exploiting motion blur for super-res –Linear system –No hallucination, image priors or training data Motion deblurring is ill-posed –Practical Solution: Flutter Shutter Camera –Raskar et al. SIGGRAPH 2006 Resolution enhancement deblurring –Using Flutter Shutter Camera
17
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar M. Ezra, A. Zomet & S.K. Nayar, ‘Video Super-Resolution Using Controlled Subpixel Detector Shifts’, PAMI 2005 Super-resolution with motion blur Jitter Camera Z. Lin & H.Y Shum, ‘Fundamental Limits of Reconstruction-Based Super- resolution Algorithms under Local Translation’, PAMI 2004
18
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Experimental Setup Resolution Chart is moved horizontally along the tracks
19
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Flutter Shutter: Shutter is OPEN and CLOSED Preserves High Spatial Frequencies
20
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Input Blurred Photo (cropped) Deblurred Image Super-Resolution Deblurring Result Deblurred Image Upsampled Unblurred Width Blurred Width Recovered at normal resolution Recovered at triple resolution
21
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Input Image Patch Deblurred Patch Our approach
22
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Future Work Exploiting Motion Blur –2D Motion Blur: Camera shake –Super-resolution with RGB Bayer mosaic –Coded exposure video with Pointgrey Dragonfly2 camera Focus blur –Coded Aperture –Handling motion and focus blur simultaneously Fluttering Codes –Binary vs Continuous –Suitable for other applications
23
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & RaskarAcknowledgments Jack Tumblin, Northwestern University Hideaki Nii, University of Tokyo MERL –Fatih Porikli, Jay Thornton –Joseph Katz, Keisuke Kojima Haruhisa Okuda, MELCO
24
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Super-res from single motion blurred photo Exploiting single photo motion blur –Letting objects blur in 1D –Solving simple linear system –More blur = more resolution but with increased noise Practical implementation –Coded Exposure (Flutter Shutter) Camera –Makes deblurring well-posed Results on 1D resolution enhancement –Using Flutter Shutter Camera
25
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Google: ‘Flutter Shutter’ Motion Deblurring Motion Deblurring + Resolution Enhancement Coded Exposure (Flutter Shutter) Camera
26
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Extra Slides
27
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Captured Blurred Photo Blurred Image Patch Traditional Deblurring Resolution Enhancement Deblurring Upsampled by three times
28
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Are all codes “good”? Alternate All ones Random Our Code
29
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Need to consider zero padded codes!!!
30
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar
31
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar
32
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Input Image
33
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Rectified Crop Deblurred Result
34
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Blurred image: Traditional camera open closed Exposure Time Shutter Continuous blur Blurred image: Coded exposure camera Exposure Time open closed Shutter Coded blur code 110 010100010011000 11 1111111111111111111111 Code: 1101101010001001100011
35
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Comparison
36
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Inverse Filter Unstable Inverse Filter stable
37
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Traditional Motion Deblurring Captured Blurred ImageDeblurred Image Deblurring is at the same resolution
38
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Traditional Camera Shutter is OPEN
39
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Our Camera Flutter Shutter
40
Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Shutter is OPEN and CLOSED
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.