CCU VISION LABORATORY Object Speed Measurements Using Motion Blurred Images 林惠勇 中正大學電機系

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 2 Images …

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 3 Blur Images … Defocus blur: Motion blur:

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 4 What Do They Tell Us? Motion of Object Region of Interest:

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 5 Information from Blur Images Two types of image blur:  Defocus blur – due to the limitation of optical sensors Image restoration Identification of region of interest Depth measurement  Motion blur – due to the relative motion between the camera and the scene Image restoration Motion analysis Increase still resolution from video Special effect Speed measurements? From the movie: “Chicken Run”

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 6 Defocus Blur Blur circle

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 7 Motion Blur

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 8 Speed Measurements Why measure speed? (motivation)  Wind  Experiments  Sports (baseball, tennis ball), athletes  Vehicle speed detection How?  RADAR (Radio Detection And Ranging)  LIDAR (Laser Infrared Detection And Ranging)  GPS  Video-Based Analysis

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 9 Image-Based Speed Measurement Key idea:  For a fixed camera exposure time: Relative motion between object and static camera Motion blur appeared in the dynamic image region

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 10 Geometric Formulation Simple pinhole camera model: Key components:  Focal length, exposure time, CCD pixel size  Object distance, blur length (blur extent)

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 11 Image Degradation Image degradation – linear space invariant system  Characterized by its point spread function (PSF) h(x,y) Degradation under uniform linear motion (whole image) How about space variant case? (partial blur & total blur)

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 12 Blur Parameter Estimation Edge detection ABC:  Sharp edge  step response  Blur edge  ramp response How to use this fact to estimate blur extent?

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 13 Image Deblurring If H is linear, space invariant:  Inverse filtering  Wiener filter Bad news:  Our case is space variant  Region segmentation Degradation function H Degradation function H Restoration filter(s) Restoration filter(s) + + f(x,y)f(x,y) g(x,y)g(x,y) Noise  (x,y) f(x,y)f(x,y) Degradation Restoration

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 14 More General Case – I What if the object is not moving parallel to the image scanlines?  Motion direction estimation  Image rectification

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 15 Motion Direction Estimation Fourier spectrum analysis: It can also be implemented in spatial domain

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 16 More General Case – II What if the object is not moving parallel to the image plane?

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 17 Extended Camera Model

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 18 Required Parameters Intrinsic camera parameters  Focal length, CCD pixel size, exposure time Extrinsic camera parameters  Distance to the object, camera orientation Softball speed measurement  Size of the softball (physical measurement) Vehicle speed detection – “parallel case”  Length of the vehicle (from manufacturer’s data sheet) Vehicle speed detection – “non-parallel case” ?  How to obtain the parameters z, , etc.?

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 19 Vehicle Speed Detection Parameters:  K = 22 pixels, s x = 11  m, f = 10 mm, T = 1/160 sec.  l = 560 pixels, L = 4750 m Detected speed – km/hr Video-based speed – km/hr, speed limit – 110 km/hr

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 20 Camera Pose Estimation Theorem:  Given a parallelogram in 3-D space with known image projection of four points, their relative depths can be determined. To obtain the unknown scale factor:  Absolute metric between two 3-D points  License plate with standard size

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 21 Vehicle Speed Detection Parameters:  K = 22 pixels, s x = 6.8  m, T = 1/400 sec., l = 560 pixels, L = 4750 m  W = 320 mm,  = , f = 26 mm Detected speed – km/hr Video-based speed – km/hr

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 22 Fully Automated? How? Intrinsic camera parameters?  JPEG EXIF header Target identification  Motion blur analysis Region segmentation  Region growing  Additional image capture Robust blur extent estimation Image synthesis  Deblurred target region + static background region

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 23 Initial Target Segmentation Horizontal ramp edge detection Run-length coding or projection Vertical continuity checking Multiple direction analysis

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 24 Spherical Object in Motion Problems on parameter estimation  Accuracy, robustness, precision (subpixel resolution…) Spherical object  circular from any viewpoint Initial blur extent identification + circle detection  Circle fitting, Hough transform More problems  Motion blur due to rotation, three-dimensional translation, shading, etc.

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 25 Speed Measurement Flowchart

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 26 Motion Direction Estimation Camera pose estimation – non-parallel case Two or more captures with fast shutter speed  Vertical projection  Post-processing  Fixed object size  Could be blurred

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 27 Softball Speed Measurement Parameters:  K = 26 pixels, T = 1/320 sec., l = 72 pixels, d = mm Detected speed – 40.5 km/hr Video-based speed – 40.9 km/hr

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 28 Conclusion Object speed measurement using a single motion blurred image  Vehicle speed detection  Softball speed measurement Advantages  Low cost – off-the-shelf digital camera  Passive device – can avoid anti-detection  Passive device – no radiation, light  Large measurement range – through adjustable shutter speed Limitation  Lighting condition  Accuracy?

L L C C V V H.Y.Lin, CCUEE CCU Vision Lab 29 Thank you for your attention! Any questions?