1. Jernej Mrovlje Department of Systems and Control, Jožef Stefan Institute DISTORTION IMPACT ON A STEREO DISTANCE 10th International PhD Workshop on Systems and Control A Young Generation Viewpoint Hluboka, 25/9/09

2. INTRODUCTION RESULTS PROBLEM REALIZATION optical aberrations (distortion) camera calibration distortion model stereoscopy STEDIMAT application experiment results CONTENT INTRODUCTIONPROBLEMREALIZATIONRESULTS

3. STEREOSCOPY Stereoscopy 1. a technique used for recording and presenting 3D images 2. the viewing or appearance of objects in or as if in three dimensions Charles Wheatstone (1838): first scientist who described “stereopsis” “...the mind perceives an object of three dimensions my means of the two dissimilar pictures projected by it on the two ritinæ ” (C.Cheatstone) First stereoscopic images and stereoscope nowadays IMAX 3D (Image MAXimum 3D) INTRODUCTIONPROBLEMREALIZATIONRESULTS

4. RECORDING STEREOSCOPIC IMAGES 1)Stereo camera 2)Stereo attachment 3)Camera and stereo slider* 4)Stereo system (two single lens cameras joined together) (2) (1) (3) (4) INTRODUCTIONPROBLEMREALIZATIONRESULTS 1947 2009

5. VEIWING STEREOSCOPIC IMAGES parallel viewing (a) cross viewing (b) anaglyph images (c) polarized images (d) INTRODUCTIONPROBLEMREALIZATIONRESULTS (a) (b) (c) (d)

6. “STEDIMAT” APPLICATION STEreo DIstance MeAsuring Tool written in Matlab >> Matlab Compiler >> standalone application(*.exe) INTRODUCTIONPROBLEMREALIZATIONRESULTS

7. THE CALCULATION OF OBJECT’S POSITION - DISTANCE B – stereo base x 0 – horizontal image resolution φ 0 – horizontal angle of view x L – object’s position in the left image x R – objects’ position in the right image D INTRODUCTIONPROBLEMREALIZATIONRESULTS

8. OPTICAL ABERRATIONS chromatic and spherical aberrations affect image quality only tangential and radial distortions affect image geometry radial distortion causes inward or outward displacement of a given image point from its ideal location negative radial displacement >> barrel distortion positive radial displacement >> pincushion distortion If the object in the image is displaced, the calculated distance is incorrect! inexpensive cameras with wide-angle lenses suffer barrel distortion (Canon PowerShot A640) tangential distortion is due to imperfect centering of the lens components INTRODUCTIONPROBLEMREALIZATIONRESULTS

9. DISTORTION MODEL distortion can be compensated mathematically: 1. applying parametric distortion model 2. estimating distortion coefficients 3. correcting the distortion polynomial approximation model of the radial distortion (Brown): calibration procedure was done using “Camera Calibration Toolbox for Matlab” toolbox uses Brown’s distortion model known as “Plumb Bob” RADIAL DISTORTIONTANGENTIAL DISTORTION INTRODUCTIONPROBLEMREALIZATIONRESULTS

10. each camera was calibrated separately a sequence of 20 images of the calibration board was taken in different orientations CALIBRATION PROCESS in each image a set of calibration points were automatically detected using the coordinates of the distorted and undistorted calibration points, the distortion parameters k c were calculated INTRODUCTIONPROBLEMREALIZATIONRESULTS

11. CALIBRATION RESULTS left cameraright camera centre of the image: displacement < 10px corners of the image: displacement > 100px INTRODUCTIONPROBLEMREALIZATIONRESULTS

12. CORRECTING THE DISTORTION #1 distortion model was used to build “distortion-correction function” (DCF) as a part of Stedimat for each distance DCF has to be applied twice: 1. using image point of the object’s location in the left image (x L ) 2. using image point of the object’s location in the right image (x D ) INTRODUCTIONPROBLEMREALIZATIONRESULTS “EXPERIMENT”/equipment: stereoscopic system with 2 digital cameras Canon PowerShot A640 StereoData Maker was used to synchronize cameras

13. CORRECTING THE DISTORTION #2 “EXPERIMENT”: 7 test objects positioned at the distance D (30, 40, 50 and 60m) >>> four sets of stereoscopic images image resolution: 3648x2736, focal distance: 35mm, stereo base: 0.56m, reference object : test object no.4 INTRODUCTIONPROBLEMREALIZATIONRESULTS distance to each test object was calculated twice: undistorted/distorted image

15. CORRECTING THE DISTORTION #2 EXPERIMENT: 7 test objects positioned at the distance D (30, 40, 50 and 60m) >>> four sets of stereoscopic images image resolution: 3648x2736, focal distance: 35mm, stereo base: 0.56m, reference object : test object no.4 INTRODUCTIONPROBLEMREALIZATIONRESULTS distance to each test object was calculated twice: 1–distorted image points, 2-distortion-free image points

16. RESULTS #1 INTRODUCTIONPROBLEMREALIZATIONRESULTS

17. RESULTS #2 INTRODUCTIONPROBLEMREALIZATIONRESULTS

18. RESULTS #3 INTRODUCTIONPROBLEMREALIZATIONRESULTS

19. RESULTS #4 INTRODUCTIONPROBLEMREALIZATIONRESULTS

20. DISTORTED IMAGE POINTS Test object D ref =30mD ref =40mD ref =50mD ref =60m 14,33,97,05,6 24,33,87,05,6 32,93,84,85,6 40 (reference point) 53,16,48,16,4 6 8,714,217,6 711,716,228,931,5 DISTORTION-FREE IMAGE POINTS Test object D ref =30mD ref =40mD ref =50mD ref =60m 1 1,50,51,90,5 2 3,31,84,41,6 3 2,22,53,03,1 4 0 (reference point) 5 0,51,92,80,5 6 1,11,71,11,2 7 5,38,02,33,4 RESULTS (absolute distance error [%]) INTRODUCTIONPROBLEMREALIZATIONRESULTS