Reporter: Wade Chang Advisor: Jian-Jiun Ding 1 Depth Estimation and Focus Recovery

Outline 2 Motivation Overview Blurring model and geometric optics Blurring function Fourier optics Linear canonical transform (LCT) Depth estimation methods Binocular vision system Monocular vision system Focus recovery methods Reference

Outline 3 Motivation Overview Blurring model and geometric optics Blurring function Fourier optics Linear canonical transform (LCT) Depth estimation methods Binocular vision system Monocular vision system Focus recovery methods Reference

Motivation 4 Focus recovery is important, it can help users to know the detail of original defocused image. Depth is a important information for focus restoration.

Outline 5 Motivation Overview Blurring model and geometric optics Fourier optics Linear canonical transform (LCT) Blurring function Depth estimation methods Binocular vision system Monocular vision system Focus recovery methods Reference

Blurring Model and Geometric Optics(1) 6 What is the perfect focus distance? Why does the blurring image generate? 。。。。。。 。。。。。。 。。。。。。 。。。。。。 。。。。。。 lens sensor Position of object Effective “depth of field” interval F This area is too small for the HVS and results in an effective focused plane

Blurring Model and Geometric Optics(2) 7 Ideal and real spherical convex lens Ideal spherical convex lensReal spherical convex lens Aspherical convex lens

Blurring Model and Geometric Optics(3) 8 combination of the convex lens and the concave lens Convex lens Concave lens Incident rays F1F1 F2F2

Blurring Model and Geometric Optics(4) 9 Effective focal length of the combination of lenses F4F4 l1l1 l2l2 F3F3 F2F2 F1F1 Combination of the thin convex lenses

Outline 10 Motivation Overview Blurring model and geometric optics Blurring function Fourier optics Linear canonical transform (LCT) Depth estimation methods Binocular vision system Monocular vision system Focus recovery methods Reference

Blurring Function (1) 11 screen F D/2 F u s v Biconvex 2R : R<0 Blurring radius: R<0 Blurring radius: R>0 F D/2 F u s 2R : R>0 screen v

Blurring Function (2) 12 Blurring radius relates to depth value: Considering of diffraction, we may suppose a blurring function as: :diffusion parameter

Outline 13 Motivation Overview Blurring model and geometric optics Blurring function Fourier optics Linear canonical transform (LCT) Depth estimation methods Binocular vision system Monocular vision system Focus recovery methods Reference

Fourier Optics(1) 14 Aperture effect(Huygens-Fresnel transform) When a plane wave progress through aperture, the observed field is a diffractive wave generated from the rim of aperture

Fourier Optics(2) 15 Where the examples are through Huygens-Fresnel transform at z=1 meter, z=14 meters and z=20 meters respectively.

Outline 16 Motivation Overview Blurring model and geometric optics Blurring function Fourier optics Linear canonical transform (LCT) Depth estimation methods Binocular vision system Monocular vision system Focus recovery methods Reference

Linear Canonical Transform (1) 17 Why we use Linear canonical transform? Definition

Linear Canonical Transform (2) 18 Effects on time frequency analysis can help us realize most properties by changing those four parameters. Let us consider one of time frequency analysis-Gabor transform: After is substituted as a LCT signal, the result in a new coordinate on time and frequency is as follows.

Linear Canonical Transform (3) 19 Characteristics Function representationTransforming parameters Chirp multiplication Chirp convolution Fractional Fourier transform Fourier transform Scaling

Linear Canonical Transform (4) 20 Consider a simple optical system. The equivalent LCT parameter: UoUo UlUl U l’ UiUi s

Linear Canonical Transform (4) 21 Special case of an optical system UoUo UlUl Ul’Ul’ UiUi f : focal length

Outline 22 Motivation Overview Blurring model and geometric optics Blurring function Fourier optics Linear canonical transform (LCT) Depth estimation methods Binocular vision system Monocular vision system Focus recovery methods Reference

Binocular Vision System(1) 23

Binocular Vision System(2) 24 Binocular vision at a gazing point. Gazing point (Corresponding point) Baseline (B) B/2 Depth (u)

Outline 25 Motivation Overview Blurring model and geometric optics Blurring function Fourier optics Linear canonical transform (LCT) Depth estimation methods Binocular vision system Monocular vision system Focus recovery methods Reference

Monocular Vision System(1) 26 Method 1: Utilizing diffusion parameter to calculate depth value. Blurring radius: R>0 F D/2D/2 F u s 2R : R>0 scr een v

Monocular Vision System(2) 27 Using power spectral density to calculate depth value.

Monocular Vision System(3) 28 Method 2: Take differentiation on equation I which respect to.

Monocular Vision System(4) 29 Method 3: Using LCT blurring models UoUo UlUl U l’ UiUi s

Outline 30 Motivation Overview Fourier optics Linear canonical transform (LCT) Blurring function Depth estimation methods Binocular vision system Monocular vision system Focus recovery methods Reference

Focus Recovery Methods(1) 31 Derive MMSE filter

Focus Recovery Methods(2) 32 Derive MMSE filter 

Focus Recovery Methods(3) 33 Derive Wiener Filter:

Reference 34 [1] M. Robinson, D. Stork, “Joint Design Lens Systems and Digital Image Processing” [2] P. C. Chen, C. H. Liu, ”Digital Decoding Design for Phase Coded Imaging” [3] Y. C. Lin, “Depth Estimation and Focus Recovery”

35 Thank You for Listening