ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Thomas Pock 1, Chrisopher Zach 2 and Horst Bischof 1 1 Institute for Computer Graphics and Vision, TU Graz 2 VRVis Forschungs GmbH {pock, zach,

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Outline Introduction to Variational Methods (I) Variational Stereo (II) Variational Optical Flow (III) Variational Fusion of Depth Images

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Introduction to Variational Methods Computer vision deals with inverse problems Bayesian framework is often used to estimate the unknown quantities Equivalent to minimizing appropriate energy functionals (neg. log likelihood) This directly leads to Variational Methods

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction (I) Variational Stereo Typical stereo problems: –Occlusions –Large untextured areas –Varying radiance

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Simple Correlation Methods Problems –Huge amount of outliers in untextured areas –Weak performance at depth discontinuities Observations –Depth discontinuities often coincides with image edges –Robust estimation in untextured areas

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Our Approach Joint color edge detection and robust depth estimation using a Mumford-Shah Energy Functional d z (z,d) = argmin { E }

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction E initial 28.8 %28.1 %25.1 %14.3 % E final 18.3 %3.45 %11.2 %7.52 % Rank12446 Results

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Real Dataset Synthesized view

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction (II) Variational Optical Flow Optical Flow plays a major task of biological and artificial visual systems Relates the motion of pixels between different image frames Highly ill-posed inverse problem. More unknowns than equations (Optical Flow Constraint)

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Our Approach We employ a robust energy functional consisting of –Total Variation Regularization ( Discontinuity preserving) –Robust L 1 data fidelity norm (Robust against noise, occlusions, …) We have developed an novel minimization algorithm Can be efficiently implemented on graphics hardware Image resolution 50 It x x x Performance (fps):

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Results

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Application to Stereo

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction (III) Variational Fusion of Depth Images Generation of 3D Models from multiple views Huge amount of Outliers but a lot of redundancy Two Requirements: –Robust against outliers –Natural Regularization of the 3D Model

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Our Approach Conversion of the depth images into 3D truncated signed distance fields (f i ) Assignment of varying confidence (w i ) Minimization of an energy functional consisting of –Total Variation Regularization –L 1 data fidelity term

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Results

ICG Professor Horst Cerjak, Thomas Pock Variational Methods for 3D Reconstruction Thank You !