Finite Element Surface-Based Stereo 3D Reconstruction Edgar J. Lobaton Lopez, UC Berkeley Tracy Xiaoxiao Wang, UC Berkeley The Project Abstract The goal is the design of an algorithm for 3D reconstruction of scenes based on stereo pair images. Here we aim for a surface based approach using finite elements and a triangular wavelets for real-time computations and compactness of the representation. Compactness is important for future transmission of the data over a network. Why A Surface Representation? A surface representation is compact compared to a dense cloud of points in 3D. A surface defining connectivity between vertices in a triangulation gives structure for the design of operators and data manipulation. Triangular Wavelets Finite Element Process Using Finite Elements, we can define operators (filters) in the same way as it is done for regular grids. Operators in our triangulated domain can include scale information for faster convergence without losing much detail. In order to compute 3D structure, we look for the disparity between two images, which can be interpreted as the horizontal shift that maps one image to another. Steps We start with a stereo pair of images (left and right) A set of nodes from the triangular representation is selected for correlation computation (NCC). Wholes are filled-in using the information that is available. Use the previous result as an initial condition for an iterative process in which we minimize an energy functional. This energy has an image-driven smoothing term and an energy term from the correlation values. Data Size The size of the data shown below is before any compression is applied to it: Original Image (352 x 288 pixels) 100 KB Representation and Disparity Map 80 KB The later includes all the information required for 3D reconstruction. Benefits No gradient computations are required, only integrations, which make it robust to noise. Most of the steps in the algorithm are designed for parallel implementation. Multiscale treatment comes naturally. Compact representation which gives connectivity information and allows the design of operators for the data. Future Work Add 3 channels of color information for more effective image-driven smoothing operators Consider trinocular setup for more robust computation of correlation Improve pre and post processing on representation Parallel computing implementation Creation of a compression standard for transmission of frame sequences over a network. Hierarchical Structure Selecting Representation Selection based on a threshold of the normalized error per triangle We obtain a Multiscale Representation Mean and Variance values propagate recursively in parallel through the tree Sample Reconstructed Views The original view and a reconstructed view are shown below. The result is a surface represented using linear finite elements. Further smoothing and interpolation is still possible for better rendering of images. April 27, 2006