1. Finite Element Surface-Based Stereo 3D Reconstruction
Edgar J. Lobaton Lopez, UC Berkeley Tracy Xiaoxiao Wang, UC Berkeley
The Project
Abstract
The goal is the design of an algorithm for 3D reconstruction of scenes based on stereo pair images. Here we aim for a surface based approach using finite elements and a triangular wavelets for real-time computations and compactness of the representation. Compactness is important for future transmission of the data over a network.
Why A Surface Representation?
A surface representation is compact compared to a dense cloud of points in 3D.
A surface defining connectivity between vertices in a triangulation gives structure for the design of operators and data manipulation.
Triangular Wavelets
Finite Element Process
Using Finite Elements, we can define operators (filters) in the same way as it is done for regular grids.
Operators in our triangulated domain can include scale information for faster convergence without losing much detail.
In order to compute 3D structure, we look for the disparity between two images, which can be interpreted as the horizontal shift that maps one image to another.
Steps
We start with a stereo pair of images (left and right)
A set of nodes from the triangular representation is selected for correlation computation (NCC).
Wholes are filled-in using the information that is available.
Use the previous result as an initial condition for an iterative process in which we minimize an energy functional. This energy has an image-driven smoothing term and an energy term from the correlation values.
Data Size
The size of the data shown below is before any compression is applied to it:
Original Image (352 x 288 pixels)
100 KB
Representation and Disparity Map
80 KB
The later includes all the information required for 3D reconstruction.
Benefits
No gradient computations are required, only integrations, which make it robust to noise.
Most of the steps in the algorithm are designed for parallel implementation.
Multiscale treatment comes naturally.
Compact representation which gives connectivity information and allows the design of operators for the data.
Future Work
Add 3 channels of color information for more effective image-driven smoothing operators
Consider trinocular setup for more robust computation of correlation
Improve pre and post processing on representation
Parallel computing implementation
Creation of a compression standard for transmission of frame sequences over a network.
Hierarchical Structure
Selecting Representation
Selection based on a threshold of the normalized error per triangle
We obtain a Multiscale Representation
Mean and Variance values propagate recursively in parallel through the tree
Sample Reconstructed Views
The original view and a reconstructed view are shown below.
The result is a surface represented using linear finite elements. Further smoothing and interpolation is still possible for better rendering of images.
April 27, 2006