1. Reconstructing depth from 2D images
Author: Alexei Masterov
Professor: Tony Jebara

2. Organization:
Goal
Motivation
Roadmap
Preliminary Results
References

3. Goal: Learn to reconstruct depth from 2D data.
Use SVM regression to learn z = f (neighborhood of z).

4. Motivation: Humans are able to reconstruct distance even with 1 eye
Still images contain pictorial cues:
a. Interposition
b. Relative height
c. Relative size
d. Linear perspective
e. Texture Gradient
f. Shadow
g. Blurring
I am hoping to learn those cues from the input dataset.

5. Roadmap:
Acquire the dataset: Arrange 3D models into scenes using AliasWavefront Maya.
Preprocess the dataset: produce the pairs of 2D rendered images and Z-Depth maps of those scenes
Program the converter to prepare the input data for use with MySvm
Learn the regression using MySvm using different parameters
Try different preprocessing techniques such as 2D fft in log polar coordinates, and edge detection.
Acquire real world data of natural scenes using laser scanner and high resolution photo camera, and try the derived algorithm on it.
Package the learned SVM into a program, so that it can be used to reconstruct depth from photographs.

6. MySVM (by Stefan Rüping)
Kernels:
a. dot
b. polynomial
c. RBF
d. two layered neural net tanh(a x*y+b)
e. (RBF) anova kernel

7. Preliminary Results (1):

8. Preliminary Results (2):
Edge Image:
61 x 61 Squares:

9. Preliminary Results (3):

10. References:
“Discriminative Random Fields: A Discriminative Framework for Contextual Interaction in Classification”. Sanjiv Kumar, Martial Hebert.
“Mixtures of Eigenfeatures for Real-Time Structure from Texture” T. Jebara, K. Russell and A. Pentland.