1. Model-Based Stereo with Occlusions
Fabiano Romeiro and Todd Zickler
TexPoint fonts used in EMF.
Read the TexPoint manual before you delete this box.: AAAAAAAAA

2. Introduction Varying illumination Varying pose Occlusions
Varying expressions

3. Introduction Past Work: Image-based For example: Eigenfaces
[Turk and Pentland, 1991]
Fisherfaces
[Belhumeur et al, 1997]

4. Introduction Past Work: Model-based 3D Morphable Models (3DMMs)
[Blanz et al, 1999; Blanz et al, 2003; Blanz et al, 2005; Smet et al, 2006]
2D AAMs
[Cootes et al, 1998; Baker et al, 2004; Mathews et al, 2004; Gross et al, 2006]
2D+3D AAMs
[Xiao et al, 2004]

5. Introduction Past Work: 3DMMs 3D Morphable Models (3DMMs) Pros
[Blanz et al, 1999; Blanz et al, 2003; Blanz et al, 2005; Smet et al, 2006]
Pros
- Self-occlusion handled by model itself
- Allows direct modeling of illumination
Cons
- Difficult and expensive fitting process

6. Introduction Past Work: Stereo 3DMMs Our work
[Fransens et al, 2005]
- Stereo based cost
- Texture model not needed
Our work
Stereo fitting with both shape and texture
→ Robust to foreign-body occlusions
→ Improved Accuracy

7. Outline 3DMM Background Joint Shape and Texture Stereo Fitting
Handling Occlusions
Conclusions

8. Background 3DMMs Vectorization of laser scans: PCA performed:
Basis for shape
- Basis for texture
[Blanz and Vetter, 1999]

9. Background 3DMMs Representation of face shape and texture:
[Blanz and Vetter, 1999]
Prior probabilities on the coefficients:

10. Stereo Match

11. Texture Match

12. Joint Shape and Texture Stereo Fitting of 3DMMs

13. Robust Stereo Fitting of 3DMMs

14. Optimization Procedure
Initial fit
- Fit Shape, Pose to minimize reprojection error on selected feature points
- Rough initial estimates of Shape and Pose
Optimization procedure
4 experiments
Stereo and texture
Stereo
Robust Stereo and texture
Robust Stereo

15. Results First 2 experiments: Stereo and Texture vs. Stereo
480 recovered shape models (60 individuals, 8 poses)
K.U. Leuven Stereo face database
[Fransens et al, 2005]

16. Results
Qualitative Results
Stereo and Texture
Stereo

17. Results
Qualitative Results
Stereo+texture
Stereo

18. Results
Quantitative Results

19. Results Under Occlusions Half-Occlusion Full Occlusion near
Full-Occlusion far

20. Results Under Occlusions Input Robust Stereo Robust S+T Robust Stereo
Shape Estimate
Occlusion Map

21. Conclusions Robust stereo fitting of 3DMMs
- Uses both stereo constraint, texture information
Increased accuracy of fit
- Ability to handle occlusions
Future Work
- More sophisticated stereo matching term
- Different feature spaces
- Break model into segments respecting occlusion boundaries