1. From Photohulls to Photoflux Optimization
BMVC 2006
From Photohulls to Photoflux Optimization
Yuri Boykov, Victor Lempitsky
University of Western Ontario
Moscow State University
Contribution: novel class of geometrically motivated functionals for volumetric multiview reconstruction: “photoflux”
Benefits:
captures properties of photohull [Kutolakos and Seits, 2002]
can be combined with regularization (e.g. based on photoconsistency)
unifies two major approaches to multiview reconstruction:
space carving and deformable models
e.g. [Kutolakos&Seits, 02] e.g. [Faugeras&Keriven, 98]
addresses “shrinking” and “over-smoothing” of regularization methods
data-driven “ballooning” recovers thin protrusions or fine shape details
can be optimized with graph cuts or variational methods, e.g. level-sets
Vector field of and its divergence
Example with 4 cameras
[Kutolakos and Seitz, 02]
Photoflux
Photohull
Flux-based
methods
Regularization
Combining Photoflux and Regularization
for Multiview
Reconstruction
Greedy methods
[this work]
[Kutolakos & Seits, 2000]
Snakes
Kass et al., 1988
Level-sets
Malladi et al., 1994
Graph cuts
Boykov and Jolly, 2001
Level-sets
Vasilevsky and Sidiqqi, 2002
Kimmel et al., 2003
Graph cuts
Kolmogorov and Boykov, 2005
Thresholding
Region growing
image
segmentation
The largest photoconsistent surface S computed by “carving” out photo-inconsistent voxels from a volume.
Mash-based
Esteban and Schmitt, 2004
Level-sets
Faugeras and Keriven, 1998
Graph cuts
Vogiatzis et al., 2005
Voxel coloring
Seitz and Dyer, 1997
Space carving
Kutulakos and Seitz, 2002
multi-view
reconstruction
(volumetric approach)
This work
Local non-binary photoconsistency P(X|S) (non-deterministic decision at each voxel X)
Local binary photoconsistency P(X|S)
(deterministic decision at each voxel X)
photoflux
regularization
Regularization +
uniform
Ballooning
(e.g. Faugeras&Keriven,
Vogiatis et.al. 2005)
predefined threshold
Sum of color-discrepancies between cameras observing voxel X given its visibility defined by S
Photohull
(Space Carving)
Kutolakos&Seits, 2002
This work
Regularization +
“intelligent
ballooning”
(photoflux)
1 out of 17 input photos X’ X
photohull 3 2 1 X 3 2 1
Low noise, but some details smoothed out
Details are fine, but noisy
for all points X on photohull
Gradient of photoconsistency is large
for all points X on photohull !!!
Low noise and no shrinking (details are preserved)
for all points X’ right outside photohull
More results are in the paper and in the tech. report