1. Silhouette Intersection
System Overview
Silhouette Intersection
View Cube
3 meters on a side
Images
Silhouettes
Calibration
Our approach to calibration is to maximize the consistency between image or alternatively the reconstructed volume. An outline of the algorithm:
Begin with rough camera estimates.
Adjust cameras
Intersect cones generated from the cameras and associated silhouettes.
Project the 3d model back onto the silhouettes.
If there is greater consistency between silhouettes and reconstruction, update cameras.
Repeat until reconstruction matches silhouette.
Miscalibrated
Cameras
Calibrated
Our system can dynamically
recalibrate the cameras
in the view cube.
Segmentation
Original
Image
Difference
Segmentation
Our system can uses a graph cut
formalism to find the optimal
segmentation very rapidly.
Real-Time
The direction of run-length encoding in each image corresponds to vertical lines in the world. We call the lines in the world worldlines, and the lines in the image, corresponding to lines in the world, imagelines.
From left to right: (1) A camera, with some imagelines, looking at a set of worldlines; (2) the same camera, oriented to show a worldline corresponding to the rightmost imageline; (3) the same camera, oriented to show a worldline corresponding to the next imageline.
Two views of the image plane with imagelines. Left: Camera is horizontal. Right: Camera looking slightly down.
Top-left: three cameras' views of a person, with a worldline and 3d model. Top-right: the three cameras with just the worldline. Middle-left: the worldline with the run from the right camera highlighted. Middle-right: the worldline with the run from the center camera highlighted. Bottom-left: the worldline with the run from the left camera highlighted. Bottom-right: the three runs along a worldline, the intersection of those runs (rightmost line) is the part that would remain after reconstruction.
Our system operates in real-time. Each camera computes a segmentation and then compresses the silhouette by run-length encoding.
The compression depends on the orientation of the camera in the world. As a result 3D reconstruction of the volume can take place without decompressing the silhouettes.
Voxel Occupancy
Left: Eight of the 16 images captured using within the acquisition volume. Right: Silhouettes computed from these images. Notice that there has been no attempt to artificially simplify the image processing necessary to compute silhouettes. The lighting and background is complex and uncontrolled. The subject is also wearing natural clothing which often matches the color of the background.
Three reconstructions. Top: reconstruction using our method. Middle: reconstruction using silhouette intersection (silhouettes were computed using image differencing, and morphological operations to remove noise). Bottom: robust silhouette intersection, where a voxel is consider occupied if 3 out of 4 cameras agree that it is within the silhouette.
with graph cuts
Our system operates uses the graph-cut formalism to find the optimal 3D volume which explains the resulting silhouettes.
The graph-cut system incorporates both a spatial prior (on shapes) and a model of noise in the images.