|
Coplanar Shadowgrams for Acquiring Visual Hulls of Intricate Objects
|
|
|
Acquiring 3D models of intricate objects (like tree branches, bicycles and insects)
is a hard problem due to severe self-occlusions, repeated thin structures and surface
discontinuities. In theory, a shape-from-silhouettes (SFS) approach can overcome
these difficulties and use many views to reconstruct visual hulls that are close
to the actual shapes. In practice, however, SFS is highly sensitive to errors in
silhouette contours and the calibration of the imaging system, and therefore not
suitable for obtaining reliable shapes with a large number of views. We present
a practical approach to SFS using a novel technique called coplanar shadowgram imaging,
that allows us to use dozens to even hundreds of views for visual hull reconstruction.
Here, a point light source is moved around an object and the shadows (silhouettes)
cast onto a single background plane are observed. We characterize this imaging system
in terms of image projection, reconstruction ambiguity, epipolar geometry, and shape
and source recovery. The coplanarity of the shadowgrams yields novel geometric properties
that are not possible in traditional multi-view camera-based imaging systems. These
properties allow us to derive a robust and automatic algorithm to recover the visual
hull of an object and the 3D positions of light source simultaneously, regardless
of the complexity of the object. We demonstrate the acquisition of several intricate
shapes with severe occlusions and thin structures, using 50 to 120 views.
|
Publications
"Coplanar Shadowgrams for Acquiring Visual Hulls of Intricate Objects,"
Shuntaro Yamazaki, Srinivasa G. Narasimhan, Simon Baker, and Takeo Kanade,
Proc. of IEEE International Conference of Computer Vision (ICCV) 2007,
Oct, 2007.
[PDF]
"On Using Shadowgrams for Visual Hull Reconstruction,"
Shuntaro Yamazaki, Srinivasa G. Narasimhan, Simon Baker, and Takeo Kanade,
Technical Report CMU-RI-TR-07-29,
Aug, 2007.
[PDF]
|
Pictures
|
|
Projecttion Model of Coplanar Shadowgram Imaging:
A point source illuminates the object and its shadow cast on a planar rear-projection
screen represents the silhouette of the object. Coplanar shadowgrams from multiple
viewpoints are obtained by translating the light source. Note that the relative
transformation between the object and the screen remains fixed across different
views.
|
 |
|
Experimental Setup:
The setup used to capture coplanar shadowgrams includes a digital camera, a single
point light source, and a rearprojection screen. The object is placed close to the
screen to cover a large field of view. Two or more spheres are used to estimate
the initial light source positions.
|
 |
|
Shadowgrams:
Example shadowgrams obtained using
the setup.
|
 |
|
Sensitivity of Visual Hull Reconstruction:
The top row shows the visual
hulls reconstructed using the light source positions estimated by our
method. As the number of silhouettes increases, the visual hull gets
closer to the actual shape. The bottom row shows the reconstructions obtained
from slightly erroneous source positions. As the number of views
increases, the error worsens significantly.
|
|
Videos
compressed using DivX codec
|
|
Shadowgram Acquisition:
This video demonstrates the acquisition of shadowgrams. The point source is moved
within a half space with respect to a rear-projection screen, while the object,
the camera and the screen all remain stationary. In order to cover the entire surface area of an intricate shape, we must capture a sufficiently large number of shadowgrams.
|
|
|
Acquired Shadowgrams:
This video shows a sequence shadowgrams acquired using our experimental setup. The
light source positions are estimated using the shadowgrams of at least two spheres.
The acquired shadowgrams can then be used to recover the visual hull of the object. |
|
|
Optimization of Light Source Positions:
This video shows an optimization of light source positions using epipolar constraints and silhouette consistency.
The ground truth and estimated source positions are presented respectively in red and yellow. |
|
|
Optimization of Silhouette Consistency:
This video illustrates the improvement of silhouette consistency.
An acquired silhouette and that generated from a reconstructed visual hull are shown respectively in green and yellow.
The match between the acquired and re-projected silhouettes increases until it becomes almost perfect. |
|
|
Robust Reconstruction of Visual Hull:
This video is the trace the progression of our optimization algorithm
for this object starting from the erroneous reconstruction.
|
|
|
Reconstructed Visual Hulls:
This video is a CG rendering of the visual hull model acquired by coplanar shadowgrams. |
|
|
Example Application to Computer Graphics:
This video is an example CG animation rendered using the 3D shape models acquired by coplanar shadowgrams. |
|
|
ICCV 2007 Supplementary Video:
This video is a compilation of the main results of this project.
|
|
Data and Downloads
Coming Soon!
|
|