3D Shape and Indirect Appearance by Structured Light Transport
Matthew O'Toole, John Mather, and Kiriakos N. Kutulakos. CVPR, 2014. (Oral)
(Best Paper Honourable Mention)
We consider the problem of deliberately manipulating the direct and indirect light flowing through a time-varying, fully-general scene in order to simplify its visual analysis. Our approach rests on a crucial link between stereo geometry and light transport: while direct light always obeys the epipolar geometry of a projector-camera pair, indirect light overwhelmingly does not. We show that it is possible to turn this observation into an imaging method that analyzes light transport in real time in the optical domain, prior to acquisition. This yields three key abilities that we demonstrate in an experimental camera prototype: (1) producing a live indirect-only video stream for any scene, regardless of geometric or photometric complexity; (2) capturing images that make existing structured-light shape recovery algorithms robust to indirect transport; and (3) turning them into one-shot methods for dynamic 3D shape capture.
14.cvpr.slt.pdf (Paper — 3.4 MB)
14.cvpr.slt.supp.zip (Supplemental material — 86.0 MB)
Global video of a white candle (RAW)
Our camera prototype captured this RAW video stream of a candle's indirect appearance. The candle appears bright white because our camera captures the light bouncing around multiple times within the candle, whereas the hand and the background appears dark because our camera blocks all one-bounce light paths from ever reaching the sensor.
Global video of a wet hand (RAW)
This indirect-only video captures the change in appearance of a hand when made wet. For more examples, please refer to the supplemental .zip file.
Indirect-invariant 3D shape
Our camera makes existing structured-light shape recovery algorithms robust to extremely challenging indirect transport phenomena. Mouse-over the image to compare our indirect-invariant imaging technique to conventional imaging.
One-shot indirect-invariant 3D shape
We recovered 6 indirect-only structured-light photos from each captured video frame to produce this 3D reconstruction of a hand in motion.
14.cvpr.slt.poster.pdf (Paper — 28.2 MB)
