Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport
Matthew O'Toole, Felix Heide, Lei Xiao, Matthias B. Hullin, Wolfgang Heidrich, and Kiriakos N. Kutulakos. ACM SIGGRAPH, 2014.
We analyze light propagation in an unknown scene using projectors and cameras that operate at transient timescales. In this new photography regime, the projector emits a spatio-temporal 3D signal and the camera receives a transformed version of it, determined by the set of all light transport paths through the scene and the time delays they induce. The underlying 3D-to-3D transformation encodes scene geometry and global transport in great detail, but individual transport components (e.g., direct reflections, inter-reflections, caustics, etc.) are coupled nontrivially in both space and time. To overcome this complexity, we observe that transient light transport is always separable in the temporal frequency domain. This makes it possible to analyze transient transport one temporal frequency at a time by trivially adapting techniques from conventional projector-to-camera transport. We use this idea in a prototype that offers three never-seen-before abilities: (1) acquiring time-of-flight depth images that are robust to general indirect transport, such as inter-reflections and caustics; (2) distinguishing between direct views of objects and their mirror reflection; and (3) using a photonic mixer device to capture sharp, evolving wavefronts of "light-in-flight".
14.siggraph.temporalprobing.pdf (Paper — 9.9 MB)
14.siggraph.temporalprobing.supp.pdf (Supplemental document — 0.3 MB)
Time-of-flight geometry (Figure 1d)
We use a photonic mixer device (PMD) coupled with a 2D projector to acquire depth maps for scenes with complex indirect transport. We recovered this time-of-flight geometry of a large kitchen bowl in a way that is not affected by strong diffuse interreflections.
Light-in-flight video
Our prototype system produced this light-in-flight video of a scene's temporal response to a pulse of light. This scene contains two large kitchen bowls, with one-bounce light paths producing sharp temporal responses, and multi-bounce paths keeping the bowls lit for a prolonged time period. The temporal resolution of this video is 0.1 ns.
Direct vs. retro-reflective points (Figure 10d)
We distinguish between direct views of objects (highlighted in red) and their mirror reflections (hightlighted in blue) by capturing and comparing the lengths of direct, retro-reflective, and caustic light paths.
Temporal probing prototype (Figure 5)
Our prototype consists of a PMD camera (left), a modified DLP projector (middle), and a modulated laser light source (right).
14.siggraph.temporalprobing.mov (Supplemental video — 52.9 MB)