The depth cameras that are available on the market today only work indoors. Outside, their active light sources are overwhelmed by sunlight. EpiToF uses a unique imaging technique that blocks almost all ambient light. This allows its low power source to compete with the Sun. This opens up all sorts of exciting new applications for depth cameras such as outdoor imaging and self-driving cars.

EpiToF can differentiate between single-bounce light that reflects directly off surfaces and more complex multiple-bounce light paths that involve interreflection and scattering. The presence of complex light paths confuses most conventional 3D scanners, but EpiToF is robust to these types of light paths and produces accurate measurements.

With regular ToF cameras, diffuse interreflections between walls and ceiling cause depths to be overestimated and the corner to be rounded out. With epipolar imaging, the walls appear straight and meet at a sharp right angle. The conference table appears specular at grazing angles, but the EpiToF captures only the first light bounce. In the bathroom scene, the ghosting on the wall due to reflections from the mirror is suppressed by epipolar imaging. The water fountain is particularly challenging because the direct return from its metallic surface is very weak, but the surface reflects a lot of indirect light back to the sensor.

As continuous wave ToF Cameras appear in more and more devices, they must be able to operate without interfering with each other. While devices of a given make and model can be configured to not interfere with each other by varying modulation frequency across them or synchronization, robustness against the broader ecosystem of CW-ToF sensors is desirable. EpiToF enables interference-free live 3D imaging even for devices that have the exact same modulation frequency and light source wavelength.

Regular ToF imaging has motion blur and strong artifacts at depth discontinuities when moved during frame capture. With epipolar ToF imaging, motion blur has basically no effect and a depth map with a rolling-shutter-like effect is acquired. This can be corrected with a simple image warp computed from the rotation measured with a MEMS gyroscope.

Contemporary depth imaging systems such as the Microsoft Kinect, Intel RealSense, and ToF cameras work indoors where there is little ambient light but not outdoors in Sunlight.

EpiToF's energy-efficient, real time depth sensing technology works both indoors and outside and could power the next generation of outdoor imaging systems.

Today's self-driving cars rely heavily on expensive LIDAR units that only produce sparse point clouds. The EpiToF produces denser pointclouds for high resolution object recognition and obstacle avoidance.

The Prototype

The EpiToF prototype is built from an off-the-shelf continuous-wave time-of-flight camera and a custom built projector.

• a continuous wave time-of-flight camera
• a custom modulated light sheet projector

Performance Specs:

• Resolution: 320x240
• FOV: 45°(h) x 35 °(v)
• Range in sunlight (white scene)*: 15 meters
• Range in darkness (white scene): 20+ meters
• Frame Rate: 7.5 Hz

*See paper for range details

The Light Sheet Projector

The modulated light sheet projector is comprised of:

• a 700mW, 658nm collimated laser diode
• a Powell Lens line generator
• a galvomirror

The laser is collimated and passes through a Powell Lens line generator to form a light sheet that is then deflected and steered by a galvomirror in sync with the ToF camera. The laser is amplitude modulated to produce a line modulating at a fixed frequency.

The Working Principle

The EpiToF sensor's light sheet projector illuminates the scene one scanline at a time. The ToF camera and projector are aligned so that by epipolar geometry, one projector scanline corresponds to a single row of pixels in the ToF camera. The ROI on the ToF camera is selected and synchronized so that the exposed row moves in lockstep with the active projector scanline. At each row the camera captures multiple images of the scene at different phases of the modulated wave. These images are then used to calculate the depth map from the phase shift principle.

Since the projected scanline and exposed camera row are synchronized, the exposure can be very short (~100us). This short exposure integrates very little ambient light while still collecting all the light from the projector. In addition, only light paths that follow the epipolar constraint between the projector and camera reach the camera sensor, which blocks almost all multipath light.