Nearly half of all accidents occur at night despite 25% less traffic on the road due to limited visibility. Visibility is further reduced during rain and snow storms creating extremely dangerous driving conditions. A fourteen-year average of accident statistics from the National Highway Safety Transportation Administration indicates that 900,000 accidents, 400,000 injuries, and 4,000 fatalities occur in rain and snow. We aim to assist drivers during evening and night storms by improving driver visibility and reducing stress by reducing the visibility of illuminated precipitation particles (e.g., raindrops, snowflakes, etc.).

Standard vehicle headlights illuminate everything in their pathway, which is necessary for nighttime driving. However, when it is raining or snowing the headlights also illuminate the falling precipitation particles (e.g., raindrops, snowflakes, etc.) causing bright distracting streaks to the driver. The smart headlight addresses this issue by quickly controlling individual light rays from the headlight to avoid particles. By drastically reducing backscattered light, the smart headlight makes precipitation particles disappear right in front of the driver.

Our lab has been researching visibility enhancement in poor weather conditions for more than a decade. We have developed tools for removing fog, haze, mist, rain, snow, dust and murky waters from images and videos. But, we thought, wouldn't it be great if the driver sees better with her/his eyes? That thought started this project a couple of years ago. Along the way, we took a slight detour and developed the first 3D display with water drops.

The idea behind the smart headlight is to integrate an imager and processing unit with a light source. The imager captures images illuminated by the light source and the processing unit detects precipitation particles in the upper part of the image, predicts the future location of the particles, then controls the light source to illuminate between particles.

We have conducted computer simulations to show that the idea is feasible, and built a prototype system in the laboratory. A stationary prototype system at 120 Hz has been tested with a rain generator. The result is a dramatic reduction in visibility of illuminated rain drops. Currently, we are investigating making the system faster and more compact in order to test on a moving platform.

The current prototype consists of a camera with gigabit ethernet interface (Point Grey, Flea3), DLP projector (Viewsonic, PJD62531), and desktop computer with Intel architecture (Intel i7 quad core processor).

The beamsplitter (50/50) permits optically co-locating the camera and projector to eliminate the need for stereo reconstruction thus reducing computations and increasing system speed. The downside is that 50% of light from the projector is lost making drop detection more difficult due to decreased contrast.

This is a perfectly plausible solution. For example, see our previous work of removing rain, snow, and fog from videos. Similarly, other systems based on thermal imaging exist to help see through rain and fog. However, the resulting videos need to be displayed somewhere in the car. This can be distracting for the driver; he or she needs to look away and refocus on the small display in a difficult situation. The proposed system enhances the visibility for the driver's naked eye and also helps the drivers in oncoming vehicles.

Yes, time-gated imaging is a very good way to block these reflections at any particular distance. The issues here though are: it may block anything else at that distance and it may not block reflections from many distances. We also did not want to gate the windshield of a car (far more expensive) or have drivers wear shuttered glasses that are synched with the headlights (more cumbersome). We are interested in having the driver see better with his/her naked eye. Same reasoning goes for methods that capture videos, post-process them to remove the rain/snow and display them in the car's display area.

Because the prototype was built with off-the-shelf components, data transfer speed is slower than if the components were more closely integrated (e.g., in an embedded system). The refresh rate of the projector and camera sensitivity for short exposure times also have an effect on system performance.

In order for the system to be used on an automobile it needs to be made faster and more compact. Improvements in speed and size can be made by developing specialized hardware that more closely integrates the camera and DLP projector with a processing unit (e.g., DSP). More sophisticated algorithms will be needed to maximize system speed and account for vehicle motion, wind turbulence, etc.

The current system runs at 120 Hz. The camera uses a 5 ms exposure time and the system has a total latency of 13 ms; 4 ms for transferring data from camera to computer, 5 ms for drop detection and prediction, and generation of the projection image, and 4 ms for transferring data from the computer to the projector.

Computer simulations show that a system operating near 1,000 Hz, with a total system latency of 1.5 ms, and exposure time of 1 ms can achieve 96.8% accuracy with 90% light throughput during a heavy rainstorm (25 mm/hr) on a vehicle traveling 30 km/hr. However, 400 Hz with less accuracy will be a significant (>= 70%) improvement for the driver.

In order for the system to be used on an automobile it needs to be made faster and more compact. Improvements in speed and size can be made by developing specialized hardware that more closely integrates the camera and DLP projector with a processing unit (e.g., DSP). More sophisticated algorithms will be needed to maximize system speed and account for vehicle motion, wind turbulence, etc.

Since people heavily depend on headlights for driving at night, the smart headlights must maintain adequate illumination of the environment. We refer to this as light throughput. A standard headlight has 100% light throughput. An extensive user study will be conducted to determine the necessary accuracy for reducing driver stress.

The technology currently being researched for building a faster, compact system is expected to take 3 to 4 years to complete. Commercializing it as a product will take additional years.