Nearly half of all accidents occur at night despite 25% less traffic on the road due to limited visibility. Visibility is further reduced during rain and snow storms creating extremely dangerous driving conditions. A fourteen-year average of accident statistics from the National Highway Safety Transportation Administration indicates that 900,000 accidents, 400,000 injuries, and 4,000 fatalities occur in rain and snow. We aim to assist drivers during evening and night storms by improving driver visibility and reducing stress by reducing the visibility of illuminated precipitation particles (e.g., raindrops, snowflakes, etc.).

Autonomous vehicles are still in early stages of development. There are still many technological challenges to solve. Despite many miles of road testing, current vehicles, e.g., Google Self-Driving Car, require extensive preparations of the vehicles route beforehand [source] and have not been extensively tested in poor weather [source]. Analysts estimate that fully autonomous vehicles will not be common for another 30 to 50 years [source], so our programmable headlight will benefit all drivers in the meantime.

Even if vehicles were fully autonomus today, they could benefit from our headlight design. For example, image quality can be enhanced, e.g., free of weather artifacts, cameras (other sensors) and drivers (passengers) can not be glared, vehicles can communicate with each other through visible light communication, the person's attention can be brought to objects that suddenly appear on the road environment for intervention, etc.

In previous work, we referred to our headlight as a smart headlight. The overall design between the programmable headlight and the smart headlight are similar. We chose to give the headlight a more descriptive name to emphasize that it has a single hardware design but can perform many tasks based on how it is programmed.

The current system is about 10 times faster than our old one in 2012, can do many more tasks (anti-glare, object highlighting, road visibility and bad weather visibility improvement), and has been demonstrated for the first time on a vehicle moving at high speeds in regular traffic. Previous system was proof-of-concept demonstrated in the lab. The detailed timeline of the project is on our website.

Standard vehicle headlights illuminate everything in their pathway, which is necessary for nighttime driving. Recently, headlights that adapt to the environment have made their way to the road. However, they tend to be one-off solutions, i.e., they can perform only one task. The programmable headlight has one hardware configuration that can be programmed to perform many tasks. The prototype that we built has unprecedented spatiotemporal resolution permitting reaction to objects of all shapes and sizes.

Many cars have adaptive front lighting technologies and the term Matrix LED is used to describe it often. These adaptive lighting methods also can achieve anti-glare but at the cost of suppressing significant light from their high beams (any system that turns off its high-beams completely also achieves this effect). As compared to a 4, 12, 24, 40 or even 100 LED beams that are in the market today, our system consists of an order of one million beams that can be controlled very fast (within a millisecond or two). The ultra short latency of our system means that we are able to achieve high light throughput from the high beam while not glaring any number of motorists, bicyclists, etc on the road without having to know how far they are from the smart headlight. In addition to anti-glare, our system can be used for numerous other applications such as poor weather visibility enhancement, road visibility enhancement and early object warning. The beam control resolution, precision and speed in our system yields unprecedented flexibility of front lighting. See also editorial and in-depth article "What's beyond the Matrix?" (for subscribers) in DrivingVisionNews.com.

Our lab has been researching visibility enhancement in poor weather conditions for more than a decade. We have developed tools for removing fog, haze, mist, rain, snow, dust and murky waters from images and videos. But, we thought, wouldn't it be great if the driver sees better with her/his eyes? That thought started this project a couple of years ago.

The idea behind the programmable headlight is to integrate a camera, processor, and spatial light modulator with a light source. The camera captures images illuminated by the light source and the processor detects objects of interest, then controls the spatial light modulator to react appropriately.

We previously conducted computer simulations to show that the idea is feasible and built our third prototype system in the laboratory. The previous prototype was exclusively stationary at 120 Hz and was tested with a rain generator. The results were a dramatic reduction in visibility of illuminated rain drops. Since then, we have built a new prototype with a latency 10 times better than before. We have tested the prototype on a vehicle and demonstrated multiple applications.

We have demonstrated anti-glare headlights, improved driver visibility during snowstorms, increased contrast of lanes, markings, and sidewalks, and early visual warning of obstacles.

The current prototype consists of a camera with a CameraLink interface (Basler), frame grabber (BitFlow), custom-built spatial light modulator (WinTech and InFocus), and desktop computer with Intel architecture (Intel i7 quad core processor).

The beamsplitter (50/50) permits optically co-locating the camera and projector to eliminate the need for stereo reconstruction thus reducing computations and increasing system speed. The downside is that 50% of light from the projector is lost making drop detection more difficult due to decreased contrast.

This is a perfectly plausible solution. For example, see our previous work of removing rain, snow, and fog from videos. Similarly, other systems based on thermal imaging exist to help see through rain and fog. However, the resulting videos need to be displayed somewhere in the car. This can be distracting for the driver; he or she needs to look away and refocus on the small display in a difficult situation. The proposed system enhances the visibility for the driver's naked eye and also helps the drivers in oncoming vehicles.

Because the prototype was built with off-the-shelf components, data transfer speed is slower than if the components were more closely integrated (e.g., in an embedded system). The refresh rate of the projector and camera sensitivity for short exposure times also have an effect on system performance.

In order for the system to be used on an automobile it needs to be made faster and more compact. It also needs to be engineered to be durable for temperature, moisture, humidity, vibrations, bumps, etc.

The current system runs at over 1,000 Hz. The camera exposures varies based on application being performed and the system has a total latency of 1-1.5 ms; 0.5 ms for transferring data from camera to computer, 0.15 ms for analysis, and generation of the projection image, and 0.6 ms for transferring data from the computer to spatial light modulator.

With a fixed set of system parameters for a given task, uncertainty exists in the system due to a several factors. First, the operating system's scheduler introduces interrupts when our system is running. Also, changes in the illumination pattern alter the time to process the image, e.g., it takes longer to process an image where many large objects were detected.

Currently, the system depends on camera exposure time, image resolution, processing algorithm, and number of pixels corresponding to detected objects.

The task being performed dictates the required speed of the system. For instance, computer simulations show that a system operating near 1,000 Hz, with a total system latency of 1.5 ms, and exposure time of 1 ms can achieve 96.8% accuracy with 90% light throughput during a heavy rainstorm (25 mm/hr) on a vehicle traveling 30 km/hr. However, 400 Hz with less accuracy will be a significant (>= 70%) improvement for the driver. Similar speed will needed during a heavy snowstorm. Computer simulations show that for the anti-glare task-- with a single car in a single lane and a relative speed of 240 kph-- the system needs a latency of 1.5 ms at 1,000 Hz to maintain more than 90% light throughput when the vehicles are 10 to 40 m apart.

In order for the system to be used on an automobile it needs to be made faster, more compact, and more durable. Improvements in speed and size can be made by developing specialized hardware that more closely integrates the camera and DLP projector with a processing unit (e.g., DSP). More sophisticated algorithms will be needed to maximize system speed and account for vehicle motion, wind turbulence, etc.

Since people heavily depend on headlights for driving at night, the programmable headlights must maintain adequate illumination of the environment. We refer to this as light throughput. A standard headlight has 100% light throughput. An extensive user study will be conducted to determine the necessary accuracy for reducing driver stress.

The technology currently being researched for building a faster, compact system is expected to take 3 to 4 years to complete. Commercializing it as a product will take additional years.