Rain and Snow Removal via Spatio-Temporal Frequency Analysis |
 
|
| Capturing good videos outdoors can be challenging due to harsh lighting, unpredictable scene changes, and most relevant to this work, dynamic weather. Particulate weather, such as rain and snow, creates complex flickering effects that are irritating to people and confusing to vision algorithms. Although each raindrop or snowflake only affects a small number of pixels, collections of them have predictable global spatio-temporal properties. In this paper, we formulate a model of these global dynamic weather frequencies. To begin, we derive a physical model of raindrops and snowflakes that is used to determine the general shape and brightness of a single streak. This streak model is combined with the statistical properties of rain and snow, to determine how they effect the spatio-temporal frequencies of an image sequence. Once detected, these frequencies can then be suppressed. At a small scale, many things appear the same as rain and snow, but by treating them as global phenomena, we achieve better performance an with just a local analysis. We show the effectiveness of removal on a variety of complex video sequences. |
Publications
"Analysis of Rain and Snow in Frequency Space,"
P. Barnum, S. G. Narasimhan, and T. Kanade
International Journal of Computer Vision (IJCV), December 2008
[PDF]
"Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Videos,"
P. Barnum, T. Kanade, and S. G. Narasimhan
Workshop on Photometric Analysis For Computer Vision (PACV), in conjunction with ICCV, October 2007.
[PDF] [Slides]
|
Videos
 |
|
All videos plus a narrated introduction
(94mb, Divx compressed) |
| |
|
A mailbox:
There are objects at various ranges, between approximately 1 to 30 meters from the camera. The writing on the mailbox looks similar to snow. Most of the snow can be removed, although there are some errors on the edges of the mailbox and on the bushes.
(16mb, Divx compressed) |
 |
|
A windowed building:
The rain is not very heavy, but this
sequence is difficult, because there are a large number of straight,
bright lines from the window frames and the branches.
(18mb, Divx compressed) |
 |
|
A man sitting:
This scene is from the movie Forrest Gump. The rain streaks are fairly large, as is common in films. The rain can be completely removed, although the letters and windows in the upper portion of the images are misclassified.
(16mb, Divx compressed) |
 |
|
Pedestrians in the snow:
This is a very difficult sequence with a lot of high frequency textures, very heavy snow, and multiple moving objects. Much of the snow is removed, but the edges of the umbrella and their legs are misclassified.
(10mb, Divx compressed) |
|