|
|
Capturing good videos outdoors can be challenging due to harsh lighting,
unpredictable scene changes, and most relevant to this work, dynamic weather.
Particulate weather, such as rain and snow, creates complex flickering effects
that are irritating to people and confusing to vision algorithms. Although
each raindrop or snowflake only affects a small number of pixels, collections
of them have predictable global spatio-temporal properties. In this paper, we
formulate a model of these global dynamic weather frequencies. To begin, we
derive a physical model of raindrops and snowflakes that is used to determine
the general shape and brightness of a single streak. This streak model is
combined with the statistical properties of rain and snow, to determine how
they effect the spatio-temporal frequencies of an image sequence. Once detected,
these frequencies can then be suppressed. At a small scale, many things appear
the same as rain and snow, but by treating them as global phenomena, we achieve
better performance an with just a local analysis. We show the effectiveness
of removal on a variety of complex video sequences.
|
Publications
"Analysis of Rain and Snow in Frequency Space"
P. Barnum, S. G. Narasimhan, and T. Kanade
International Journal of Computer Vision (IJCV),
December 2008.
[PDF]
"Spatio-Temporal Frequency Analysis for Removing
Rain and Snow from Videos"
P. Barnum, T. Kanade, and S. G. Narasimhan,
IEEE Workshop on Photometric Analysis For Computer Vision (PACV), in conjunction with ICCV,
October 2007.
[PDF]
[Slides]
|
Videos
(Video Result Playlist)
|
All videos plus a narrated
introduction:
(94mb, Divx compressed)
|
|
|
A mailbox:
There are objects at various ranges, between approximately 1 to 30 meters from
the camera. The writing on the mailbox looks similar to snow. Most of the snow
can be removed, although there are some errors on the edges of the mailbox and
on the bushes. (16mb, Divx compressed)
|
|
A windowed building:
The rain is not very heavy, but this
sequence is difficult, because there are a large number of straight,
bright lines from the window frames and the branches. (18mb, Divx compressed)
|
|
A man sitting:
This scene is from the movie Forrest Gump. The rain streaks are fairly large,
as is common in films. The rain can be completely removed, although the letters
and windows in the upper portion of the images are misclassified. (16mb, Divx compressed)
|
|
Pedestrians in the snow:
This is a very difficult sequence with a lot of high frequency textures,
very heavy snow, and multiple moving objects. Much of the snow is removed,
but the edges of the umbrella and their legs are misclassified. (10mb, Divx compressed)
|
|
|