|
|
A video sequence of an underwater scene taken from above the water
surface suffers from severe distortions due to water
fluctuations. In this paper, we simultaneously estimate the shape of
the water surface and recover the planar underwater scene without
using any calibration patterns, image priors, multiple viewpoints or
active illumination. The key idea is to build a compact spatial
distortion model of the water surface using the wave equation. Based
on this model, we present a novel tracking technique that is
designed specifically for water surfaces and addresses two unique
challenges --- the absence of an object model or template and the
presence of complex appearance changes in the scene due to water
fluctuation. We show the effectiveness of our approach on both
simulated and real scenes, with text and texture.
|
Publications
"Seeing through Water: Image Restoration using Model-based Tracking"
Yuandong Tian and Srinivasa G. Narasimhan,
Proc. of IEEE International Conference of Computer Vision (ICCV),
Oct, 2009.
[PDF]
"The Relationship between water depth and distortion,"
Yuandong Tian and Srinivasa G. Narasimhan,
Technical Report CMU RI,
Aug, 2009.
[PDF]
|
Code and Data
Codes and Data can be
downloaded here.
Additional water-distorted test images can be downloaded
here:
Middle Font, Small Font, Tiny Font.
|
Poster and Presentation
Poster
Presentation and Additional Results
|
Pictures
|
Image distorted by wavy water surface:
Due to light refraction at the fluctuating water surface, a
stationary scene immersed in water appears distorted in the
camera. By Snell's law, the distortion function w(x, t) at
position x and time t is related to the height h(x,t) of the
water surface.
|
|
Typical image deformation by water
distortion:
In each row, the first patch shows
the undistorted image; the rest show the distorted ones. One
can perceive severe distortions and topological changes in the
examples. Note that all the images are synthesized by the wave
equation.
|
|
The water bases B:
The extracted water
bases using PCA on the synthesized data. The bases capture the
local wave-like structures of water dynamics and reduce the
number of distortion parameters in favor of the upcoming
tracking procedure.
|
|
The reconstructed distortion using water
bases:
The subspace spanned by the water bases
has a dimension of ten. However, it captures the basic
characteristics of distortion.
|
|
Two difficulties in water tracking:
1)
There is no template; 2) Unlike affine transform that forms a
group, the water distortion is not one-to-one. As a result, a
novel tracking technique is proposed to handle these problems.
|
|
Results
(Video Result Playlist)
|
Tracking results on synthesized
images:
Water tracking validation using simulated
data. The distortion estimations, although not perfect, are reasonable
given the complex appearance changes. Details are in the paper.
|
|
Image restoration results:
The proposed
method is tested on four different sizes of text fonts, as
well as check board and brick textures. The first column shows
a sample frame from the input video, which is severely
distorted. Then the second and third column shows the result
by pixel-wise mean/median, which is less distorted but
severely blurred. Finally our results are shown on the last
two columns using two patch partitions (detailed in the
paper). Notice our method alleviates the distortion but still
retains image details.
|
|
Image synthesis using estimated
distortion:
Given the scene with still water
surface (the first column), a distorted image can be
synthesized using estimated distortion (the second column),
which is similar to the corresponding frame of the input video
sequence (the third column).
|
|
Video stabilization results
In addition,
the estimated distortion can be used to stabilize a distorted
video by applying the tracking procedure to a temporal sliding
window.
|
|
|
