A
Theory of Multi-Layer Flat Refractive Geometry
Amit Agrawal, Srikumar Ramalingam, Yuichi Taguchi and Visesh Chari
Mitsubishi Electric Research Labs (MERL) and
INRIA
CVPR 2012
 Single photo of
three checkerboards looking through a water tank.
From this single photo, we show how to estimate
(a) unknown orientation of the tank with respect
to the camera, (b) thickness of tank along viewing
direction, (c) Refractive index of liquid inside
the tank and (d) Pose (orientation) of
checkerboards.
|
|
 |
Summary
Calibration and 3D reconstruction for flat refractive imaging systems. Our paper shows how to do calibration using a single photo (e.g. planar checkerboard). Calibration involves recovering (a) unknown orientation of layers, (b) unknown pose of checkerboard, (c) unknown layer distances, and (d) unknown refractive indices of each layer, from a single photo. The only constraint is that all layers are flat (planar refraction) and parallel to each other.
We also derive the forward projection equations for single layer and two layer scenarios. 3D reconstruction can be done by replacing the perspective projection equations with analytical forward projection equations in any bundle-adjustment algorithm.
Abstract
Flat refractive geometry corresponds to a perspective camera
looking through single/multiple parallel flat refractive
mediums. We show that the underlying geometry of rays
corresponds to an axial camera. This realization, while
missing from previous works, leads us to develop a general
theory of calibrating such systems using 2D-3D
correspondences. The pose of 3D points is assumed to be
unknown and is also recovered. Calibration can be done even
using a single image of a plane.Calibration and 3D reconstruction for flat refractive imaging systems. Our paper shows how to do calibration using a single photo (e.g. planar checkerboard). Calibration involves recovering (a) unknown orientation of layers, (b) unknown pose of checkerboard, (c) unknown layer distances, and (d) unknown refractive indices of each layer, from a single photo. The only constraint is that all layers are flat (planar refraction) and parallel to each other.
We also derive the forward projection equations for single layer and two layer scenarios. 3D reconstruction can be done by replacing the perspective projection equations with analytical forward projection equations in any bundle-adjustment algorithm.
Abstract
We show that the unknown orientation of the refracting layers corresponds to the underlying axis, and can be obtained independently of the number of layers, their distances from the camera and their refractive indices. Interestingly, the axis estimation can be mapped to the classical essential matrix computation and 5-point algorithm can be used. After computing the axis, the thicknesses of layers can be obtained linearly when refractive indices are known, and we derive analytical solutions when they are unknown. We also derive the analytical forward projection (AFP) equations to compute the projection of a 3D point via multiple flat refractions, which allows non-linear refinement by minimizing the reprojection error. For two refractions, AFP is either 4th or 12th degree equation depending on the refractive indices. We analyze ambiguities due to small field of view, stability under noise, and show how a two layer system can be well approximated as a single layer system. Real experiments using a water tank validate our theory.
pdf, Supplementary pdf,
Talk Slides (pdf)