Abstract
Thermal cameras measure the temperature of objects based on radiation emitted in the infrared spectrum. In this work, we propose a novel shape recovery approach that exploits the properties of heat transport, specifically heat conduction, induced on objects when illuminated using simple light bulbs.
Although heat transport occurs in the entirety of an object's volume, we show a surface approximation that enables shape recovery and empirically analyze its validity for objects with varying thicknesses. We develop an algorithm that solves a linear system of equations to estimate the intrinsic shape Laplacian from thermal videos along with several properties including heat capacity, convection coefficient, and absorbed heat flux under uncalibrated lighting of arbitrary shapes. Further, we propose a novel shape from Laplacian objective that aims to resolve the inherent shape ambiguities by drawing insights from absorbed heat flux images using two unknown lights sources. Finally, we devise a coarse-to-fine refinement strategy that faithfully recovers both low- and high-frequency shape details.
We validate our method by showing accurate reconstructions, to within an error of 1-2mm (object size ≤ 13.5cm), in both simulations and from noisy thermal videos of real-world objects with complex shapes and material properties including those that are transparent and translucent to visible light. We believe leveraging heat transport as a novel cue for vision can enable new imaging capabilities.
Image Formation in Thermal
In the visible spectrum, an object's appearance is determined solely by light transport effects, which can be characterized through BSDF of the material. However, in the thermal spectrum, accurately modeling thermal appearance requires considering not only light transport effects like reflections but also heat transport within and around the object.
Modeling Heat Transport
Heat transport within an object and its surroundings is described by the transient heat equation, which incorporates all three modes of heat transfer: conduction, convection, and radiation. Here, conduction within an object is modeled through a shape-dependent Laplacian term. Retrieving this Laplacian from thermal videos is the key to our approach.
Solution to the Linearized Heat Equation
The resulting linearized discrete heat equation is non-linear to shape but linear with respect to the unknowns of the Laplace Operator. Solving the linearized form across timeframes of a thermal video, yields scene, material, and geometric properties such as scaled heat capacity, convection coefficient, absorbed heat flux, and the shape-dependent Laplace Operator.
Shape from Laplacian
Estimating shape from Laplacian is challenging due to its non-convex nature and multiple global minima. We draw insights from the absorbed heat flux images using two uncalibrated light sources to resolve the inherent shape ambiguities. Our physics-based optimization objective adjusts mesh vertices along the camera rays to retrieve the shape of the object.
Qualtitative Results
We show results on a variety of objects with different shapes and materials, including plastic bottles, a 3D-printed bunny, an aluminum soda can, and acrylic items like a ball, bear, and pineapple. Imaged with a thermal camera under uncalibrated lighting, the captured heat flow visualizes iso-contour lines of temperature which are used to estimate the object's shape.
(Hover over the heat-flow videos to reveal the imaged object. The shown reconstructions on the right are interactive.)
Captured Heat-flow
Coarse-to-fine Optimization
Reconstructed Shape
The results below show the versatility of our approach in reconstructing materials with varying visible reflectances (black, varying albedo, transparent, and translucent). The visualized iso-contour temperature lines in the second column indicate observable heat flow in the thermal spectrum, even for varying visible reflectance properties. Notably, the reflectance properties of many visibly transparent objects appear opaque in thermal enabling heat flow observations in such transparent or translucent objects.
Object Imaged
Captured Heat-flow
Reconstructed Shape
Related Projects
-
A Theory of Joint Light and Heat Transport for Lambertian Scenes, CVPR 2024
Mani Ramanagopal, Sriram Narayanan, Aswin C. Sankaranarayanan, Srinivasa G. Narasimhan -
Projecting Trackable Thermal Patterns for Dynamic Computer Vision, CVPR 2024
Mark Sheinin, Aswin C. Sankaranarayanan, Srinivasa G. Narasimhan
BibTeX
@article{narayanan2024shape,
author = {Narayanan, Sriram and Ramanagoal, Mani and Sheinin, Mark and Sankaranarayanan, Aswin C. and Narasimhan, Srinivasa G.},
title = {Shape from Heat Conduction},
journal = {European Conference on Computer Vision (ECCV)},
year = {2024},
}