VT-Intrinsic: Physics-based Decomposition of Reflectance and Shading from
a Single Visible-Thermal Image Pair

Zeqing Leo Yuan, Mani Ramanagopal, Aswin C. Sankaranarayanan, Srinivasa G. Narasimhan
Carnegie Mellon University
Paper arXiv Video Code BibTeX

"Absence, the highest form of presence."

— James Joyce

VT-Intrinsic Method Overview

Decomposing scene appearance into reflectance (material albedo) and shading (incident illumination) is a fundamental challenge in computer vision. What is absent in the visible image—absorbed light—manifests as heat in the thermal spectrum and presents complementary cues for intrinsic decomposition. Our method exploits this energy conservation principle through ordinality theory to achieve decomposition from a single visible-thermal image pair without learned priors.

Qualitative Results

Ordinality Theory

How light and heat transport reveals reflectance and shading cues?

theory_0

Given the absorbed heat (𝑆), intrinsic image decomposition becomes well-posed [JoLHT-Video].
However, measuring the absorbed heat from light requires capturing thermal transients with a calibrated video under controlled illumination, which is expensive for general application.


What can we do with a single thermal image?

theory_1

The light and heat transport equations reveal albedo/shading ordinalities of arbitrary point pairs.


How these ordinalities bridge a thermal image and absorbed heat?

theory_2

Hence, albedo/shading ordinalities are preserved when substituting absorbed heat with thermal image intensity. This enables extracting rich ordinality information from a single thermal image, eliminating the need for transient thermal video under controlled illumination. (Details in paper.)


Interactive Ordinality Demo

Click two points on the image to explore albedo/shading ordinalities, derived purely from visible-thermal ordinalities.

Visible Image

Visible Image

Thermal Image

Thermal Image

Optimization Method

How these ordinalities enable intrinsic image decomposition?

pipeline

Our theory enables albedo/shading ordinality queries between arbitrary two points in the image. We densely sample these point-pair ordinalities and thus classify edges as albedo- or shading-dominant, then use these constraints as supervision to optimize randomly initialized CNNs parameterizing albedo and shading (Double-DIP, offering low-level image prior underlying in CNN architecture).

Abstract

Decomposing an image into its underlying photometric factors—surface reflectance and shading—is a long-standing challenge due to the lack of extensive ground-truth data for real-world scenes. We introduce a novel physics-based approach for intrinsic image decomposition using a pair of visible and thermal images. We leverage the principle that light not reflected from an opaque surface is absorbed and detected as heat by a thermal camera. This allows us to relate the ordinalities (or relative magnitudes) between visible and thermal image intensities to the ordinalities of shading and reflectance, which enables a dense self-supervision of an optimizing neural network to recover shading and reflectance. We perform quantitative evaluations with known reflectance and shading under natural and artificial lighting, and qualitative experiments across diverse scenes. The results demonstrate superior performance over both physics-based and recent learning-based methods, providing a path toward scalable real-world data curation with supervision.

Citation

@article{yuan2025vt-intrinsic,
          title={Physics-Based Decomposition of Reflectance and Shading using a Single Visible-Thermal Image Pair}, 
          author={Zeqing Leo Yuan and Mani Ramanagopal and Aswin C. Sankaranarayanan and Srinivasa G. Narasimhan},
          year={2025},
          journal = {arXiv}, 
    }