透过你的眼睛看世界

摘要

人眼的反射特性是一个被低估的关于周围世界样貌的信息源。通过成像移动人的眼睛，我们可以通过眼睛中的反射收集到摄像机直接视线之外的场景的多个视角。在本文中，我们利用包含眼睛反射的肖像图像重建摄像机视线之外的三维场景。这项任务具有挑战性，原因在于：1）准确估计眼睛姿势的困难，2）眼睛虹膜和场景反射外观的交织。我们的方法联合优化角膜姿势、描绘场景的辐射场以及观察者的眼睛虹膜纹理。我们进一步提出了对虹膜纹理图案的简单正则化先验，以提高重建质量。通过在具有不同眼睛颜色的人的合成和真实世界捕获上进行各种实验，我们展示了利用眼睛反射恢复三维场景的可行性。

English

The reflective nature of the human eye is an underappreciated source of information about what the world around us looks like. By imaging the eyes of a moving person, we can collect multiple views of a scene outside the camera's direct line of sight through the reflections in the eyes. In this paper, we reconstruct a 3D scene beyond the camera's line of sight using portrait images containing eye reflections. This task is challenging due to 1) the difficulty of accurately estimating eye poses and 2) the entangled appearance of the eye iris and the scene reflections. Our method jointly refines the cornea poses, the radiance field depicting the scene, and the observer's eye iris texture. We further propose a simple regularization prior on the iris texture pattern to improve reconstruction quality. Through various experiments on synthetic and real-world captures featuring people with varied eye colors, we demonstrate the feasibility of our approach to recover 3D scenes using eye reflections.

透过你的眼睛看世界

Seeing the World through Your Eyes

摘要

Support