透過你的眼睛看世界

摘要

人眼的反射特性是一個被低估的資訊來源，提供了關於我們周圍世界的外觀的信息。透過對移動中人眼的成像，我們可以透過眼睛中的反射收集攝像機視線之外的場景的多個視角。本文中，我們使用包含眼睛反射的肖像圖像重建攝像機視線之外的3D場景。這個任務具有挑戰性，原因在於1）準確估計眼睛姿勢的困難，以及2）眼睛虹膜和場景反射的混合外觀。我們的方法聯合優化角膜姿勢、描繪場景的輻射場，以及觀察者的眼睛虹膜紋理。我們進一步提出了對虹膜紋理圖案的簡單正則化先驗，以提高重建質量。通過對合成和真實世界拍攝的各種實驗，展示了我們的方法在恢復使用眼睛反射的3D場景方面的可行性，這些實驗包括具有不同眼睛顏色的人物。

English

The reflective nature of the human eye is an underappreciated source of information about what the world around us looks like. By imaging the eyes of a moving person, we can collect multiple views of a scene outside the camera's direct line of sight through the reflections in the eyes. In this paper, we reconstruct a 3D scene beyond the camera's line of sight using portrait images containing eye reflections. This task is challenging due to 1) the difficulty of accurately estimating eye poses and 2) the entangled appearance of the eye iris and the scene reflections. Our method jointly refines the cornea poses, the radiance field depicting the scene, and the observer's eye iris texture. We further propose a simple regularization prior on the iris texture pattern to improve reconstruction quality. Through various experiments on synthetic and real-world captures featuring people with varied eye colors, we demonstrate the feasibility of our approach to recover 3D scenes using eye reflections.

透過你的眼睛看世界

Seeing the World through Your Eyes

摘要

Support