EgoForce：前臂引導的攝影機空間3D手部姿勢，來自單眼自我中心攝影機

EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera

May 12, 2026

作者: Christen Millerdurai, Shaoxiang Wang, Yaxu Xie, Vladislav Golyanik, Didier Stricker, Alain Pagani

cs.AI

摘要

從使用者視角利用單一頭戴式攝影機重建手部的絕對三維姿態與形狀，對於擴增實境/虛擬實境（AR/VR）、遠距臨場感及以手為核心的操作任務等實際應用至關重要，且感測系統必須保持輕巧且不顯眼。雖然單眼RGB方法已取得進展，但仍受限於深度尺度模糊性，且難以泛化至頭戴式裝置的多樣光學配置。因此，模型通常需要在特定裝置資料集上進行大量訓練，而這類資料集的獲取既昂貴又耗時。本文提出EgoForce——一種單眼三維手部重建架構，能從使用者（攝影機空間）視角恢復穩健且絕對的三維手部姿態與位置。EgoForce可透過單一統一網路，在魚眼、透視及變形廣視角攝影機模型上運作。我們的方法結合了可微分前臂表示以穩定手部姿態、統一臂-手變換器以從單一自我中心視角預測手部與前臂幾何形狀（從而減輕深度尺度模糊性），以及光線空間閉合解求解器，能在多種頭戴式攝影機模型下實現絕對三維姿態恢復。在三個自我中心基準資料集上的實驗顯示，EgoForce達到了最先進的三維準確度，與先前方法相比，在HOT3D資料集上將攝影機空間MPJPE降低了最高28%，並在不同攝影機配置下保持一致的表現。更多詳情請參閱專案頁面：https://dfki-av.github.io/EgoForce。

English

Reconstructing the absolute 3D pose and shape of the hands from the user's viewpoint using a single head-mounted camera is crucial for practical egocentric interaction in AR/VR, telepresence, and hand-centric manipulation tasks, where sensing must remain compact and unobtrusive. While monocular RGB methods have made progress, they remain constrained by depth-scale ambiguity and struggle to generalize across the diverse optical configurations of head-mounted devices. As a result, models typically require extensive training on device-specific datasets, which are costly and laborious to acquire. This paper addresses these challenges by introducing EgoForce, a monocular 3D hand reconstruction framework that recovers robust, absolute 3D hand pose and its position from the user's (camera-space) viewpoint. EgoForce operates across fisheye, perspective, and distorted wide-FOV camera models using a single unified network. Our approach combines a differentiable forearm representation that stabilizes hand pose, a unified arm-hand transformer that predicts both hand and forearm geometry from a single egocentric view, mitigating depth-scale ambiguity, and a ray space closed-form solver that enables absolute 3D pose recovery across diverse head-mounted camera models. Experiments on three egocentric benchmarks show that EgoForce achieves state-of-the-art 3D accuracy, reducing camera-space MPJPE by up to 28% on the HOT3D dataset compared to prior methods and maintaining consistent performance across camera configurations. For more details, visit the project page at https://dfki-av.github.io/EgoForce.