手部可见性检测器：面向手部的逐关键点可见性估计

摘要

手部姿态估计（Hand Pose Estimation，HPE）是增强现实/虚拟现实（AR/VR）和机器人等多种应用的基础技术。在这些应用中，图像中每个手部关节点的可见性对于评估遮挡条件下估计结果的可靠性至关重要。然而，大多数现有的HPE方法仅输出关节点位置，并未明确指示其可见性。尽管某些方法考虑了遮挡或可见性因素，但可见性估计主要被用作改进姿态估计的辅助信号。据我们所知，手部逐关节可见性估计尚未作为独立任务被系统性地研究。在这项工作中，我们提出了Hand Visibility Detector（手部可见性检测器），一个用于估计单个手部关节点可见性的模型，并首次将可见性估计作为独立任务进行系统性研究。我们证明，利用在大规模数据上预训练的HPE模型作为骨干网络所蕴含的先验知识，能够在该任务中取得优异性能。我们进一步展示了Hand Visibility Detector在通过多视角三角化二维关键点进行三维手部姿态标注这一下游任务中的实用性，结果表明可见性加权三角化能够有效降低重投影误差。我们的方法已作为可直接使用的软件包发布，代码和演示可在 https://github.com/ryhara/hand_visibility_detector 获取。

English

Hand Pose Estimation (HPE) is a fundamental technology for various applications such as AR/VR and robotics. In these applications, the visibility of each hand joint in the image is crucial for assessing the reliability of estimation results under occlusion. However, most existing HPE methods output joint positions without explicitly indicating their visibility. Although some methods account for occlusion or visibility, visibility estimation has mainly been used as an auxiliary signal for improving pose estimation. To our knowledge, per-joint hand visibility estimation has not been systematically studied as a standalone task. In this work, we propose Hand Visibility Detector, a model for estimating the visibility of individual hand joints, and present the first systematic investigation of visibility estimation as an independent task. We show that leveraging the prior knowledge of HPE models pretrained on large-scale data as a backbone yields high performance in this task. We further demonstrate the utility of Hand Visibility Detector on a downstream task of 3D hand pose annotation via multi-view triangulation of 2D keypoints, showing that visibility-weighted triangulation reduces reprojection error. Our method is released as a ready-to-use package, and the code and demo are available at https://github.com/ryhara/hand_visibility_detector .