手部可見性檢測器：手部逐關鍵點可見性估計

摘要

手部姿態估計（HPE）是擴增實境/虛擬實境（AR/VR）與機器人技術等各種應用中的基礎技術。在這些應用中，影像中每個手部關節的可見性對於在遮擋情況下評估估計結果的可靠性至關重要。然而，大多數現有的HPE方法僅輸出關節位置，並未明確指示其可見性。儘管有些方法考量了遮擋或可見性，但可見性估計主要被用作改善姿態估計的輔助信號。據我們所知，逐關節的手部可見性估計尚未作為一項獨立任務進行系統性研究。在本研究中，我們提出了手部可見性偵測器（Hand Visibility Detector），這是一個用於估計個別手部關節可見性的模型，並對作為獨立任務的可見性估計進行了首次系統性研究。我們證明，利用在大規模資料上預訓練的HPE模型作為骨幹，並運用其先驗知識，能在這項任務中取得高效能。我們進一步展示了手部可見性偵測器在一個下游任務中的效用：透過二維關鍵點的多視角三角測量來進行三維手部姿態標註，結果顯示可見性加權三角測量能降低重投影誤差。我們的方法已作為即用型套件發布，程式碼與示範可於 https://github.com/ryhara/hand_visibility_detector 取得。

English

Hand Pose Estimation (HPE) is a fundamental technology for various applications such as AR/VR and robotics. In these applications, the visibility of each hand joint in the image is crucial for assessing the reliability of estimation results under occlusion. However, most existing HPE methods output joint positions without explicitly indicating their visibility. Although some methods account for occlusion or visibility, visibility estimation has mainly been used as an auxiliary signal for improving pose estimation. To our knowledge, per-joint hand visibility estimation has not been systematically studied as a standalone task. In this work, we propose Hand Visibility Detector, a model for estimating the visibility of individual hand joints, and present the first systematic investigation of visibility estimation as an independent task. We show that leveraging the prior knowledge of HPE models pretrained on large-scale data as a backbone yields high performance in this task. We further demonstrate the utility of Hand Visibility Detector on a downstream task of 3D hand pose annotation via multi-view triangulation of 2D keypoints, showing that visibility-weighted triangulation reduces reprojection error. Our method is released as a ready-to-use package, and the code and demo are available at https://github.com/ryhara/hand_visibility_detector .