基于视觉的手势定制技术：从单次演示中学习

Vision-Based Hand Gesture Customization from a Single Demonstration

February 13, 2024

作者: Soroush Shahi, Cori Tymoszek Park, Richard Kang, Asaf Liberman, Oron Levy, Jun Gong, Abdelkareem Bedri, Gierad Laput

cs.AI

摘要

手势识别正逐渐成为人机交互中更为普遍的模式，尤其是随着摄像头在日常设备中的普及。尽管在这一领域取得了持续进展，手势定制往往被忽视。定制至关重要，因为它使用户能够定义和展示更加自然、易记和易访问的手势。然而，定制需要高效利用用户提供的数据。我们提出了一种方法，使用户能够通过单目摄像头从一个演示轻松设计定制手势。我们采用了Transformer和元学习技术来解决少样本学习的挑战。与先前的工作不同，我们的方法支持任意组合的单手、双手、静态和动态手势，包括不同视角。我们通过对来自21名参与者的20种手势进行的用户研究评估了我们的定制方法，从一个演示中实现了高达97%的平均识别准确率。我们的工作为基于视觉的手势定制提供了可行途径，为该领域未来的进展奠定了基础。

English

Hand gesture recognition is becoming a more prevalent mode of human-computer interaction, especially as cameras proliferate across everyday devices. Despite continued progress in this field, gesture customization is often underexplored. Customization is crucial since it enables users to define and demonstrate gestures that are more natural, memorable, and accessible. However, customization requires efficient usage of user-provided data. We introduce a method that enables users to easily design bespoke gestures with a monocular camera from one demonstration. We employ transformers and meta-learning techniques to address few-shot learning challenges. Unlike prior work, our method supports any combination of one-handed, two-handed, static, and dynamic gestures, including different viewpoints. We evaluated our customization method through a user study with 20 gestures collected from 21 participants, achieving up to 97% average recognition accuracy from one demonstration. Our work provides a viable path for vision-based gesture customization, laying the foundation for future advancements in this domain.