Alexa，和機器人一起玩：介紹首屆Alexa Prize SimBot挑戰賽，探討具體化人工智慧。

Alexa, play with robot: Introducing the First Alexa Prize SimBot Challenge on Embodied AI

August 9, 2023

作者: Hangjie Shi, Leslie Ball, Govind Thattai, Desheng Zhang, Lucy Hu, Qiaozi Gao, Suhaila Shakiah, Xiaofeng Gao, Aishwarya Padmakumar, Bofei Yang, Cadence Chung, Dinakar Guthy, Gaurav Sukhatme, Karthika Arumugam, Matthew Wen, Osman Ipek, Patrick Lange, Rohan Khanna, Shreyas Pansare, Vasu Sharma, Chao Zhang, Cris Flagg, Daniel Pressel, Lavina Vaz, Luke Dai, Prasoon Goyal, Sattvik Sahai, Shaohua Liu, Yao Lu, Anna Gottardi, Shui Hu, Yang Liu, Dilek Hakkani-Tur, Kate Bland, Heather Rocker, James Jeun, Yadunandana Rao, Michael Johnston, Akshaya Iyengar, Arindam Mandal, Prem Natarajan, Reza Ghanadan

cs.AI

摘要

Alexa Prize計劃賦予了眾多大學生探索、實驗和展示他們在建構對話代理方面的才能的機會，其中包括SocialBot Grand Challenge和TaskBot Challenge等挑戰。隨著對話代理在多模態和具體化情境中的應用日益增多，探索透過計算機視覺和實體化增強對話互動的可能性變得至關重要。本文描述了SimBot Challenge，這是一項新挑戰，大學團隊將競爭建構在模擬物理環境中完成任務的機器人助手。本文概述了SimBot Challenge，包括線上和線下挑戰階段。我們描述了為團隊提供的基礎設施和支援，包括Alexa Arena、模擬環境以及提供給團隊加速建構視覺和語言模型的ML工具包。我們總結了參賽團隊應對研究挑戰的方法，並提取了關鍵的經驗教訓。最後，我們對比賽期間參賽SimBot的表現進行了分析。

English

The Alexa Prize program has empowered numerous university students to explore, experiment, and showcase their talents in building conversational agents through challenges like the SocialBot Grand Challenge and the TaskBot Challenge. As conversational agents increasingly appear in multimodal and embodied contexts, it is important to explore the affordances of conversational interaction augmented with computer vision and physical embodiment. This paper describes the SimBot Challenge, a new challenge in which university teams compete to build robot assistants that complete tasks in a simulated physical environment. This paper provides an overview of the SimBot Challenge, which included both online and offline challenge phases. We describe the infrastructure and support provided to the teams including Alexa Arena, the simulated environment, and the ML toolkit provided to teams to accelerate their building of vision and language models. We summarize the approaches the participating teams took to overcome research challenges and extract key lessons learned. Finally, we provide analysis of the performance of the competing SimBots during the competition.