LLAMAPIE：主动式入耳对话助手

LLAMAPIE: Proactive In-Ear Conversation Assistants

May 7, 2025

作者: Tuochao Chen, Nicholas Batchelder, Alisa Liu, Noah Smith, Shyamnath Gollakota

cs.AI

摘要

我们推出LlamaPIE，这是首个旨在通过可听设备提供低调、简洁指导来增强人类对话的实时主动助手。与需要用户明确调用的传统语言模型不同，该助手在后台运行，预测用户需求而不打断对话。我们解决了多项挑战，包括确定何时响应、制作简洁对话增强的回复、利用用户知识实现情境感知辅助，以及实时设备端处理。为此，我们构建了一个半合成对话数据集，并提出了一种双模型流水线：一个小型模型决定何时响应，一个大型模型生成回复。我们在真实世界数据集上评估了该方法，证明了其在提供有益且不引人注目的辅助方面的有效性。在Apple Silicon M2硬件上实现的用户研究表明，相较于无辅助基线和反应式模型，用户显著偏好主动助手，凸显了LlamaPIE在提升实时对话体验方面的潜力。

English

We introduce LlamaPIE, the first real-time proactive assistant designed to enhance human conversations through discreet, concise guidance delivered via hearable devices. Unlike traditional language models that require explicit user invocation, this assistant operates in the background, anticipating user needs without interrupting conversations. We address several challenges, including determining when to respond, crafting concise responses that enhance conversations, leveraging knowledge of the user for context-aware assistance, and real-time, on-device processing. To achieve this, we construct a semi-synthetic dialogue dataset and propose a two-model pipeline: a small model that decides when to respond and a larger model that generates the response. We evaluate our approach on real-world datasets, demonstrating its effectiveness in providing helpful, unobtrusive assistance. User studies with our assistant, implemented on Apple Silicon M2 hardware, show a strong preference for the proactive assistant over both a baseline with no assistance and a reactive model, highlighting the potential of LlamaPie to enhance live conversations.