LLAMAPIE：主動式耳內對話輔助系統

LLAMAPIE: Proactive In-Ear Conversation Assistants

May 7, 2025

作者: Tuochao Chen, Nicholas Batchelder, Alisa Liu, Noah Smith, Shyamnath Gollakota

cs.AI

摘要

我們推出LlamaPIE，首款旨在通過可聽設備提供隱蔽、簡潔指導來提升人類對話的實時主動助手。與傳統需用戶明確調用的語言模型不同，此助手在背景中運行，預測用戶需求而不打斷對話。我們解決了多項挑戰，包括決定何時回應、構建簡潔且能提升對話的回應、利用用戶知識進行情境感知輔助，以及實現實時、設備端處理。為此，我們構建了一個半合成對話數據集，並提出了一個雙模型管道：一個小型模型決定何時回應，而一個更大的模型生成回應。我們在真實世界數據集上評估了該方法，證明了其在提供有益且不顯眼輔助方面的有效性。在Apple Silicon M2硬件上實現的用戶研究顯示，相比無輔助基線和反應式模型，用戶對主動助手表現出強烈偏好，凸顯了LlamaPIE在增強實時對話方面的潛力。

English

We introduce LlamaPIE, the first real-time proactive assistant designed to enhance human conversations through discreet, concise guidance delivered via hearable devices. Unlike traditional language models that require explicit user invocation, this assistant operates in the background, anticipating user needs without interrupting conversations. We address several challenges, including determining when to respond, crafting concise responses that enhance conversations, leveraging knowledge of the user for context-aware assistance, and real-time, on-device processing. To achieve this, we construct a semi-synthetic dialogue dataset and propose a two-model pipeline: a small model that decides when to respond and a larger model that generates the response. We evaluate our approach on real-world datasets, demonstrating its effectiveness in providing helpful, unobtrusive assistance. User studies with our assistant, implemented on Apple Silicon M2 hardware, show a strong preference for the proactive assistant over both a baseline with no assistance and a reactive model, highlighting the potential of LlamaPie to enhance live conversations.