利用大型語言模型加速與嚴重運動障礙用戶的溝通

Using Large Language Models to Accelerate Communication for Users with Severe Motor Impairments

December 3, 2023

作者: Shanqing Cai, Subhashini Venugopalan, Katie Seaver, Xiang Xiao, Katrin Tomanek, Sri Jalasutram, Meredith Ringel Morris, Shaun Kane, Ajit Narayanan, Robert L. MacDonald, Emily Kornman, Daniel Vance, Blair Casey, Steve M. Gleason, Philip Q. Nelson, Michael P. Brenner

cs.AI

摘要

尋找加速深度運動障礙者文本輸入方式的方法一直是研究的重要領域。縮小輔助和替代溝通（AAC）設備的速度差距，如眼動追蹤鍵盤，對於提升這些個體的生活質量至關重要。自然語言神經網絡的最新進展為重新思考增強AAC使用者文本輸入的策略和用戶界面提供了新機遇。本文介紹了SpeakFaster，包括大型語言模型（LLMs）和一個共同設計的用戶界面，以高度縮寫的形式進行文本輸入，比傳統預測鍵盤在離線模擬中節省了57%的運動動作。一項由19名非AAC參與者手動在移動設備上輸入的初步研究展示了與離線模擬相符的運動節省，同時對整體打字速度產生了相對較小的影響。在兩名患有肌萎縮性脊髓側索硬化症（ALS）的眼神注視打字使用者身上進行的實驗室和現場測試顯示，由於從上下文感知的LLMs實現的短語和單詞預測大幅節省了昂貴的按鍵次數，文本輸入速度比傳統基準快29-60%。這些發現為進一步探索為運動障礙用戶大幅加速文本溝通提供了堅實基礎，並展示了將LLMs應用於基於文本的用戶界面的方向。

English

Finding ways to accelerate text input for individuals with profound motor impairments has been a long-standing area of research. Closing the speed gap for augmentative and alternative communication (AAC) devices such as eye-tracking keyboards is important for improving the quality of life for such individuals. Recent advances in neural networks of natural language pose new opportunities for re-thinking strategies and user interfaces for enhanced text-entry for AAC users. In this paper, we present SpeakFaster, consisting of large language models (LLMs) and a co-designed user interface for text entry in a highly-abbreviated form, allowing saving 57% more motor actions than traditional predictive keyboards in offline simulation. A pilot study with 19 non-AAC participants typing on a mobile device by hand demonstrated gains in motor savings in line with the offline simulation, while introducing relatively small effects on overall typing speed. Lab and field testing on two eye-gaze typing users with amyotrophic lateral sclerosis (ALS) demonstrated text-entry rates 29-60% faster than traditional baselines, due to significant saving of expensive keystrokes achieved through phrase and word predictions from context-aware LLMs. These findings provide a strong foundation for further exploration of substantially-accelerated text communication for motor-impaired users and demonstrate a direction for applying LLMs to text-based user interfaces.