利用大型语言模型加速与严重运动障碍用户的交流
Using Large Language Models to Accelerate Communication for Users with Severe Motor Impairments
December 3, 2023
作者: Shanqing Cai, Subhashini Venugopalan, Katie Seaver, Xiang Xiao, Katrin Tomanek, Sri Jalasutram, Meredith Ringel Morris, Shaun Kane, Ajit Narayanan, Robert L. MacDonald, Emily Kornman, Daniel Vance, Blair Casey, Steve M. Gleason, Philip Q. Nelson, Michael P. Brenner
cs.AI
摘要
寻找加速深度运动障碍者文本输入的方法一直是一个长期研究领域。缩小替代和辅助沟通（AAC）设备（如眼动跟踪键盘）的速度差距对于提高这些个体的生活质量至关重要。自然语言神经网络的最新进展为重新思考AAC用户的增强文本输入策略和用户界面提供了新机遇。在本文中，我们提出了SpeakFaster，包括大型语言模型（LLMs）和一个共同设计的用户界面，用于高度缩写形式的文本输入，离线模拟中比传统预测键盘节省了57%的动作。在一个由19名非AAC参与者手动在移动设备上输入的试点研究中，展示了与离线模拟一致的动作节省增益，同时对整体输入速度产生了相对较小的影响。对两名患有肌萎缩侧索硬化症（ALS）的眼球注视输入用户进行的实验室和现场测试表明，由于通过上下文感知LLMs的短语和单词预测实现了显著的节省昂贵击键，文本输入速度比传统基线快29-60%。这些发现为进一步探索面向运动受损用户的大幅加速文本通信奠定了坚实基础，并展示了将LLMs应用于基于文本的用户界面的方向。
English
Finding ways to accelerate text input for individuals with profound motor
impairments has been a long-standing area of research. Closing the speed gap
for augmentative and alternative communication (AAC) devices such as
eye-tracking keyboards is important for improving the quality of life for such
individuals. Recent advances in neural networks of natural language pose new
opportunities for re-thinking strategies and user interfaces for enhanced
text-entry for AAC users. In this paper, we present SpeakFaster, consisting of
large language models (LLMs) and a co-designed user interface for text entry in
a highly-abbreviated form, allowing saving 57% more motor actions than
traditional predictive keyboards in offline simulation. A pilot study with 19
non-AAC participants typing on a mobile device by hand demonstrated gains in
motor savings in line with the offline simulation, while introducing relatively
small effects on overall typing speed. Lab and field testing on two eye-gaze
typing users with amyotrophic lateral sclerosis (ALS) demonstrated text-entry
rates 29-60% faster than traditional baselines, due to significant saving of
expensive keystrokes achieved through phrase and word predictions from
context-aware LLMs. These findings provide a strong foundation for further
exploration of substantially-accelerated text communication for motor-impaired
users and demonstrate a direction for applying LLMs to text-based user
interfaces.