StreamArena：迈向连续、交互式、长时程智能体流式视频理解

摘要

在连续、真实世界环境中部署自主多模态智能体，要求其摄入无界的音视频流并维持小时级记忆。然而，当前的评估主要依赖于短视频片段和多项选择格式。这种设计使得仅处理最后四帧的最小基线模型能够匹配甚至超越复杂的流式模型，同时答案选项也暴露了语言捷径。我们提出StreamArena，一个面向小时级、交互式流式视频理解的基准测试。StreamArena包含243个全长视频，平均时长88.8分钟，以及3,646个经过严格标注的开放式问答对，用于评估实时感知、历史回溯、主动交互和多模态工具利用能力。跨多种系统的评估揭示了持续交互与长时程多模态理解之间的张力。仅保留最近帧的方法无法恢复遥远事件，将过去观察转换为文本的方法会丢失视觉证据，而反复压缩视觉记忆的方法难以随时间保留细粒度细节。针对这一张力，我们提出StreamMind，一种两层架构，将延迟关键型交互和主动监控分配给独立调度的前端工作者，而后端工作者异步构建持久化多模态记忆，并执行历史回召和外部搜索。StreamMind在全部四项能力上优于现有流式基线，并通过复用持久化状态降低了查询到回答的延迟。

English

Deploying autonomous multimodal agents in continuous, real-world environments requires them to ingest unbounded audio-visual streams and maintain hour-scale memory. However, current evaluations predominantly rely on brief clips and multiple-choice formats. This design allows minimal baselines that process only the last four frames to match or surpass complex streaming models, while answer options also expose language shortcuts. We introduce StreamArena, a benchmark for hour-scale, interactive streaming video understanding. StreamArena contains 243 full-length videos averaging 88.8 minutes and 3,646 rigorously annotated, open-ended question-answer pairs that evaluate real-time perception, historical retrospection, proactive interaction, and multimodal tool utilization. Evaluation across diverse systems exposes a tension between continuous interaction and long-horizon multimodal comprehension. Methods that retain only recent frames cannot recover distant events, methods that convert past observations into text lose visual evidence, and methods that repeatedly compress visual memory struggle to preserve fine-grained details over time. We address this tension with StreamMind, a two-tier architecture that assigns latency-critical interaction and proactive monitoring to independently scheduled frontend workers, while backend workers asynchronously construct persistent multimodal memory and perform historical recall and external search. StreamMind outperforms existing streaming baselines across all four capabilities and reduces query-to-answer latency by reusing persistent state.