ReflectWorld-MM：面向实体的多模态记忆系统，用于开放式视频流

摘要

构建能够持续观察世界、记忆所见并基于积累的经验进行推理的智能助手是一个长期目标。近年来，配备视频流长期记忆的多模态智能体引起了越来越多的关注。然而，现有系统要么将记忆保留在模型上下文中，要么存储在扁平的特征存储中，并且围绕帧而非流实际描述的持久实体来组织记忆，这将其限制在有限长度的视频中，并削弱了其随时间追踪谁和什么再次出现的能力。在本文中，我们提出了 ReflectWorld-MM，这是一种面向实体的多模态记忆系统，专为开放式视频流设计。该系统由三部分组成。第一部分是感知前端，将视听流转化为在有限短期记忆下的实体解析观测。第二部分是基于人类记忆理论的分层长期记忆，它结合了多尺度情景记忆、演化的以实体为中心的语义记忆和程序性记忆。第三部分是一个完整的实现，专为实际运行构建，能够接收任意流并接入现成的智能助手。在六个长视频和终身记忆基准测试中，ReflectWorld-MM 在所有六个任务上取得了最佳准确率，超越了强大的记忆智能体和前沿模型。

English

Building assistants that can continually watch the world, remember what they see, and reason over their accumulated experience is a long-standing goal, and recently multimodal agents equipped with long-term memory over video streams have attracted increasing interest. Unfortunately, existing systems either keep their memory inside the model context or in a flat feature store, and organize it around frames rather than around the persistent entities a stream is really about, which confines them to bounded videos and weakens their ability to track who and what reappears over time. In this paper, we propose ReflectWorld-MM, an entity-oriented multimodal memory system for open-ended video streams. It consists of three parts. The first is a perception front-end that turns an audiovisual stream into entity-resolved observations under a bounded short-term memory. The second is a hierarchical long-term memory, grounded in human memory theory, that couples a multi-scale episodic memory, an evolving entity-centric semantic memory, and a procedural memory. The third is a complete realization, built for real-world operation, that ingests arbitrary streams and plugs into off-the-shelf assistants. Across six long-video and lifelong-memory benchmarks, ReflectWorld-MM achieves the best accuracy on all six, outperforming strong memory agents and a frontier model.