ReflectWorld-MM：一個以實體為導向的多模態記憶系統，用於開放式影片串流

摘要

打造一個能持續觀察世界、記憶所見、並根據累積經驗進行推理的助手，是長久以來的目標。近年來，配備影片串流長期記憶的多模態代理引起了越來越多的關注。不幸的是，現有系統要嘛將記憶保留在模型上下文內，要嘛存放在扁平的特徵儲存庫中，且以幀為組織單位，而非以串流真正關注的持續性實體為核心，這使得它們受限於有限的影片，削弱了追蹤哪些人、哪些事物隨時間反覆出現的能力。在本文中，我們提出 ReflectWorld-MM，一個專為開放式影片串流設計的實體導向多模態記憶系統。該系統由三部分組成。第一部分是感知前端，能將視聽串流轉換為在有限短期記憶下的實體解析觀測。第二部分是基於人類記憶理論的分層長期記憶，結合了多尺度情節記憶、逐步演進的以實體為中心的語義記憶，以及程序性記憶。第三部分是完整的實現，專為真實世界應用而建置，可攝入任意串流並整合至現成助手中。在六個長影片與終身記憶基準測試中，ReflectWorld-MM 在所有六項指標上均達到最佳準確率，超越了強大的記憶代理與前沿模型。

English

Building assistants that can continually watch the world, remember what they see, and reason over their accumulated experience is a long-standing goal, and recently multimodal agents equipped with long-term memory over video streams have attracted increasing interest. Unfortunately, existing systems either keep their memory inside the model context or in a flat feature store, and organize it around frames rather than around the persistent entities a stream is really about, which confines them to bounded videos and weakens their ability to track who and what reappears over time. In this paper, we propose ReflectWorld-MM, an entity-oriented multimodal memory system for open-ended video streams. It consists of three parts. The first is a perception front-end that turns an audiovisual stream into entity-resolved observations under a bounded short-term memory. The second is a hierarchical long-term memory, grounded in human memory theory, that couples a multi-scale episodic memory, an evolving entity-centric semantic memory, and a procedural memory. The third is a complete realization, built for real-world operation, that ingests arbitrary streams and plugs into off-the-shelf assistants. Across six long-video and lifelong-memory benchmarks, ReflectWorld-MM achieves the best accuracy on all six, outperforming strong memory agents and a frontier model.