GROVE：基于流式视频经验的时间分层记忆生长与推理

摘要

可穿戴助手既应能回答关于其视觉历史的问题，也应能识别该历史对当前情境的适用性。现有的视频记忆系统主要支持基于问题的条件式回忆，而主动式助手通常采用分离的记忆与控制机制。我们提出GROVE，一个免训练框架，通过从连续视频流中因果式地增长单一记忆，同时支持上述两种行为。GROVE保留细粒度的感知证据，并将其增量式整合为带时间戳的瞬间、连贯的情节片段以及跨天的重复模式。每个层级均配有一种尺度原生的检索技能，用于定位观测、回放活动或遍历长时程规律。反应式问答与主动式辅助共享这一记忆及访问接口，区别仅在于检索是由用户查询触发还是由当前情境触发。在包括具有挑战性的MM-lifelong和EgoServe在内的多个基准测试中，GROVE在所比较的方法中取得了最佳结果。受控消融实验表明，各时间层级及其访问技能具有互补性，其中模式在证据跨越数天时带来的收益最大。代码将在https://github.com/SitongGong/GROVE 上公开。

English

A wearable assistant should both answer questions about its visual history and recognize when that history is useful to the present situation. Existing video-memory systems primarily support question-conditioned recall, whereas proactive assistants typically use separate memory and control mechanisms. We introduce GROVE, a training-free framework that supports both behaviors with one memory grown causally from a continuous video stream. GROVE retains fine-grained perceptual evidence and incrementally consolidates it into time-stamped moments, coherent episodes, and recurring cross-day patterns. Each stratum is paired with a scale-native retrieval skill for locating an observation, replaying an activity, or traversing long-range regularities. Reactive QA and proactive assistance share this memory and access interface, differing in whether retrieval is initiated by a user query or the current situation. Across multiple benchmarks including the challenging MM-lifelong and EgoServe, GROVE achieves the best results among the compared methods. Controlled ablations show that the temporal strata and their access skills are complementary, with patterns providing the largest benefit when evidence spans multiple days. Code will be available at https://github.com/SitongGong/GROVE.