EvoEmbedding：用於長上下文檢索與智能體記憶的可演化表示

摘要

现有的嵌入模型本质上是静态的：它们孤立地编码文本片段，忽略了周围的上下文和时间顺序。本文提出EvoEmbedding，一种创新的嵌入模型，能够生成可演化的检索表示。该模型专为长上下文场景设计，此类场景中的信息具有动态性、序列性，并需要持续的状态追踪。我们的设计简洁明了：EvoEmbedding在顺序处理输入时维护一个持续更新的潜在记忆，并将其与原始内容联合生成可演化的嵌入。因此，对于同一查询，我们的模型能够根据不断演化的上下文调整其表示，以检索不同的目标，超越了静态语义搜索。为使模型具备此能力，我们构建了EvoTrain-180K，一个多样化的数据集，用于潜在记忆与检索的联合优化。此外，我们引入了记忆队列以防止循环编码过程中的表征坍塌，以及分段批处理技术，以应对显著的长度差异并将训练速度提升3.8倍。大量实验表明，我们的模型不仅在多项长上下文检索基准测试中超越了更大规模的专业模型（例如Qwen3-Embedding-8B和KaLM-Embedding-Gemma3-12B），还能很好地泛化到上下文长度为其训练窗口10倍的下游任务（如个性化）。值得注意的是，EvoEmbedding能够无缝集成到智能体工作流中以提升性能。例如，一个配备了我们模型的简单RAG流水线，其表现超过了专门的智能体记忆系统。项目页面：https://clare-nie.github.io/EvoEmbedding。

English

Existing embedding models are inherently static: they encode text segments in isolation, ignoring their surrounding context and temporal order. This paper introduces EvoEmbedding, a novel embedding model that generates evolvable representations for retrieval. It is tailored for long-context scenarios, where information is dynamic, sequential, and requires continuous state tracking. Our design is simple: EvoEmbedding maintains a continuously updated latent memory as it sequentially processes inputs, and uses it alongside the raw content to jointly generate evolvable embeddings. Consequently, for the same query, our model adapts its representation to retrieve distinct targets based on the evolving context, going beyond static semantic search. To equip the model with this capability, we construct EvoTrain-180K, a diverse dataset for the joint optimization of latent memory and retrieval. Furthermore, we introduce a memory queue to prevent representation collapse during recurrent encoding, alongside segment-batching techniques that tackle significant length variance and accelerate training by 3.8times. Extensive experiments show that our model not only outperforms larger-scale specialists (e.g., Qwen3-Embedding-8B and KaLM-Embedding-Gemma3-12B) across a range of long-context retrieval benchmarks, but also generalizes well to downstream tasks (e.g., personalization) with contexts 10times longer than its training window. Notably, EvoEmbedding seamlessly integrates into agentic workflows to boost performance. For instance, a naive RAG pipeline equipped with our model surpasses dedicated agentic memory systems. Project Page: https://clare-nie.github.io/EvoEmbedding.