匹配需成双：生成式检索器与强化学习的协同进化

摘要

检索是现代搜索与广告系统的第一阶段，目的是从大规模物料全集中选取候选集，供下游排序与拍卖使用。近期研究日益借助大语言模型（LLM），通过查询扩展、数据合成与检索反馈训练来改进检索效果。然而，生成组件通常仅用于查询侧增广，最终匹配仍交由下游检索器完成。我们提出 CoGR——一种检索框架，其核心思路是直接训练 LLM 在查询侧与物料侧构建检索表征。每个生成器产出紧凑的关键词集合，并通过倒排索引直接完成匹配，从而保持与现有基于关键词检索基础设施的兼容性。CoGR 采用两阶段训练流程：监督微调先建立对齐的关键词空间，随后协同进化强化学习在对方侧索引保持冻结的条件下，以 GRPO 算法交替优化查询侧与物料侧生成器。双方共享相同的查询—物料检索 F₁ 目标：查询侧直接获得检索 F₁ 奖励，物料侧则获得反事实边际奖励，用以衡量其生成关键词所导致的查询侧 F₁ 变化。在 10 个具有代表性的稀疏、稠密与生成式基线上，CoGR 分别在内部 APP 应用市场数据集和公开 WANDS 基准上取得最佳性能，相对最强基线的 F₁ 提升幅度分别达到 10.9% 与 36.1%。进一步分析显示，训练过程中协同进化保持稳定，查询侧与物料侧关键词空间的对齐程度持续增强。

English

Retrieval is the first stage of modern search and advertising systems, selecting a candidate set from a large item universe for downstream ranking and auction. Recent work increasingly leverages LLMs to improve retrieval through query expansion, data synthesis, and retrieval-feedback training. However, the generative component is typically used for query-side augmentation, while final matching is still delegated to a downstream retriever. We introduce CoGR, a retrieval framework that instead trains LLMs to directly construct retrieval representations on both query and item sides. Each generator produces a compact set of keywords, which are matched directly through an inverted index, preserving compatibility with existing keyword-based retrieval infrastructure. CoGR uses a two-stage training pipeline. Supervised fine-tuning first establishes an aligned keyword space, after which co-evolving reinforcement learning alternately optimizes the query- and item-side generators with GRPO against the opposite side's frozen index. Both sides optimize the same query-to-item retrieval F_1 objective: the query side receives retrieval F_1 directly, while the item side receives a counterfactual marginal reward measuring the change in query-side F_1 caused by its generated keywords. Across 10 representative sparse, dense, and generative baselines, CoGR achieves the best performance on both an internal APP Marketplace dataset and the public WANDS benchmark, improving F_1 over the strongest baseline by 10.9% and 36.1%, respectively. Further analysis shows stable co-evolution and increasingly aligned query--item keyword spaces over training.