AVA-编码器：迈向智能体原生的视频表征学习

摘要

创意代理仍然缺乏从高质量人类电影中学习的有效方式，这限制了它们制作电影级视频的能力。一个关键挑战是缺乏结构化的视频表示，这种表示既要忠实于电影内容，又要可直接用于代理推理和操作。为应对这一挑战，我们提出了代理式视频自编码器（AVA-Encoder），一种通过代理式自编码学习代理原生视频表示的框架。 AVA-Encoder将视频转换为知识图谱（KG）表示，然后将其重建回视频。其层级结构节点和状态节点存储结构化文本，而关联的资源层则保存生成的图像、音频和视频。类型化边以代理易于理解、查询和编辑的形式保留这些文本描述与资源之间的关系。视频重建差异驱动一个文本梯度优化框架，该框架将评估反馈表达为自然语言更新方向，用于外循环中的数据无关编码策略伪训练，以及测试时内循环中可选的数据相关知识图谱表示细化。大量实验表明，AVA-Encoder相较于最强外部基线提升了20.7个百分点。在受控的仅策略设置中，其经过伪训练的镜头级代理式视频编码器策略也优于精心人工调优的策略，同时使用的系统提示词令牌减少了74.3%。我们发布了完整的AVA-Encoder框架、一个可靠的代理式视频重建基准，以及首个高质量电影知识图谱表示数据集。

English

Creative agents still lack an effective way to learn from high-quality human films, limiting their ability to produce cinematic-grade videos. A key challenge is the absence of a structured video representation that is both faithful to film content and directly usable for agentic reasoning and manipulation. To address the challenge, we propose the Agentic Video Auto-Encoder (AVA-Encoder), a framework for learning agent-native video representations via agentic auto-encoding. AVA-Encoder transforms a video into a knowledge graph (KG) representation and then reconstructs it back into video. Its hierarchy and state nodes store structured text, while a linked asset layer holds generated images, audio, and video. Typed edges preserve the relations between these text descriptions and assets in a form that agents can easily understand, query, and edit. The video reconstruction differences drive a textual-gradient optimization framework, which expresses evaluation feedback as natural-language update directions for Data-Independent Encoding Policy Pseudo-Training in the outer loop and optional Data-Dependent KG Representation Refinement in the test-time inner loop. Extensive experiments show that AVA-Encoder improves by 20.7 percentage points over the strongest external baseline. In the controlled policy-only setting, its pseudo-trained shot-level Agentic Video Encoder policy also outperforms a carefully human-tuned policy while using 74.3% fewer system-prompt tokens. We release the complete AVA-Encoder framework, a reliable agentic video reconstruction benchmark, and the first dataset of high-quality film KG representations.