训练智能体与其框架协同进化：TaoLive数字人智能体技术报告

摘要

AI驱动的数字人主播必须实时回答商品问题、与观众互动并执行营销策略，这对低延迟、频繁的策略更新以及准确且有效的响应提出了要求。可进化Harness（Evolvable Harnesses）的Skills、Hooks、提示词和工具可以独立于模型权重进行更新，从而支持快速迭代，但这也带来了一个权衡：大模型虽然能零样本适应但速度过慢，而紧凑模型虽然能满足延迟目标却会过拟合于固定的Harness配置。我们提出Harness感知训练（Harness-Aware Training, HAT），训练紧凑模型以适应变化的Harness。其核心组件Harness状态增强（Harness-State Augmentation, HSA）对Skill标识符及其内容、工具模式、提示词结构和Hook函数施加保持任务不变的变换。训练分为三个阶段：HSA-SFT阶段从强模型在多样化环境中的轨迹中学习推理和工具使用；通用在线蒸馏（General On-Policy Distillation）阶段恢复SFT过程中丢失的泛化能力；HSA-RL阶段通过增强环境中的强化学习提升模型对变化Harness的鲁棒性。在四个评测集上，HAT在直播问答（Live-Stream QA）上达到94.8分（基线模型：80.3分；最强通用大语言模型：93.0分），在Harness变体问答（Harness-Variant QA）上达到94.6分（基线模型：75.4分）。与固定Harness的SFT相比——后者使IFEval相较基线模型下降7.7个百分点——HAT避免了这一退化并达到83.5分。在单块NVIDIA H20 GPU上，优化后的系统实现了3.4秒的P50延迟和8.1秒的P95延迟。该系统已部署于淘宝直播的数字人服务中，并在GMV和商品详情页浏览量方面取得了正向的线上A/B测试结果。

English

AI-powered digital avatar streamers must answer product questions, engage viewers, and execute marketing strategies in real time, demanding low latency, frequent strategy updates, and accurate yet effective responses. Evolvable Harnesses, whose Skills, Hooks, prompts, and tools can be updated independently of model weights, enable rapid iteration but expose a trade-off: large models adapt zero-shot yet are too slow, whereas compact models meet latency targets but overfit to fixed Harness configurations. We propose Harness-Aware Training (HAT), which trains compact models to adapt to changing Harnesses. Its key component, Harness-State Augmentation (HSA), applies task-preserving transformations to Skill identifiers and content, tool schemas, prompt structures, and Hook functions. Training proceeds in three stages: HSA-SFT learns reasoning and tool use from strong-model trajectories across diverse environments; General On-Policy Distillation restores generalization lost during SFT; and HSA-RL improves robustness to changing Harnesses through reinforcement learning in augmented environments. Across four evaluation sets, HAT achieves 94.8 on Live-Stream QA (base: 80.3; strongest general LLM: 93.0) and 94.6 on Harness-Variant QA (base: 75.4). Unlike Fixed-Harness SFT, which lowers IFEval by 7.7 points from the base model, HAT avoids this regression and reaches 83.5. On one NVIDIA H20 GPU, the optimized system delivers P50 and P95 latencies of 3.4 s and 8.1 s. Deployed in Taobao Live's digital-avatar service, it also yields positive online A/B test results for GMV and item-page views.