訓練智能體與其框架共同演化：TaoLive 數位化身智能體技術報告

摘要

AI 驅動的數位虛擬主播必須即時回答產品問題、與觀眾互動並執行行銷策略，這要求低延遲、頻繁的策略更新以及準確且有效的回應。可演化的 Harness（框架）其技能、Hook、提示詞與工具可獨立於模型權重進行更新，從而實現快速迭代，但也揭示了一種取捨：大型模型能零樣本適應但速度太慢，而緊湊型模型雖能滿足延遲目標，卻會過度擬合於固定的 Harness 配置。我們提出 Harness 感知訓練（HAT），訓練緊湊型模型以適應不斷變化的 Harness。其關鍵組件 Harness 狀態擴增（HSA）對技能識別碼與內容、工具結構描述、提示詞結構以及 Hook 函式施加保留任務的轉換。訓練分三個階段進行：HSA 監督式微調（HSA-SFT）從跨多種環境的強模型軌跡中學習推理與工具使用；通用同策略蒸餾恢復 SFT 期間喪失的泛化能力；HSA 強化學習（HSA-RL）則在擴增環境中透過強化學習提高對不斷變化的 Harness 的穩健性。在四個評估集上，HAT 在直播問答（Live-Stream QA）上達到 94.8（基線：80.3；最強通用 LLM：93.0），在 Harness 變體問答（Harness-Variant QA）上達到 94.6（基線：75.4）。與固定 Harness 的 SFT 不同——後者使 IFEval 比基線模型下降 7.7 分——HAT 避免了這項退化，達到 83.5。在單張 NVIDIA H20 GPU 上，優化後的系統達到 P50 延遲 3.4 秒及 P95 延遲 8.1 秒。該系統部署於淘寶直播的數位虛擬主播服務中，並在 GMV 與商品頁瀏覽量上取得正向的線上 A/B 測試結果。

English

AI-powered digital avatar streamers must answer product questions, engage viewers, and execute marketing strategies in real time, demanding low latency, frequent strategy updates, and accurate yet effective responses. Evolvable Harnesses, whose Skills, Hooks, prompts, and tools can be updated independently of model weights, enable rapid iteration but expose a trade-off: large models adapt zero-shot yet are too slow, whereas compact models meet latency targets but overfit to fixed Harness configurations. We propose Harness-Aware Training (HAT), which trains compact models to adapt to changing Harnesses. Its key component, Harness-State Augmentation (HSA), applies task-preserving transformations to Skill identifiers and content, tool schemas, prompt structures, and Hook functions. Training proceeds in three stages: HSA-SFT learns reasoning and tool use from strong-model trajectories across diverse environments; General On-Policy Distillation restores generalization lost during SFT; and HSA-RL improves robustness to changing Harnesses through reinforcement learning in augmented environments. Across four evaluation sets, HAT achieves 94.8 on Live-Stream QA (base: 80.3; strongest general LLM: 93.0) and 94.6 on Harness-Variant QA (base: 75.4). Unlike Fixed-Harness SFT, which lowers IFEval by 7.7 points from the base model, HAT avoids this regression and reaches 83.5. On one NVIDIA H20 GPU, the optimized system delivers P50 and P95 latencies of 3.4 s and 8.1 s. Deployed in Taobao Live's digital-avatar service, it also yields positive online A/B test results for GMV and item-page views.