TLive-Omni：面向电商直播的全模态理解模型

摘要

电商直播要求对嘈杂、时间上延展的流媒体进行全模态理解，其中商品事实分布在语音、视频帧、商品图片、叠加文本和用户查询中。我们提出TLive-Omni，一种面向直播电商场景的全模态理解模型。它将图像、视频、音频和文本输入映射到统一的表示空间中。针对长时直播流分析，我们引入了Per-vGrid，一种带时间戳的令牌组织形式，将每个视频网格与其时间上对应的音频分组在显式边界令牌内，以促进时间对齐。我们设计了一种三阶段监督训练方案，从全模态感知到指令跟随响应，逐步培养直播电商理解能力。随后我们提出Faithful-RFT，一种强化微调阶段，在满足实时性需求的同时进一步提升回答忠实度和表达质量，直接使用任务可验证的反馈对最终响应进行评分，而非在滚动生成过程中优化推理式探索。此外，TLive-Omni由面向场景的原子能力分类体系和紧凑的数据生产引擎支撑，该引擎将直播电商的音频、图像和视频流转换为语音识别、说话人分析、商品视觉定位、文本识别、时间定位、视频密集描述和全模态问答等任务的训练信号。为实现可扩展训练，同步长度分组采样器在保持各工作节点负载相当的同时减少填充，而轻量级动态采样策略以近乎零奖励方差重新生成滚动组，从而为GRPO保持有意义的相对优势。在电商直播基准上的实验表明，该模型在直播电商领域任务上表现强劲，同时在通用基准上展现出优异的泛化能力。

English

E-commerce live streaming requires omni-modal understanding of noisy, temporally extended streams, where product facts are distributed across speech, video frames, product images, overlaid text, and user queries. We present TLive-Omni, an omni-modal understanding model tailored to live-commerce scenarios. It maps image, video, audio, and text inputs into a unified representation space. For long-form live streaming analysis, we introduce Per-vGrid, a timestamped token organization that groups each video grid with its temporally corresponding audio within explicit boundary tokens to facilitate temporal alignment. We design a three-stage supervised training recipe that progressively develops live-commerce understanding, from omni-modal perception to instruction-following responses. We then propose Faithful-RFT, a reinforcement fine-tuning stage that further improves answer faithfulness and expression quality while meeting real-time demands, scoring final responses directly with task-verifiable feedback rather than optimizing for reasoning-style exploration during rollout. Moreover, TLive-Omni is supported by a scenario-oriented atomic capability taxonomy and a compact data production engine that converts live-commerce audio, image, and video streams into training signals for speech recognition, speaker analysis, product visual grounding, text recognition, temporal grounding, video dense caption, and omni-modal QA, etc. For scalable training, a synchronized length-grouped sampler reduces padding while preserving comparable workloads across workers, while a lightweight dynamic sampling strategy regenerates rollout groups with near-zero reward variance to maintain meaningful relative advantages for GRPO. Experiments on e-commerce live streaming benchmarks demonstrate strong performance across live-commerce domain tasks, together with excellent generalization on general benchmarks.