CLI-Universe：面向終端智能體的可驗證任務合成引擎

摘要

尽管近期基于大型语言模型的终端代理展现出令人振奋的能力，但高质量、可执行训练数据的匮乏仍是关键瓶颈。现有合成流程通常通过将表层人工产物重构为任务来实现规模扩展，这往往导致指令模糊、执行路径浅显、测试用例脆弱，难以提供有效的学习信号。为突破这一局限，我们提出CLI-Universe——一个构建终端代理任务的原理性合成引擎。该引擎通过在多维能力分类体系（涵盖领域、技能类型、能力维度与工程支柱）中采样组合生成候选任务，随后基于引导式证据对真实世界技术资料进行深度研究，将每个候选任务落地为可执行方案。为确保严格的监督信号，验证通过的任务蓝图会被实例化为Docker化环境，并经过多阶段可执行验证流程：该流程包含基于评估准则的测试用例构建、提示条件过滤以及严格的失败转通过校验。从候选生成到验证的完整流程中，约三分之二的候选任务被淘汰，仅保留那些真实、可验证且具备非平凡挑战性的任务。为验证框架有效性，我们实例化了一个高度精简的数据集CLI-Universe-6K，包含6000条轨迹。值得关注的是，在CLI-Universe-6K上微调Qwen3-32B模型，在Terminal-Bench 2.0基准上达到了33.4%的准确率。这不仅在基于开源数据训练的32B及以下参数模型中树立了新标杆，更超越多个参数规模大一个数量级的模型，充分展现了结构化高保真合成策略在数据效率上的显著优势。

English

While recent LLM-based terminal agents have demonstrated promising capabilities, the scarcity of high-quality, executable training data remains a critical bottleneck. Existing synthesis pipelines typically scale by retrofitting surface-level artifacts into tasks, frequently yielding ambiguous instructions, shallow execution paths, and brittle tests that provide weak learning signals. To overcome this, we introduce CLI-Universe, a principled synthesis engine that constructs terminal-agent tasks. CLI-Universe generates candidate tasks by sampling combinations across a multi-dimensional capability taxonomy (domain, skill type, capability, and engineering pillar), then grounds each candidate through evidence-guided deep research over real-world technical materials. To ensure rigorous supervision, validated blueprints are instantiated into Dockerized environments and subjected to a multi-stage executable verification pipeline featuring rubric-gated test construction, hint-conditional filtering, and strict fail-to-pass checking. Across the full pipeline, from candidate generation to verification, approximately two-thirds of candidates are discarded, retaining only those that are genuine, verifiable, and non-trivially challenging. To validate our framework, we instantiate a highly distilled dataset of 6,000 trajectories called CLI-Universe-6K. Remarkably, fine-tuning Qwen3-32B on CLI-Universe-6K achieves 33.4% on Terminal-Bench 2.0. This sets a new state-of-the-art for models trained on open-source data at or below 32B parameters, and outperforms several models an order of magnitude larger, demonstrating the profound data efficiency of structured, high-fidelity synthesis.