Goku：百萬規模的通用資料集與基於指令的影片編輯基準

摘要

現有的基於指令的影片編輯資料集通常專注於單一任務的外觀編輯，無法滿足真實場景中複雜的創意需求。為填補此差距，我們提出Goku，一個大型資料集，包含200萬個高品質、與指令對齊的影片編輯對，這是首個將任務邊界從基本外觀編輯擴展到多任務與結構性操作（例如精確控制主體運動）的資料集。為應對這些複雜任務中固有的資料合成挑戰，我們設計了一個高效的資料合成管線，將複雜編輯分解為可控的子問題，並在整個過程中引入漸進式過濾系統以確保資料可靠性。此外，我們在Goku上探索了最優網路結構，並提出了Goku-Edit。為深入理解複雜編輯指令，Goku-Edit採用多模態大語言模型（MLLM）作為其文字編碼器，並採用解耦的雙分支設計：一個專用遮罩分支處理結構控制，釋放主分支專注於外觀渲染。同時提出了一個全面的影片編輯基準Goku-Bench，包含1,000個人類驗證的測試案例與7項新穎的編輯專用指標。在Goku-Bench上評估，Goku-Edit在指令遵循方面相比其他開源模型獲得了高達8%的提升。

English

Existing instruction-based video editing datasets commonly focus on single-task appearance editing, failing to meet the complex creative demands of real-world scenarios. To bridge this gap, we present Goku, a large-scale dataset featuring 2 million high-quality, instruction-aligned video editing pairs, which is the first to extend task boundaries from basic appearance editing to multi-task and structural manipulations(e.g., precise control of subject movement). To tackle the data synthesis challenges inherent in these complex tasks, we design an efficient data synthesis pipeline that decomposes complex edits into controllable sub-problems and introduce a progressive filtering system for data reliability throughout the whole process. Furthermore, we explore the optimal network structures on Goku, and propose Goku-Edit. To deeply comprehend complex editing instructions, Goku-Edit leverages an MLLM as its text encoder and adopts a decoupled dual-branch design: a dedicated mask branch handles structural control, freeing the main branch for appearance rendering. A comprehensive video editing benchmark, Goku-Bench, is also proposed with 1,000 human-verified test cases and 7 novel editing-specific metrics. Evaluated on Goku-Bench, Goku-Edit obtains up to +8% improvement on other open-source models in terms of instruction following.