揭示原生统一多模态模型中理解与生成的协同效应：从表征、任务到系统

摘要

雖然統一多模態模型（UMMs）能在單一模型內同時執行視覺理解與生成任務，但功能上的統一並不保證學習上的綜效：這兩種目標可能相互增強、競爭容量，或僅是並存。我們在一個受控、結構原生且無預訓練視覺先驗的設定中，從表徵、任務與系統三個層面探討兩者之間的關係。在表徵層面，我們發現每個目標都能為另一個目標提供有用的訊號：生成任務豐富了用於理解任務的視覺特徵，而理解任務則強化了生成任務所需的視覺－語言對齊。然而，當兩種目標被強迫經由同一條計算路徑時，往往會有一方主導另一方。一種將衝突的視覺計算加以專業化、同時保留語義互動的任務解耦架構，能夠避免這種不對稱退化。在任務層面，透過三個案例研究，我們發現當理解與生成任務依賴共享知識時，會產生正向的雙向遷移。在系統層面，我們證明在明確需要同時具備影像理解與生成能力的複雜任務上，端到端UMM的表現優於條件相當的規劃器－執行器流程。綜合以上結果，我們顯示UMM的價值不僅止於統一的介面：適當的專業分工、共享的任務知識以及端到端最佳化，能將並存轉化為綜效。

English

While unified multimodal models (UMMs) jointly perform visual understanding and generation within a single model, functional unification does not guarantee learning synergy: the two objectives may reinforce each other, compete for capacity, or merely coexist. We investigate their relationship at the representation, task, and system levels in a controlled, structurally native setting without pretrained vision priors. At the representation level, we find that each objective provides useful signal to the other: generation enriches the visual features learned for understanding, while understanding strengthens vision--language alignment for generation. However, when both objectives are forced through the same computation path, one tends to dominate. A task-decoupled architecture that specializes conflicting visual computation while preserving semantic interaction avoids this asymmetric degradation. At the task level, through three case studies, we find positive bidirectional transfer when understanding and generation tasks rely on shared knowledge. At the system level, we show that an end-to-end UMM outperforms a matched planner--executor pipeline on complex tasks that explicitly require both image understanding and generation. Together, these results show that the value of UMMs extends beyond a unified interface: appropriate specialization, shared task knowledge, and end-to-end optimization can turn coexistence into synergy.