用於自監督跨表徵學習的一致性驅動共同演化

摘要

隨著圖表圖像、表格數據與可視化代碼在諸多領域中扮演日益重要的角色，跨這些模態的跨表示理解對人工智慧系統構成了根本性的挑戰：表示之間的關係本質上是一對多的，監督訊號具有模糊性且成本高昂，而模型最佳化缺乏一個既具方向自適應性、又能超越任務特定目標而實現表示泛化的原則性訊號。我們提出CoCoEvolve以提升圖表、表格與代碼表示之間的一致性。我們不將跨表示映射視為一對多問題，而是定義明確的一對一對應關係，並利用表示之間的一致性來最佳化模型，無需額外的標註。在訓練期間，CoCoEvolve@Train在圖表-表格-代碼循環中進行共同演化，而CoCoEvolve@Test則在推論時應用相同的一致性目標，實現測試時共同最佳化。我們亦提出CoCoEvolve@Eval，一個涵蓋全部六種跨表示任務的評估套件。在四個基準測試中，CoCoEvolve在訓練時與測試時兩種設定下均提升了效能。我們的專案頁面：https://xhguo7.github.io/CoCoEvolve/。

English

As chart images, tabular data, and visualization code play increasingly important roles across diverse domains, cross-representation understanding across these modalities poses fundamental challenges for AI systems: the relationships across representations are inherently one-to-many, supervision is ambiguous and costly, and model optimization lacks a principled signal that is both direction-adaptive and representation-generalizable beyond task-specific objectives. We introduce CoCoEvolve to improve consistency across chart, table, and code representations. Instead of treating cross-representation mapping as a one-to-many problem, we define explicit one-to-one correspondences and optimize models using agreement between representations, without additional annotations. During training, CoCoEvolve@Train performs co-evolution across the chart-table-code cycle, while CoCoEvolve@Test applies the same consistency objective at inference time for test-time co-optimization. We also present CoCoEvolve@Eval, an evaluation suite covering all six cross-representation tasks. Across four benchmarks, CoCoEvolve improves performance in both training-time and test-time settings. Our project page: https://xhguo7.github.io/CoCoEvolve/.