一致性驱动的协同演化用于自监督跨表示学习

摘要

随着图表图像、表格数据和可视化代码在众多领域中扮演越来越重要的角色，跨这些模态的跨表示理解对人工智能系统构成了根本性挑战：表示之间的关系本质上是一对多的，监督信号模糊且代价高昂，模型优化也缺乏一种既有方向适应性又具备表示泛化性的原则性信号，且该信号应超越特定任务目标。我们提出CoCoEvolve来提升图表、表格和代码表示之间的一致性。我们并未将跨表示映射视为一对多问题，而是定义明确的一一对应关系，并利用表示之间的一致性来优化模型，而无需额外标注。在训练阶段，CoCoEvolve@Train在图表-表格-代码循环中进行协同进化；CoCoEvolve@Test则在推理时应用相同的一致性目标，以实现测试时协同优化。我们还提出了CoCoEvolve@Eval，这是一个涵盖全部六项跨表示任务的评测套件。在四个基准上，CoCoEvolve在训练时和测试时设置中均提升了性能。我们的项目主页：https://xhguo7.github.io/CoCoEvolve/。

English

As chart images, tabular data, and visualization code play increasingly important roles across diverse domains, cross-representation understanding across these modalities poses fundamental challenges for AI systems: the relationships across representations are inherently one-to-many, supervision is ambiguous and costly, and model optimization lacks a principled signal that is both direction-adaptive and representation-generalizable beyond task-specific objectives. We introduce CoCoEvolve to improve consistency across chart, table, and code representations. Instead of treating cross-representation mapping as a one-to-many problem, we define explicit one-to-one correspondences and optimize models using agreement between representations, without additional annotations. During training, CoCoEvolve@Train performs co-evolution across the chart-table-code cycle, while CoCoEvolve@Test applies the same consistency objective at inference time for test-time co-optimization. We also present CoCoEvolve@Eval, an evaluation suite covering all six cross-representation tasks. Across four benchmarks, CoCoEvolve improves performance in both training-time and test-time settings. Our project page: https://xhguo7.github.io/CoCoEvolve/.