超越独立同分布：表格基础模型究竟有多通用？

摘要

近年来，面向表格数据的预测性机器学习基础模型已在学术界和工业界引起广泛关注。各学科研究群体正日益在多样化的数据集和任务上对这些表格基础模型进行评估。然而，由于基准测试软件与评估协议分散割裂，这些针对特定任务和学科开展的评估对模型研究人员而言仍难以触及。因此，模型研究者只能依赖标准基准测试，而这些基准测试大多针对表格基础模型已表现优异的任务设定，导致最具挑战性的场景被排除在外。这使得领域进步局限于在独立同分布数据上追求边际改进，而无法聚焦更广泛、更艰巨的挑战。为突破这一瓶颈，我们提出BeyondArena——首个针对表格数据的统一全局基准测试，它支持多样化的任务类型（独立同分布、时序、分组），覆盖不同样本量与特征维度规模，包含来自广泛学科领域的多种特征类型（含文本特征、高基数特征）。为实现超越标准基准的统一评测，我们引入Data Foundry——一个用于整理预测性机器学习表格数据集的Python框架与元数据模式。基于11个模型与142个整理数据集的结果显示，现有表格基础模型在小型至中型独立同分布数据上表现优异，而传统基于树的模型与深度学习模型仍在非独立同分布、大规模及高维数据集上占据主导地位。BeyondArena引导模型研究聚焦表格数据中最具挑战性的难题，推动真正基础性表格模型的进步。

English

Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry. Research communities across disciplines are increasingly evaluating tabular foundation models on diverse datasets and tasks. However, these task- and discipline-specific evaluations remain largely inaccessible to model researchers because benchmark software and evaluation protocols are fragmented. As a result, model researchers rely on standard benchmarks, which are mostly defined for tasks where tabular foundation models already excel. The most challenging scenarios are excluded, limiting meaningful progress in the field by focusing on marginal improvements on IID data rather than on broader, more demanding challenges. To overcome this, we introduce BeyondArena, the first unified holistic benchmark for tabular data that supports diverse task types (IID, temporal, grouped), across sample size and feature dimensionality scales, with diverse feature types (with text, with high cardinality) from a broad range of disciplines. To enable unified benchmarking beyond standard benchmarks, we introduce Data Foundry, a Python framework and metadata schema for curating tabular datasets for predictive machine learning. Our results across 11 models and 142 curated datasets show that existing tabular foundation models excel on tiny- to medium-sized IID data, while traditional tree-based and deep learning models still dominate on non-IID, large, and high-dimensional datasets. BeyondArena guides model research for the most demanding challenges in tabular data, enabling progress towards truly foundational tabular models.