超越獨立同分布假設：表格基礎模型究竟具有多大的通用性？

摘要

針對表格型數據的預測性機器學習基礎模型，近來在學術界與業界備受關注。各領域研究社群正逐步在多樣資料集與任務上評估這些表格基礎模型。然而，這些任務與學科特定的評估結果，對模型研究人員而言仍難以取得，因為基準測試軟體與評估協定各自分散。因此，模型研究人員只能依賴標準基準測試，而這些測試大多定義在表格基礎模型已表現優異的任務上。最具挑戰性的場景因此被排除，使得該領域的進展侷限於在獨立同分佈（IID）數據上追求邊際改進，而非面對更廣泛、更嚴峻的挑戰。為解決此問題，我們提出 **BeyondArena**，首個全面統一的表格型數據基準測試框架，支援多種任務類型（IID、時序、分組），涵蓋樣本數與特徵維度尺度，並納入來自廣泛學科的多樣化特徵類型（包含文字、高基數特徵）。為實現超越標準基準測試的統一評估，我們推出 **Data Foundry**，一個用於整理預測性機器學習表格資料集的 Python 框架與元數據架構。我們在 11 個模型與 142 個整理後的資料集上得到的結果顯示，現有表格基礎模型在小型至中型 IID 數據上表現卓越，而傳統樹型模型與深度學習模型仍在非 IID、大型及高維度資料集上佔據主導地位。BeyondArena 引領模型研究聚焦於表格數據中最具挑戰性的問題，推動朝向真正基礎性表格模型的進展。

English

Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry. Research communities across disciplines are increasingly evaluating tabular foundation models on diverse datasets and tasks. However, these task- and discipline-specific evaluations remain largely inaccessible to model researchers because benchmark software and evaluation protocols are fragmented. As a result, model researchers rely on standard benchmarks, which are mostly defined for tasks where tabular foundation models already excel. The most challenging scenarios are excluded, limiting meaningful progress in the field by focusing on marginal improvements on IID data rather than on broader, more demanding challenges. To overcome this, we introduce BeyondArena, the first unified holistic benchmark for tabular data that supports diverse task types (IID, temporal, grouped), across sample size and feature dimensionality scales, with diverse feature types (with text, with high cardinality) from a broad range of disciplines. To enable unified benchmarking beyond standard benchmarks, we introduce Data Foundry, a Python framework and metadata schema for curating tabular datasets for predictive machine learning. Our results across 11 models and 142 curated datasets show that existing tabular foundation models excel on tiny- to medium-sized IID data, while traditional tree-based and deep learning models still dominate on non-IID, large, and high-dimensional datasets. BeyondArena guides model research for the most demanding challenges in tabular data, enabling progress towards truly foundational tabular models.