RelBench：關聯式數據庫上深度學習的基準測試

RelBench: A Benchmark for Deep Learning on Relational Databases

July 29, 2024

作者: Joshua Robinson, Rishabh Ranjan, Weihua Hu, Kexin Huang, Jiaqi Han, Alejandro Dobles, Matthias Fey, Jan E. Lenssen, Yiwen Yuan, Zecheng Zhang, Xinwei He, Jure Leskovec

cs.AI

摘要

我們介紹了 RelBench，這是一個用於利用圖神經網絡解決關聯式數據庫中預測任務的公共基準。RelBench提供了跨越不同領域和規模的數據庫和任務，旨在成為未來研究的基礎基礎設施。我們使用RelBench進行了對關聯式深度學習（RDL）（Fey等，2024年）的第一次全面研究，該研究將圖神經網絡預測模型與（深度）表格模型結合起來，從原始表格中提取初始實體級表示。端到端學習的RDL模型充分利用了主外鍵鏈中編碼的預測信號，標誌著從主導範式手動特徵工程結合表格模型的重大轉變。為了徹底評估RDL與這個先前的黃金標準，我們進行了一項深入的用戶研究，其中一位經驗豐富的數據科學家為每個任務手動工程化特徵。在這項研究中，RDL學習到了更好的模型，同時將人類工作量減少了一個數量級以上。這展示了深度學習在解決關聯式數據庫中的預測任務方面的威力，為通過RelBench實現的許多新研究機會打開了大門。

English

We present RelBench, a public benchmark for solving predictive tasks over relational databases with graph neural networks. RelBench provides databases and tasks spanning diverse domains and scales, and is intended to be a foundational infrastructure for future research. We use RelBench to conduct the first comprehensive study of Relational Deep Learning (RDL) (Fey et al., 2024), which combines graph neural network predictive models with (deep) tabular models that extract initial entity-level representations from raw tables. End-to-end learned RDL models fully exploit the predictive signal encoded in primary-foreign key links, marking a significant shift away from the dominant paradigm of manual feature engineering combined with tabular models. To thoroughly evaluate RDL against this prior gold-standard, we conduct an in-depth user study where an experienced data scientist manually engineers features for each task. In this study, RDL learns better models whilst reducing human work needed by more than an order of magnitude. This demonstrates the power of deep learning for solving predictive tasks over relational databases, opening up many new research opportunities enabled by RelBench.