擴展擴散和基於流的 XGBoost 模型

摘要

對於表格式數據生成，新穎的機器學習方法通常在規模不符合科學應用所需的小數據集上開發。我們研究了最近提出的在表格式數據上使用 XGBoost 作為擴散和流匹配模型中的函數逼近器的方法，即使在微小數據集上也證明了極高的內存消耗。在這項工作中，我們從工程角度對現有實施進行了批判性分析，並表明這些限制並非是該方法的根本問題；通過更好的實施，它可以擴展到比以前使用的數據集大 370 倍的規模。我們的高效實施還可以將模型擴展到更大的尺寸，我們直接展示這將導致在基準任務上的性能改善。我們還提出了可以進一步改善資源使用和模型性能的算法改進，包括適合生成建模的多輸出樹。最後，我們展示了從實驗粒子物理學中衍生的大型科學數據集的結果，作為快速量能器模擬挑戰的一部分。代碼可在 https://github.com/layer6ai-labs/calo-forest 找到。

English

Novel machine learning methods for tabular data generation are often developed on small datasets which do not match the scale required for scientific applications. We investigate a recent proposal to use XGBoost as the function approximator in diffusion and flow-matching models on tabular data, which proved to be extremely memory intensive, even on tiny datasets. In this work, we conduct a critical analysis of the existing implementation from an engineering perspective, and show that these limitations are not fundamental to the method; with better implementation it can be scaled to datasets 370x larger than previously used. Our efficient implementation also unlocks scaling models to much larger sizes which we show directly leads to improved performance on benchmark tasks. We also propose algorithmic improvements that can further benefit resource usage and model performance, including multi-output trees which are well-suited to generative modeling. Finally, we present results on large-scale scientific datasets derived from experimental particle physics as part of the Fast Calorimeter Simulation Challenge. Code is available at https://github.com/layer6ai-labs/calo-forest.

擴展擴散和基於流的 XGBoost 模型

Scaling Up Diffusion and Flow-based XGBoost Models

摘要

Support