Boolformer:使用Transformer进行逻辑函数的符号回归
Boolformer: Symbolic Regression of Logic Functions with Transformers
September 21, 2023
作者: Stéphane d'Ascoli, Samy Bengio, Josh Susskind, Emmanuel Abbé
cs.AI
摘要
在这项工作中,我们介绍了Boolformer,这是第一个经过训练以执行端到端布尔函数符号回归的Transformer架构。首先,我们展示了当提供一个干净的真值表时,它能够预测复杂函数的紧凑公式,即使这些函数在训练过程中没有见过。然后,我们展示了当提供不完整和带噪声的观测时,它能够找到近似表达式的能力。我们在一系列真实世界的二元分类数据集上评估了Boolformer,展示了它作为经典机器学习方法的可解释替代方案的潜力。最后,我们将其应用于对基因调控网络动态建模的广泛任务。使用最近的基准测试,我们展示了Boolformer在速度上比当前最先进的遗传算法具有竞争力,并且速度提升了数个数量级。我们的代码和模型已公开可用。
English
In this work, we introduce Boolformer, the first Transformer architecture
trained to perform end-to-end symbolic regression of Boolean functions. First,
we show that it can predict compact formulas for complex functions which were
not seen during training, when provided a clean truth table. Then, we
demonstrate its ability to find approximate expressions when provided
incomplete and noisy observations. We evaluate the Boolformer on a broad set of
real-world binary classification datasets, demonstrating its potential as an
interpretable alternative to classic machine learning methods. Finally, we
apply it to the widespread task of modelling the dynamics of gene regulatory
networks. Using a recent benchmark, we show that Boolformer is competitive with
state-of-the art genetic algorithms with a speedup of several orders of
magnitude. Our code and models are available publicly.