SymbolicAI：一个结合生成模型和求解器的基于逻辑的方法的框架

SymbolicAI: A framework for logic-based approaches combining generative models and solvers

February 1, 2024

作者: Marius-Constantin Dinu, Claudiu Leoveanu-Condrei, Markus Holzleitner, Werner Zellinger, Sepp Hochreiter

cs.AI

摘要

我们介绍了SymbolicAI，这是一个多功能且模块化的框架，采用基于逻辑的方法来进行概念学习和流程管理，用于生成过程。SymbolicAI通过将大型语言模型（LLMs）视为语义解析器，执行基于自然语言和形式语言指令的任务，从而实现了生成模型与各种求解器的无缝集成，从而弥合了符号推理与生成人工智能之间的差距。我们利用概率编程原则来解决复杂任务，并利用可微分和经典编程范式及其各自的优势。该框架引入了一组多态的、组合的、自引用的操作，用于数据流操作，将LLM输出与用户目标对齐。因此，我们可以在各种基础模型之间进行转换，这些模型具有零次和少次学习能力，以及专门的、经过精细调整的模型或求解器，擅长解决特定问题。反过来，该框架促进了可解释计算图的创建和评估。最后，我们介绍了一种用于评估这些计算图的质量度量及其经验分数，并提出了一个基准，用于比较各种最先进的LLMs在一组复杂工作流中的表现。我们将这种经验分数称为“通过交叉相似性进行关系轨迹评估的向量嵌入”，简称为VERTEX分数。下方链接了该框架的代码库和基准。

English

We introduce SymbolicAI, a versatile and modular framework employing a logic-based approach to concept learning and flow management in generative processes. SymbolicAI enables the seamless integration of generative models with a diverse range of solvers by treating large language models (LLMs) as semantic parsers that execute tasks based on both natural and formal language instructions, thus bridging the gap between symbolic reasoning and generative AI. We leverage probabilistic programming principles to tackle complex tasks, and utilize differentiable and classical programming paradigms with their respective strengths. The framework introduces a set of polymorphic, compositional, and self-referential operations for data stream manipulation, aligning LLM outputs with user objectives. As a result, we can transition between the capabilities of various foundation models endowed with zero- and few-shot learning capabilities and specialized, fine-tuned models or solvers proficient in addressing specific problems. In turn, the framework facilitates the creation and evaluation of explainable computational graphs. We conclude by introducing a quality measure and its empirical score for evaluating these computational graphs, and propose a benchmark that compares various state-of-the-art LLMs across a set of complex workflows. We refer to the empirical score as the "Vector Embedding for Relational Trajectory Evaluation through Cross-similarity", or VERTEX score for short. The framework codebase and benchmark are linked below.