SymbolicAI：一個結合生成模型和求解器的基於邏輯方法的框架

SymbolicAI: A framework for logic-based approaches combining generative models and solvers

February 1, 2024

作者: Marius-Constantin Dinu, Claudiu Leoveanu-Condrei, Markus Holzleitner, Werner Zellinger, Sepp Hochreiter

cs.AI

摘要

我們介紹了SymbolicAI，這是一個多功能且模塊化的框架，採用基於邏輯的方法來進行概念學習和流程管理在生成過程中。SymbolicAI通過將大型語言模型（LLMs）視為語義解析器，執行基於自然和形式語言指令的任務，從而實現生成模型與各種求解器的無縫集成，從而彌合符號推理和生成式人工智能之間的差距。我們利用概率編程原則來應對複雜任務，並利用可微分和傳統編程範式及其各自的優勢。該框架引入了一組多態、組合和自指操作，用於數據流操作，將LLM輸出與用戶目標對齊。因此，我們可以在各種具有零次和少次學習能力的基礎模型之間切換，以及專門的、經過精心調整的模型或求解器，能夠擅長解決特定問題。反過來，該框架促進了可解釋計算圖的創建和評估。最後，我們介紹了一個用於評估這些計算圖的質量度量及其實證分數，並提出了一個基準，比較各種最先進的LLMs在一組複雜工作流中的表現。我們將這個實證分數稱為“通過交叉相似性進行關係軌跡評估的向量嵌入”，簡稱VERTEX分數。該框架的代碼庫和基準測試已在下方鏈接。

English

We introduce SymbolicAI, a versatile and modular framework employing a logic-based approach to concept learning and flow management in generative processes. SymbolicAI enables the seamless integration of generative models with a diverse range of solvers by treating large language models (LLMs) as semantic parsers that execute tasks based on both natural and formal language instructions, thus bridging the gap between symbolic reasoning and generative AI. We leverage probabilistic programming principles to tackle complex tasks, and utilize differentiable and classical programming paradigms with their respective strengths. The framework introduces a set of polymorphic, compositional, and self-referential operations for data stream manipulation, aligning LLM outputs with user objectives. As a result, we can transition between the capabilities of various foundation models endowed with zero- and few-shot learning capabilities and specialized, fine-tuned models or solvers proficient in addressing specific problems. In turn, the framework facilitates the creation and evaluation of explainable computational graphs. We conclude by introducing a quality measure and its empirical score for evaluating these computational graphs, and propose a benchmark that compares various state-of-the-art LLMs across a set of complex workflows. We refer to the empirical score as the "Vector Embedding for Relational Trajectory Evaluation through Cross-similarity", or VERTEX score for short. The framework codebase and benchmark are linked below.