RuleChef：将LLM任务知识扎根于人类可编辑的规则

摘要

我們提出 RuleChef，一個利用大型語言模型（LLMs）為 NLP 任務（如文本分類、命名實體辨識（NER）或關係抽取）生成可執行規則的框架。這些規則根據任務描述和一組標註範例生成，接著根據更多範例以及對現有規則的人工回饋進行迭代改進。RuleChef 也可用於引導規則的建立：透過觀察任何現有模型在給定任務上的輸入-輸出對來初始化規則。LLMs 僅在學習階段使用，負責合成規則，並根據在留出驗證集上測得的失敗情況進行迭代修補。此流程的產物是一套快速、確定性且可檢視的規則系統。初步評估在分類與 NER 任務上進行。我們以 Apache 2.0 授權釋出 RuleChef 為開源軟體。

English

We present RuleChef, a framework that uses large language models (LLMs) to generate executable rules for NLP tasks such as text classification, Named Entity Recognition (NER), or relation extraction. Rules are generated based on a task description and a set of labeled examples, then they are iteratively improved based both on additional examples and on human feedback overexisting rules. RuleChef can also be used to bootstrap rules using the observed input-output pairs from any existing model for a given task. LLMs are used only at learning time, synthesizing rules and iteratively patching them based on failures measured on a held-out split. The result of this process is a fast, deterministic, and inspectable rule system. Preliminary evaluation is performed on both classification and NER tasks. We release RuleChef as open-source software under an Apache 2.0