與多個語言模型協作學習解碼

摘要

我們提出了一種方法，通過在標記級別交替生成，教導多個大型語言模型（LLM）進行協作。我們將決定哪個LLM生成下一個標記的過程建模為潛在變量。通過在我們的潛在變量模型下優化訓練集的邊際概率，基本LLM自動學習何時生成自身以及何時呼叫其中一個“助手”語言模型進行生成，而無需直接監督。在解碼期間進行標記級別的協作允許以符合特定任務的方式融合每個模型的專業知識。我們的協作解碼在跨領域設置中特別有用，其中一個通用基礎LLM學習調用領域專家模型。在遵循指示、特定領域的問答和推理任務中，我們展示聯合系統的性能優於個別模型。通過對學習的潛在決策進行定性分析，我們展示用我們的方法訓練的模型表現出幾種有趣的協作模式，例如模板填充。我們的代碼可在https://github.com/clinicalml/co-llm找到。

English

We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent variable. By optimizing the marginal likelihood of a training set under our latent variable model, the base LLM automatically learns when to generate itself and when to call on one of the ``assistant'' language models to generate, all without direct supervision. Token-level collaboration during decoding allows for a fusion of each model's expertise in a manner tailored to the specific task at hand. Our collaborative decoding is especially useful in cross-domain settings where a generalist base LLM learns to invoke domain expert models. On instruction-following, domain-specific QA, and reasoning tasks, we show that the performance of the joint system exceeds that of the individual models. Through qualitative analysis of the learned latent decisions, we show models trained with our method exhibit several interesting collaboration patterns, e.g., template-filling. Our code is available at https://github.com/clinicalml/co-llm.

與多個語言模型協作學習解碼

Learning to Decode Collaboratively with Multiple Language Models

摘要

Support