融合就是你所需要的一切：比兆參數LLM更便宜、更好的替代方案

摘要

在對話式人工智慧研究中，有一個明顯的趨勢是朝著開發具有更多參數的模型發展，這一趨勢以ChatGPT等模型為例。雖然這些龐大的模型往往能夠生成越來越好的對話回應，但它們需要大量的計算資源和記憶體。這項研究探討了一個相關問題：一組較小模型的組合是否可以協作達到與單一大型模型相當或更好的性能？我們提出了一種稱為「混合」的方法，這是一種簡單而有效的方法，用於整合多個對話式人工智慧。我們的實證證據表明，當特定的較小模型協同混合時，它們有可能超越或匹敵遠大於它們的對應大型模型的能力。例如，僅集成三個中等大小的模型（6B/13B參數）就可以與大型模型ChatGPT（175B+參數）的性能指標相匹敵甚至超越。這一假設是通過在Chai研究平台上對擁有龐大用戶基礎的A/B測試方法在三十天內進行嚴格測試的。研究結果強調了「混合」策略作為一種可行方法，可提升對話式人工智慧的效能，而無需相應地增加計算需求。

English

In conversational AI research, there's a noticeable trend towards developing models with a larger number of parameters, exemplified by models like ChatGPT. While these expansive models tend to generate increasingly better chat responses, they demand significant computational resources and memory. This study explores a pertinent question: Can a combination of smaller models collaboratively achieve comparable or enhanced performance relative to a singular large model? We introduce an approach termed "blending", a straightforward yet effective method of integrating multiple chat AIs. Our empirical evidence suggests that when specific smaller models are synergistically blended, they can potentially outperform or match the capabilities of much larger counterparts. For instance, integrating just three models of moderate size (6B/13B paramaeters) can rival or even surpass the performance metrics of a substantially larger model like ChatGPT (175B+ paramaters). This hypothesis is rigorously tested using A/B testing methodologies with a large user base on the Chai research platform over a span of thirty days. The findings underscore the potential of the "blending" strategy as a viable approach for enhancing chat AI efficacy without a corresponding surge in computational demands.

融合就是你所需要的一切：比兆參數LLM更便宜、更好的替代方案

Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM

摘要

Support