語言模型組合何時有幫助？路由、投票與混合智能體的共同失敗上限：基於67個前沿模型的研究

摘要

多模型LLM系統如路由、投票、級聯、融合與混合智能體，常被用來超越單一模型的準確率。我們發現，這類系統的增益存在一個上界，而這個數值極少被學界報告。對於任何最終輸出為某個成員模型答案的策略，其準確率不可能超過一減去β，其中β是每個模型對同一查詢都犯錯的比率。相比之下，常規診斷指標——平均成對誤差相關性ρ——無法識別β：具有相同邊際分佈與成對相關性的誤差律，可能對應不同的全錯率。對β應用克洛珀-皮爾森（Clopper-Pearson）上界，可在訓練路由之前，為任何路由、投票或級聯系統能達到的最大增益提供有限樣本認證。在來自21家供應商的67個模型中，經四格相關校正的單因子模型仍然低估了全錯尾部：在開放式數學問題上，觀測到的β為0.052，而完整67模型高斯連接函數下的β為0.023，低估約2.5倍，90%信賴區間為1.7至3.4，k值為17。此效應在執行評分的程式碼問題中重現，β值為0.079。將同樣的GPQA-Diamond問題以自由回答而非選擇題形式重新提問，會重新打開尾部，β值為0.127，由五名評判組成的評審小組的κ值為0.73至0.92，表明共同失敗源於回答格式而非學科本身。在品質匹配的情況下，低ρ異質集成優於高ρ的Self-MoA，但在我們測試庫中可檢查的任務上，若缺乏強查詢級路由訊號，組合模型鮮少能超越單一最佳模型。增益來自於模型在不同問題上失敗，而非單純增加模型數量。

English

Multi-model LLM systems such as routing, voting, cascades, fusion, and mixture-of-agents are used to beat single-model accuracy. We show that their gain is capped by a quantity the field rarely reports. For any policy whose output is one member model answer, accuracy cannot exceed one minus beta, where beta is the rate at which every model is wrong on the same query. In contrast, the usual diagnostic, average pairwise error correlation rho, cannot identify beta: error laws with identical marginals and pairwise correlations can have different all-wrong rates. A Clopper-Pearson bound on beta gives a finite-sample certificate on the largest gain any router, vote, or cascade could deliver before training a router. Across 67 models from 21 providers, a tetrachoric-calibrated single-factor model still underprices the all-wrong tail: on open-ended mathematics, observed beta is 0.052 versus 0.023 under the full 67-model Gaussian copula, about 2.5 times underpricing, with 90 percent CI 1.7 to 3.4 and k equals 17. The effect recurs on execution-graded code, where beta is 0.079. Re-asking the same GPQA-Diamond questions in free-response rather than multiple-choice form reopens the tail, with beta 0.127 and a five-judge panel with kappa 0.73 to 0.92, locating co-failure in answer format rather than subject. At matched quality, low-rho heterogeneous ensembles beat high-rho Self-MoA, but on checkable tasks in our pool, combining models rarely beats the single best model without a strong query-level routing signal. Gains come from models failing on different questions, not from adding more models.