嵌入者的困境：大型語言模型更優越，但代價為何？

摘要

您應該用大型語言模型取代文本嵌入管道嗎？我們以受控且考量成本的比較來回答這個問題，比較對象涵蓋六個家族的十個大型語言模型，以及二十六個嵌入模型（參數量從1.18億到140億），並在三十七項任務上進行評估，涵蓋分類、語意文本相似度（STS）、聚類、配對分類與檢索。整體而言，兩種範式幾乎打成平手：最佳大型語言模型（Gemini 3.1 Pro，77.6）與最佳嵌入模型（77.2）僅相差0.4分。兩者的強項因任務而異：大型語言模型在推理密集的檢索上領先，嵌入模型在分類上領先，而兩者在聚類、STS與配對分類上表現相當。然而要達到此等水準的平手代價高昂。大型語言模型的成本可高達品質相近之嵌入模型的1,431倍（每次基準測試通過花費154美元對比0.11美元），而在相同GPU上，所測試的開放大型語言模型處理token的速度慢了2.5至736倍。推理token佔大型語言模型推論成本的28%至81%；在我們的消融實驗中，對多數模型而言，較低的推理預算可維持或甚至提升檢索品質。帕雷托前緣包含領先的嵌入模型以及一個大型語言模型，即Gemini 3.1 Pro。這些結果支持分工的做法：相似度、分類與聚類任務使用嵌入模型，而將大型語言模型保留給推理密集的檢索。我們的程式碼、資料集與結果公開於 https://github.com/embeddings-benchmark/embedders-dilemma。

English

Should you replace your text-embedding pipeline with a large language model? We answer this with a controlled, cost-aware comparison of ten LLMs across six families and 26 embedding models (118M to 14B parameters) on 37 tasks spanning classification, semantic textual similarity (STS), clustering, pair classification, and retrieval. In aggregate the two paradigms are effectively tied: the best LLM (Gemini 3.1 Pro, 77.6) and the best embedding model (77.2) differ by 0.4 points. Their strengths differ by task: LLMs lead on reasoning-heavy retrieval, embedding models lead on classification, and the two match on clustering, STS, and pair classification. Reaching that parity is expensive. An LLM costs up to 1,431x more than an embedding model of comparable quality (USD 154 vs. USD 0.11 per benchmark pass), and the open LLMs tested process tokens 2.5 to 736x more slowly on the same GPU. Reasoning tokens account for 28 to 81% of LLM inference cost; lower reasoning budgets preserve or improve retrieval quality for most models in our ablation. The Pareto frontier contains the leading embedding models and one LLM, Gemini 3.1 Pro. These results support a division of labour: use embedding models for similarity, classification, and clustering, and reserve LLMs for reasoning-intensive retrieval. Our code, datasets, and results are publicly available at https://github.com/embeddings-benchmark/embedders-dilemma.