探討量化方法對大型語言模型安全性與可靠性的影響

摘要

大型語言模型（LLMs）已成為應對現代挑戰並實現實際應用的強大工具。然而，其高昂的計算成本仍是廣泛採用的主要障礙。量化技術作為一種有前景的方法，旨在普及這些模型的應用並支持低資源設備的部署。儘管取得了這些進展，量化模型的安全性和可信度仍未被充分探討，因為先前的研究往往忽視了當代架構，並依賴過於簡化的基準和評估方法。為填補這一空白，我們引入了OpenSafetyMini，這是一個新穎的開放式安全數據集，旨在更好地區分不同模型。我們使用四個基準（包括人工評估）對LLaMA和Mistral模型上的四種最先進的量化技術進行了評估。我們的研究結果表明，在4位精度下，最佳量化方法因模型而異，而在2位精度下，向量量化技術提供了最佳的安全性和可信度表現，為未來研究奠定了基礎。

English

Large Language Models (LLMs) have emerged as powerful tools for addressing modern challenges and enabling practical applications. However, their computational expense remains a significant barrier to widespread adoption. Quantization has emerged as a promising technique to democratize access and enable low resource device deployment. Despite these advancements, the safety and trustworthiness of quantized models remain underexplored, as prior studies often overlook contemporary architectures and rely on overly simplistic benchmarks and evaluations. To address this gap, we introduce OpenSafetyMini, a novel open-ended safety dataset designed to better distinguish between models. We evaluate 4 state-of-the-art quantization techniques across LLaMA and Mistral models using 4 benchmarks, including human evaluations. Our findings reveal that the optimal quantization method varies for 4-bit precision, while vector quantization techniques deliver the best safety and trustworthiness performance at 2-bit precision, providing foundation for future research.

探討量化方法對大型語言模型安全性與可靠性的影響

Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models

摘要

Support