早期退出與即時置信度翻譯品質評估

Early-Exit and Instant Confidence Translation Quality Estimation

February 20, 2025

作者: Vilém Zouhar, Maike Züfle, Beni Egressy, Julius Cheng, Jan Niehues

cs.AI

摘要

品質評估在機器翻譯中無處不在，無論是對於評估還是生成皆然。然而，品質評估模型往往既晦澀難懂又計算成本高昂，這使得它們難以融入大規模的處理流程中。在本研究中，我們著手解決兩個相互關聯的挑戰：(1) 降低大規模品質評估的成本，以及 (2) 開發一種成本低廉的品質評估不確定性估計方法。針對後者，我們引入了即時信心COMET，這是一個具備不確定性感知能力的品質評估模型，它以極低的成本達到了先前方法的性能水平。我們進一步將其擴展為早期退出COMET，這是一種能在模型早期層次就計算品質分數及相關置信度的品質評估模型，從而允許我們提前終止計算，降低評估成本。我們還將此模型應用於機器翻譯的重排序任務中。通過將早期退出COMET與上置信界帶狀算法結合，我們能夠在不對所有候選者運行完整評估模型的情況下，從大量候選中找出最佳選項。無論是在評估還是重排序的場景下，我們的方法都將所需的計算量減少了50%，而性能僅有極小幅度的下降。

English

Quality estimation is omnipresent in machine translation, for both evaluation and generation. Unfortunately, quality estimation models are often opaque and computationally expensive, making them impractical to be part of large-scale pipelines. In this work, we tackle two connected challenges: (1) reducing the cost of quality estimation at scale, and (2) developing an inexpensive uncertainty estimation method for quality estimation. To address the latter, we introduce Instant Confidence COMET, an uncertainty-aware quality estimation model that matches the performance of previous approaches at a fraction of their costs. We extend this to Early-Exit COMET, a quality estimation model that can compute quality scores and associated confidences already at early model layers, allowing us to early-exit computations and reduce evaluation costs. We also apply our model to machine translation reranking. We combine Early-Exit COMET with an upper confidence bound bandit algorithm to find the best candidate from a large pool without having to run the full evaluation model on all candidates. In both cases (evaluation and reranking) our methods reduce the required compute by 50% with very little degradation in performance.