Shepherd：語言模型生成的評論者

摘要

隨著大型語言模型的不斷改進，人們對利用這些模型的能力來完善其輸出的技術越來越感興趣。在這項工作中，我們介紹了Shepherd，這是一個專門調校以批評回應並提出改進建議的語言模型，擴展了未調校模型的能力，能夠識別各種錯誤並提供改正建議。我們方法的核心是一個高質量的反饋數據集，我們從社區反饋和人類標註中精心策劃。儘管Shepherd規模較小（7B參數），其批評要麼與ChatGPT等已建立模型的批評相當，要麼更受歡迎。在使用GPT-4進行評估時，Shepherd在與競爭對手相比的平均勝率達到53-87%。在人類評估中，Shepherd嚴格優於其他模型，並且平均與ChatGPT持平。

English

As large language models improve, there is increasing interest in techniques that leverage these models' capabilities to refine their own outputs. In this work, we introduce Shepherd, a language model specifically tuned to critique responses and suggest refinements, extending beyond the capabilities of an untuned model to identify diverse errors and provide suggestions to remedy them. At the core of our approach is a high quality feedback dataset, which we curate from community feedback and human annotations. Even though Shepherd is small (7B parameters), its critiques are either equivalent or preferred to those from established models including ChatGPT. Using GPT-4 for evaluation, Shepherd reaches an average win-rate of 53-87% compared to competitive alternatives. In human evaluation, Shepherd strictly outperforms other models and on average closely ties with ChatGPT.

Shepherd：語言模型生成的評論者

Shepherd: A Critic for Language Model Generation

摘要

Support