Shepherd：一种用于语言模型生成的评论者

摘要

随着大型语言模型的不断改进，人们对利用这些模型能力来改进其输出的技术越来越感兴趣。在这项工作中，我们介绍了Shepherd，这是一个专门调整以批判回应并提出改进建议的语言模型，扩展了未调整模型的能力，能够识别各种错误并提供改正建议。我们方法的核心是一个高质量的反馈数据集，我们从社区反馈和人工注释中精心筛选而来。尽管Shepherd规模较小（70亿参数），但其批评要么与ChatGPT等已建立模型的批评相当，要么更受青睐。在使用GPT-4进行评估时，Shepherd在与竞争对手的比较中达到了平均胜率为53-87%。在人类评估中，Shepherd严格优于其他模型，并且平均与ChatGPT持平。

English

As large language models improve, there is increasing interest in techniques that leverage these models' capabilities to refine their own outputs. In this work, we introduce Shepherd, a language model specifically tuned to critique responses and suggest refinements, extending beyond the capabilities of an untuned model to identify diverse errors and provide suggestions to remedy them. At the core of our approach is a high quality feedback dataset, which we curate from community feedback and human annotations. Even though Shepherd is small (7B parameters), its critiques are either equivalent or preferred to those from established models including ChatGPT. Using GPT-4 for evaluation, Shepherd reaches an average win-rate of 53-87% compared to competitive alternatives. In human evaluation, Shepherd strictly outperforms other models and on average closely ties with ChatGPT.

Shepherd：一种用于语言模型生成的评论者

Shepherd: A Critic for Language Model Generation

摘要

Support