系統級自然語言回饋

摘要

自然語言（NL）反饋包含了豐富的用戶體驗信息。現有研究主要關注於實例級別方法，其中反饋用於完善特定示例，忽略了其在系統範圍應用上的潛力。本文提出了一個通用框架，用於開啟 NL 反饋的系統級應用。我們展示了如何利用反饋來形式化人在迴圈過程中的系統級設計決策，以便生成更好的模型。具體而言，這是通過：（i）任務的度量設計；和（ii）語言模型提示設計以完善模型回應。我們進行了兩個案例研究，以改進搜索查詢生成和對話回應生成，展示了系統級反饋的有效性。我們表明系統級反饋和實例級反饋的結合帶來進一步的收益，而人類編寫的實例級反饋比 GPT-3.5 編寫的反饋產生更具基礎性的改進，突顯了人類反饋在構建系統時的重要性。

English

Natural language (NL) feedback contains rich information about the user experience. Existing studies focus on an instance-level approach, where feedback is used to refine specific examples, disregarding its system-wide application. This paper proposes a general framework for unlocking the system-level use of NL feedback. We show how to use feedback to formalize system-level design decisions in a human-in-the-loop-process -- in order to produce better models. In particular this is done through: (i) metric design for tasks; and (ii) language model prompt design for refining model responses. We conduct two case studies of this approach for improving search query generation and dialog response generation, demonstrating the effectiveness of the use of system-level feedback. We show the combination of system-level feedback and instance-level feedback brings further gains, and that human written instance-level feedback results in more grounded refinements than GPT-3.5 written ones, underlying the importance of human feedback for building systems.

系統級自然語言回饋

System-Level Natural Language Feedback

摘要

Support