使用開源模型為使用者偏好生成系統訊息

摘要

系統訊息在與大型語言模型（LLMs）互動中扮演著至關重要的角色，通常作為啟動對話的提示。透過系統訊息，使用者可以指定特定角色、執行預期任務、整合背景資訊、指定各種輸出格式和溝通風格。儘管具有如此多樣性，公開可用的數據通常缺乏系統訊息，並受到行業領域嚴格的許可限制。將公開可用的數據手動標記上符合使用者指示的系統訊息需要大量資源。鑑於這些挑戰，我們的工作引入了SysGen，一個從帶有系統訊息的監督微調數據集中生成更好對齊助理回應的流程。在SysGen數據上進行訓練已經顯示出模型回應與系統訊息和使用者指示之間對齊程度的顯著提升，這在Multifacet基準測試中的各種開源模型上得到了證明，同時對其他未見基準測試（如Open LLM Leaderboard 2）的影響最小。我們的定性分析凸顯了多樣的系統訊息對確保在不同情境下更好地適應的重要性。

English

System messages play a crucial role in interactions with large language models (LLMs), often serving as prompts to initiate conversations. Through system messages, users can assign specific roles, perform intended tasks, incorporate background information, specify various output formats and communication styles. Despite such versatility, publicly available data are often lack system messages and subject to strict license constraints in the industry field. Manual labeling of publicly available data with system messages that align with user instructions demands significant resources. In view of such challenges, our work introduces SysGen, a pipeline for generating system messages with better aligned assistant responses from the supervised fine-tuning dataset without system messages. Training on SysGen data has demonstrated substantial improvements in the alignment of model responses with system messages and user instructions, as demonstrated across various open-source models on the Multifacet benchmark, while maintaining minimal impact on other unseen benchmarks such as Open LLM Leaderboard 2. Our qualitative analysis highlights the importance of diverse system messages to ensure better adaptability across different contexts.

使用開源模型為使用者偏好生成系統訊息

System Message Generation for User Preferences using Open-Source Models

摘要

Support