設計一個儀表板，用於透明度和對話式人工智慧的控制。

摘要

對話式LLMs 作為黑盒系統運作，讓使用者猜測他們看到特定輸出的原因。這種缺乏透明度可能存在問題，尤其在關注偏見和真實性的情況下。為了解決這個問題，我們提出了一個端對端原型，將可解釋性技術與使用者體驗設計相結合，旨在使聊天機器人更加透明。我們首先展示了一個知名的開源LLM具有“使用者模型”的證據：通過檢查系統的內部狀態，我們可以提取與使用者年齡、性別、教育水平和社會經濟地位相關的數據。接下來，我們描述了一個附帶在聊天機器人界面中的儀表板的設計，實時顯示這個使用者模型。該儀表板還可用於控制使用者模型和系統行為。最後，我們討論了一項研究，其中使用者與該儀器化系統進行了對話。我們的結果表明，使用者喜歡看到內部狀態，這有助於他們揭露偏見行為並增加他們的控制感。參與者還提出了寶貴的建議，指出了未來設計和機器學習研究的方向。我們的TalkTuner系統的項目頁面和視頻演示可在https://bit.ly/talktuner-project-page 上找到。

English

Conversational LLMs function as black box systems, leaving users guessing about why they see the output they do. This lack of transparency is potentially problematic, especially given concerns around bias and truthfulness. To address this issue, we present an end-to-end prototype-connecting interpretability techniques with user experience design-that seeks to make chatbots more transparent. We begin by showing evidence that a prominent open-source LLM has a "user model": examining the internal state of the system, we can extract data related to a user's age, gender, educational level, and socioeconomic status. Next, we describe the design of a dashboard that accompanies the chatbot interface, displaying this user model in real time. The dashboard can also be used to control the user model and the system's behavior. Finally, we discuss a study in which users conversed with the instrumented system. Our results suggest that users appreciate seeing internal states, which helped them expose biased behavior and increased their sense of control. Participants also made valuable suggestions that point to future directions for both design and machine learning research. The project page and video demo of our TalkTuner system are available at https://bit.ly/talktuner-project-page

設計一個儀表板，用於透明度和對話式人工智慧的控制。

Designing a Dashboard for Transparency and Control of Conversational AI

摘要

Support