Real-TurnTurk：一個用於話輪轉換預測的多模態土耳其語語料庫

摘要

話輪轉換是人類對話的基本組織特徵，且在自然、同步的對話系統中仍難以建模。儘管現有研究已探索用於話輪結束預測的多模態方法與大型語言模型，但針對土耳其語話輪轉換動態的自然對話語料庫仍然缺乏。本研究提出一個多模態土耳其語對話資料集，內容為非腳本的雙人互動，包含同步的正面視訊、可將重疊語音歸屬至個別說話者的分軌音訊，以及時間對齊的轉寫。話輪轉換預測被表述為二元分類問題，並採用基因演算法（GA）來最佳化從視覺、聲學及語言特徵推導出的可解釋決策規則。在所提出的框架中，採用混合 AND-OR 規則表示法，以表示觸發話輪轉換之前的替代性線索組合。

English

Turn-taking is a basic organizational feature of human conversation and remains difficult to model in natural, synchronous dialog systems. While existing research has explored multimodal approaches and large language models for turn-ending prediction, there is a lack of naturalistic conversational corpora specifically addressing turn-taking dynamics in Turkish. This study introduces a multimodal Turkish conversational dataset of unscripted dyadic interactions, comprising synchronized front-facing video, per-speaker audio channels that allow overlapping speech to be attributed to individual speakers, and time-aligned transcriptions. Turn-taking prediction is formulated as a binary classification problem, and a Genetic Algorithm (GA) is employed to optimize interpretable decision rules derived from visual, acoustic, and linguistic features. A hybrid AND-OR rule representation is adopted in the proposed framework to represent the alternative cue combinations that precede a turn transition.