Real-TurnTurk: 用于话轮转换预测的多模态土耳其语语料库

摘要

话轮转换是人类对话的基本组织特征，在自然的同步对话系统中仍然难以建模。尽管已有研究探索了用于话轮结束预测的多模态方法和大语言模型，但目前缺乏专门针对土耳其语话轮转换动态的自然会话语料库。本研究引入了一个多模态土耳其语会话数据集，包含无脚本的双人互动，由同步的正面摄像头视频、可将重叠语音归因于单个说话人的每说话人独立音频通道，以及时间对齐的转录文本组成。话轮转换预测被表述为一个二分类问题，并采用遗传算法（GA）来优化从视觉、声学和语言特征中推导出的可解释决策规则。所提出的框架采用了混合AND-OR规则表示，以表示话轮转换之前出现的多种替代性线索组合。

English

Turn-taking is a basic organizational feature of human conversation and remains difficult to model in natural, synchronous dialog systems. While existing research has explored multimodal approaches and large language models for turn-ending prediction, there is a lack of naturalistic conversational corpora specifically addressing turn-taking dynamics in Turkish. This study introduces a multimodal Turkish conversational dataset of unscripted dyadic interactions, comprising synchronized front-facing video, per-speaker audio channels that allow overlapping speech to be attributed to individual speakers, and time-aligned transcriptions. Turn-taking prediction is formulated as a binary classification problem, and a Genetic Algorithm (GA) is employed to optimize interpretable decision rules derived from visual, acoustic, and linguistic features. A hybrid AND-OR rule representation is adopted in the proposed framework to represent the alternative cue combinations that precede a turn transition.