Afri-MCQA：面向非洲语言的多模态文化问答系统

摘要

非洲拥有全球超过三分之一的语言，却在人工智能研究中代表性不足。我们推出首个多文化问答基准数据集Afri-MCQA，涵盖12个国家的15种非洲语言，包含7500组平行问答对。该基准提供文本与语音双模态的英语-非洲语言平行问答对，全部由母语者创建。基于Afri-MCQA对大语言模型的测试表明，开源模型在不同文化场景中表现欠佳：当使用本土语言或语音进行开放式视觉问答时，准确率接近为零。为评估语言能力，我们设置了控制实验以区分文化知识与语言能力，观察到模型在非洲本土语言与英语的文本及语音处理上存在显著性能差距。这些发现揭示了采用语音优先策略、文化背景预训练及跨语言文化迁移的必要性。为支持非洲语言的多模态AI包容性发展，我们在HuggingFace平台以学术许可或CC BY-NC 4.0协议开源Afri-MCQA数据集（https://huggingface.co/datasets/Atnafu/Afri-MCQA）。

English

Africa is home to over one-third of the world's languages, yet remains underrepresented in AI research. We introduce Afri-MCQA, the first Multilingual Cultural Question-Answering benchmark covering 7.5k Q&A pairs across 15 African languages from 12 countries. The benchmark offers parallel English-African language Q&A pairs across text and speech modalities and was entirely created by native speakers. Benchmarking large language models (LLMs) on Afri-MCQA shows that open-weight models perform poorly across evaluated cultures, with near-zero accuracy on open-ended VQA when queried in native language or speech. To evaluate linguistic competence, we include control experiments meant to assess this specific aspect separate from cultural knowledge, and we observe significant performance gaps between native languages and English for both text and speech. These findings underscore the need for speech-first approaches, culturally grounded pretraining, and cross-lingual cultural transfer. To support more inclusive multimodal AI development in African languages, we release our Afri-MCQA under academic license or CC BY-NC 4.0 on HuggingFace (https://huggingface.co/datasets/Atnafu/Afri-MCQA)

Afri-MCQA：面向非洲语言的多模态文化问答系统

Afri-MCQA: Multimodal Cultural Question Answering for African Languages

摘要

Support