CaRGo-T：因果推理思维图提升多模态幽默理解

摘要

大规模视觉语言模型（VLMs）已在广泛的多模态任务中展现出卓越的多功能性。然而，理解幽默仍具挑战性，因为幽默内容往往依赖于跨图像和文本模态的实体、事件、上下文及隐含关系之间的细微交互。这些交互可能涉及复杂的推理链，难以通过常规提示或线性思维链推理加以捕获。在这项工作中，我们提出 CaRGo-T（Causal Reasoning Graph-of-Thought，因果推理思维图），一种将多模态幽默背后的因果与上下文关系表示为轻量级图结构的推理框架。该图被序列化为由 VLM 生成的基于代码的表示，随后可由相同或不同的 VLM 解释，以在零样本或上下文学习设置中产生最终预测。我们在涵盖讽刺、反讽和模因等多种喜剧形式的四个数据集上，对 CaRGo-T 进行了幽默理解与幽默检测的评估。基于最先进的商业及开源 VLM 的实验表明，CaRGo-T 持续优于现有的基于推理的基线方法，在幽默理解上取得约 1%-20% 的提升，在幽默检测上取得约 1%-3% 的提升。进一步的互信息分析表明，CaRGo-T 生成的推理表示比基线推理方法生成的表示包含更多与目标输出相关的信息。代码可在 https://github.com/abhi1nandy2/CaRGo-T 获取。

English

Large-scale vision-language models (VLMs) have demonstrated remarkable versatility across a wide range of multimodal tasks. However, understanding humor remains challenging because humorous content often depends on subtle interactions among entities, events, context, and implicit relationships across image and text modalities. These interactions can involve complex chains of reasoning that are difficult to capture through conventional prompting or linear chain-of-thought reasoning. In this work, we propose CaRGo-T (Causal Reasoning Graph-of-Thought), a reasoning framework that represents the causal and contextual relationships underlying multimodal humor as a lightweight graph-based reasoning structure. The graph is serialized into a code-based representation generated by a VLM, which can subsequently be interpreted by the same or a different VLM to produce the final prediction in zero-shot or in-context learning settings. We evaluate CaRGo-T on humor understanding and humor detection across four datasets spanning diverse forms of comedic content, including satire, sarcasm, and memes. Experiments with state-of-the-art commercial and open-source VLMs show that CaRGo-T consistently improves performance over existing reasoning-based baselines, achieving gains of approximately 1-20% on humor understanding and 1-3% on humor detection. Further analysis using mutual information indicates that the reasoning representations produced by CaRGo-T contain more information relevant to the target output than those generated by baseline reasoning approaches. Code is available at https://github.com/abhi1nandy2/CaRGo-T.