CaRGo-T：因果推理思維圖譜提升多模態幽默理解

摘要

大型視覺語言模型（VLMs）已在廣泛的多模態任務中展現出卓越的多樣性。然而，理解幽默仍具挑戰性，因為幽默內容往往依賴於實體、事件、情境以及跨影像與文字模態之間的隱性互動。這些互動可能涉及複雜的推理鏈，難以透過傳統提示或線性思維鏈推理來捕捉。在本工作中，我們提出CaRGo-T（Causal Reasoning Graph-of-Thought，因果推理思維圖譜），這是一個推理框架，將多模態幽默背後的因果與情境關係表示為輕量級的圖形化推理結構。該圖結構被序列化為由VLM生成的程式碼式表徵，隨後可由相同或不同的VLM進行解譯，以在零樣本或上下文學習設定中產生最終預測。我們在四個涵蓋多種喜劇內容形式（包括諷刺、譏諷與迷因）的資料集上評估CaRGo-T的幽默理解與幽默偵測表現。實驗採用最先進的商業與開源VLM，結果顯示CaRGo-T consistently 優於現有的推理式基線方法，在幽默理解上提升約1–20%，在幽默偵測上提升約1–3%。進一步的互信息分析顯示，CaRGo-T產生的推理表徵比基線推理方法所生成者包含更多與目標輸出相關的資訊。程式碼可於 https://github.com/abhi1nandy2/CaRGo-T 取得。

English

Large-scale vision-language models (VLMs) have demonstrated remarkable versatility across a wide range of multimodal tasks. However, understanding humor remains challenging because humorous content often depends on subtle interactions among entities, events, context, and implicit relationships across image and text modalities. These interactions can involve complex chains of reasoning that are difficult to capture through conventional prompting or linear chain-of-thought reasoning. In this work, we propose CaRGo-T (Causal Reasoning Graph-of-Thought), a reasoning framework that represents the causal and contextual relationships underlying multimodal humor as a lightweight graph-based reasoning structure. The graph is serialized into a code-based representation generated by a VLM, which can subsequently be interpreted by the same or a different VLM to produce the final prediction in zero-shot or in-context learning settings. We evaluate CaRGo-T on humor understanding and humor detection across four datasets spanning diverse forms of comedic content, including satire, sarcasm, and memes. Experiments with state-of-the-art commercial and open-source VLMs show that CaRGo-T consistently improves performance over existing reasoning-based baselines, achieving gains of approximately 1-20% on humor understanding and 1-3% on humor detection. Further analysis using mutual information indicates that the reasoning representations produced by CaRGo-T contain more information relevant to the target output than those generated by baseline reasoning approaches. Code is available at https://github.com/abhi1nandy2/CaRGo-T.