让仿生人梦见电子羊：一个类人的图像隐含意义理解与推理框架

Let Androids Dream of Electric Sheep: A Human-like Image Implication Understanding and Reasoning Framework

May 22, 2025

作者: Chenhao Zhang, Yazhe Niu

cs.AI

摘要

图像中的隐喻理解仍是AI系统面临的关键挑战，现有模型难以把握视觉内容中蕴含的微妙文化、情感及语境含义。尽管多模态大语言模型（MLLMs）在基础视觉问答（VQA）任务上表现出色，但在图像隐含意义任务上存在根本性局限：语境缺失导致不同视觉元素与其抽象意义间的关系模糊不清。受人类认知过程启发，我们提出了“让安卓梦游”（LAD）这一新颖框架，用于图像隐含意义的理解与推理。LAD通过三阶段框架解决语境缺失问题：（1）感知：将视觉信息转化为多层次丰富的文本表示；（2）搜索：迭代搜索并整合跨领域知识以消除歧义；（3）推理：通过显式推理生成与语境对齐的图像隐含意义。我们的框架结合轻量级GPT-4o-mini模型，在英文图像隐含意义基准测试中相比15+个MLLMs达到SOTA性能，并在中文基准测试上取得显著提升，在多项选择题（MCQ）上与GPT-4o模型表现相当，在开放式问题（OSQ）上超出36.7%。此外，我们的工作为AI如何更有效解读图像隐含意义提供了新见解，推动了视觉语言推理与人机交互领域的发展。项目已公开于https://github.com/MING-ZCH/Let-Androids-Dream-of-Electric-Sheep。

English

Metaphorical comprehension in images remains a critical challenge for AI systems, as existing models struggle to grasp the nuanced cultural, emotional, and contextual implications embedded in visual content. While multimodal large language models (MLLMs) excel in basic Visual Question Answer (VQA) tasks, they struggle with a fundamental limitation on image implication tasks: contextual gaps that obscure the relationships between different visual elements and their abstract meanings. Inspired by the human cognitive process, we propose Let Androids Dream (LAD), a novel framework for image implication understanding and reasoning. LAD addresses contextual missing through the three-stage framework: (1) Perception: converting visual information into rich and multi-level textual representations, (2) Search: iteratively searching and integrating cross-domain knowledge to resolve ambiguity, and (3) Reasoning: generating context-alignment image implication via explicit reasoning. Our framework with the lightweight GPT-4o-mini model achieves SOTA performance compared to 15+ MLLMs on English image implication benchmark and a huge improvement on Chinese benchmark, performing comparable with the GPT-4o model on Multiple-Choice Question (MCQ) and outperforms 36.7% on Open-Style Question (OSQ). Additionally, our work provides new insights into how AI can more effectively interpret image implications, advancing the field of vision-language reasoning and human-AI interaction. Our project is publicly available at https://github.com/MING-ZCH/Let-Androids-Dream-of-Electric-Sheep.