識別偽造：基於大型多模態模型的合成影像檢測與偽影解析

Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanation

March 19, 2025

作者: Siwei Wen, Junyan Ye, Peilin Feng, Hengrui Kang, Zichen Wen, Yize Chen, Jiang Wu, Wenjun Wu, Conghui He, Weijia Li

cs.AI

摘要

隨著人工智慧生成內容（AIGC）技術的快速發展，合成圖像在日常生活中日益普遍，這為真實性評估與檢測帶來了新的挑戰。儘管現有方法在評估圖像真實性和定位偽造方面頗具成效，但這些方法往往缺乏人類可解釋性，且未能完全應對合成數據日益增長的複雜性。為應對這些挑戰，我們推出了FakeVLM，這是一款專為通用合成圖像及深度偽造檢測任務設計的大型多模態模型。FakeVLM不僅在區分真實與偽造圖像方面表現卓越，還能提供清晰、自然的語言解釋來描述圖像偽造痕跡，從而增強了可解釋性。此外，我們還推出了FakeClue，這是一個包含超過10萬張圖像的綜合數據集，涵蓋七個類別，並以自然語言標註了細粒度的偽造線索。FakeVLM在性能上可與專家模型相媲美，同時無需額外的分類器，使其成為合成數據檢測的強大解決方案。在多個數據集上的廣泛評估證實了FakeVLM在真實性分類和偽造痕跡解釋任務中的優越性，為合成圖像檢測樹立了新標杆。數據集和代碼將發佈於：https://github.com/opendatalab/FakeVLM。

English

With the rapid advancement of Artificial Intelligence Generated Content (AIGC) technologies, synthetic images have become increasingly prevalent in everyday life, posing new challenges for authenticity assessment and detection. Despite the effectiveness of existing methods in evaluating image authenticity and locating forgeries, these approaches often lack human interpretability and do not fully address the growing complexity of synthetic data. To tackle these challenges, we introduce FakeVLM, a specialized large multimodal model designed for both general synthetic image and DeepFake detection tasks. FakeVLM not only excels in distinguishing real from fake images but also provides clear, natural language explanations for image artifacts, enhancing interpretability. Additionally, we present FakeClue, a comprehensive dataset containing over 100,000 images across seven categories, annotated with fine-grained artifact clues in natural language. FakeVLM demonstrates performance comparable to expert models while eliminating the need for additional classifiers, making it a robust solution for synthetic data detection. Extensive evaluations across multiple datasets confirm the superiority of FakeVLM in both authenticity classification and artifact explanation tasks, setting a new benchmark for synthetic image detection. The dataset and code will be released in: https://github.com/opendatalab/FakeVLM.