GPT-4o在图像复原领域的初步研究

摘要

OpenAI的GPT-4o模型，在自回归架构中整合了多模态输入与输出，已在图像生成领域展现出前所未有的性能。本研究探讨了其对图像修复领域的潜在影响，并首次系统性地评估了GPT-4o在多种修复任务中的表现。实验表明，尽管GPT-4o生成的修复图像在视觉上颇具吸引力，但与真实图像相比，常存在像素级结构保真度不足的问题，如图像比例变化、物体位置与数量偏移以及视角改变等。针对这些问题，我们以图像去雾、去雨及低光增强为例，展示了GPT-4o输出作为强大视觉先验的潜力，显著提升了现有去雾网络的性能。本研究提供了实用指南及基础框架，旨在促进GPT-4o融入未来图像修复流程。我们期望对GPT-4o图像修复的研究能加速图像生成领域的创新进程。为支持进一步研究，我们将公开GPT-4o修复的来自10余个广泛使用的图像修复数据集的图像。

English

OpenAI's GPT-4o model, integrating multi-modal inputs and outputs within an autoregressive architecture, has demonstrated unprecedented performance in image generation. In this work, we investigate its potential impact on the image restoration community. We present the first systematic evaluation of GPT-4o across diverse restoration tasks. Our experiments reveal that, although restoration outputs from GPT-4o are visually appealing, they often suffer from pixel-level structural fidelity when compared to ground-truth images. Common issues are variations in image proportions, shifts in object positions and quantities, and changes in viewpoint.To address it, taking image dehazing, derainning, and low-light enhancement as representative case studies, we show that GPT-4o's outputs can serve as powerful visual priors, substantially enhancing the performance of existing dehazing networks. It offers practical guidelines and a baseline framework to facilitate the integration of GPT-4o into future image restoration pipelines. We hope the study on GPT-4o image restoration will accelerate innovation in the broader field of image generation areas. To support further research, we will release GPT-4o-restored images from over 10 widely used image restoration datasets.

GPT-4o在图像复原领域的初步研究

A Preliminary Study for GPT-4o on Image Restoration

摘要

Summary

Support

Support