Synth-SONAR：通过双扩散模型和GPT提示实现具有增强多样性和逼真性的声纳图像合成

Synth-SONAR: Sonar Image Synthesis with Enhanced Diversity and Realism via Dual Diffusion Models and GPT Prompting

October 11, 2024

作者: Purushothaman Natarajan, Kamal Basha, Athira Nambiar

cs.AI

摘要

声纳图像合成对推动水下探测、海洋生物学和国防等领域的应用至关重要。传统方法通常依赖于使用声纳传感器进行大量昂贵的数据收集，危及数据质量和多样性。为了克服这些限制，本研究提出了一种新的声纳图像合成框架，名为Synth-SONAR，利用扩散模型和GPT提示。Synth-SONAR的三个关键创新点是：首先，通过将基于生成式人工智能的样式注入技术与公开可用的真实/模拟数据相结合，从而为声纳研究产生了最大的声纳数据语料库之一。其次，双文本调节声纳扩散模型层次结构合成粗粒度和细粒度声纳图像，提高了质量和多样性。第三，基于文本的声纳生成方法分为高级（粗略）和低级（详细）两种，利用视觉语言模型（VLMs）和GPT提示中可用的先进语义信息。在推断过程中，该方法从文本提示生成多样且逼真的声纳图像，弥合了文本描述与声纳图像生成之间的差距。据我们所知，这是首次在声纳图像领域应用GPT提示。Synth-SONAR在生成高质量合成声纳数据集方面取得了最新成果，显著增强了数据集的多样性和逼真性。

English

Sonar image synthesis is crucial for advancing applications in underwater exploration, marine biology, and defence. Traditional methods often rely on extensive and costly data collection using sonar sensors, jeopardizing data quality and diversity. To overcome these limitations, this study proposes a new sonar image synthesis framework, Synth-SONAR leveraging diffusion models and GPT prompting. The key novelties of Synth-SONAR are threefold: First, by integrating Generative AI-based style injection techniques along with publicly available real/simulated data, thereby producing one of the largest sonar data corpus for sonar research. Second, a dual text-conditioning sonar diffusion model hierarchy synthesizes coarse and fine-grained sonar images with enhanced quality and diversity. Third, high-level (coarse) and low-level (detailed) text-based sonar generation methods leverage advanced semantic information available in visual language models (VLMs) and GPT-prompting. During inference, the method generates diverse and realistic sonar images from textual prompts, bridging the gap between textual descriptions and sonar image generation. This marks the application of GPT-prompting in sonar imagery for the first time, to the best of our knowledge. Synth-SONAR achieves state-of-the-art results in producing high-quality synthetic sonar datasets, significantly enhancing their diversity and realism.