Synth-SONAR：透過雙擴散模型和GPT提示增強多樣性和真實感的聲納影像合成

Synth-SONAR: Sonar Image Synthesis with Enhanced Diversity and Realism via Dual Diffusion Models and GPT Prompting

October 11, 2024

作者: Purushothaman Natarajan, Kamal Basha, Athira Nambiar

cs.AI

摘要

Sonar圖像合成對於推動水下探索、海洋生物學和防禦應用至關重要。傳統方法通常依賴於使用聲納傳感器進行廣泛且昂貴的數據收集，危及數據質量和多樣性。為了克服這些限制，本研究提出了一種新的sonar圖像合成框架，Synth-SONAR，利用擴散模型和GPT提示。Synth-SONAR的三個關鍵創新點如下：首先，通過將生成式人工智慧風格注入技術與公開可用的真實/模擬數據相結合，從而為sonar研究生成了最大的sonar數據庫之一。其次，雙文本條件sonar擴散模型層次結構合成了粗粒和細粒度的sonar圖像，提高了質量和多樣性。第三，高層（粗糙）和低層（詳細）基於文本的sonar生成方法利用了視覺語言模型（VLMs）和GPT提示中可用的先進語義信息。在推理過程中，該方法從文本提示中生成多樣且逼真的sonar圖像，彌合了文本描述與sonar圖像生成之間的差距。據我們所知，這是首次將GPT提示應用於sonar圖像。Synth-SONAR在生成高質量合成sonar數據集方面取得了最新成果，顯著增強了數據集的多樣性和逼真性。

English

Sonar image synthesis is crucial for advancing applications in underwater exploration, marine biology, and defence. Traditional methods often rely on extensive and costly data collection using sonar sensors, jeopardizing data quality and diversity. To overcome these limitations, this study proposes a new sonar image synthesis framework, Synth-SONAR leveraging diffusion models and GPT prompting. The key novelties of Synth-SONAR are threefold: First, by integrating Generative AI-based style injection techniques along with publicly available real/simulated data, thereby producing one of the largest sonar data corpus for sonar research. Second, a dual text-conditioning sonar diffusion model hierarchy synthesizes coarse and fine-grained sonar images with enhanced quality and diversity. Third, high-level (coarse) and low-level (detailed) text-based sonar generation methods leverage advanced semantic information available in visual language models (VLMs) and GPT-prompting. During inference, the method generates diverse and realistic sonar images from textual prompts, bridging the gap between textual descriptions and sonar image generation. This marks the application of GPT-prompting in sonar imagery for the first time, to the best of our knowledge. Synth-SONAR achieves state-of-the-art results in producing high-quality synthetic sonar datasets, significantly enhancing their diversity and realism.