Gen2Det：生成以檢測

Gen2Det: Generate to Detect

December 7, 2023

作者: Saksham Suri, Fanyi Xiao, Animesh Sinha, Sean Chang Culatana, Raghuraman Krishnamoorthi, Chenchen Zhu, Abhinav Shrivastava

cs.AI

摘要

最近擴散模型在合成圖像質量和生成控制方面均有所改善。我們提出了Gen2Det，這是一個簡單的模塊化流程，通過利用最先進的基於圖像生成的方法，免費創建用於物體檢測的合成訓練數據。與現有方法不同，這些方法生成單個物體實例，需要識別前景，然後將其貼在其他圖像上，我們簡化為直接生成以場景為中心的圖像。除了合成數據外，Gen2Det還提出了一套技術，以最佳方式利用生成的數據，包括圖像級過濾、實例級過濾以及更好的訓練配方，以應對生成過程中的不完美之處。使用Gen2Det，我們展示了在各種設置下對物體檢測和分割任務的顯著改進，並且不受檢測方法的限制。在LVIS的長尾檢測設置中，Gen2Det大幅提高了罕見類別的性能，同時還顯著提高了其他類別的性能，例如，相對於僅在LVIS上使用Mask R-CNN的真實數據進行訓練，我們看到Box AP提高了2.13，Mask AP提高了1.84。在COCO的低數據範疇設置中，Gen2Det持續提高了Box和Mask AP，分別提高了2.27和1.85個點。在最一般的檢測設置中，Gen2Det仍然展示出穩健的性能增益，例如，它提高了COCO上的Box和Mask AP分別為0.45和0.32個點。

English

Recently diffusion models have shown improvement in synthetic image quality as well as better control in generation. We motivate and present Gen2Det, a simple modular pipeline to create synthetic training data for object detection for free by leveraging state-of-the-art grounded image generation methods. Unlike existing works which generate individual object instances, require identifying foreground followed by pasting on other images, we simplify to directly generating scene-centric images. In addition to the synthetic data, Gen2Det also proposes a suite of techniques to best utilize the generated data, including image-level filtering, instance-level filtering, and better training recipe to account for imperfections in the generation. Using Gen2Det, we show healthy improvements on object detection and segmentation tasks under various settings and agnostic to detection methods. In the long-tailed detection setting on LVIS, Gen2Det improves the performance on rare categories by a large margin while also significantly improving the performance on other categories, e.g. we see an improvement of 2.13 Box AP and 1.84 Mask AP over just training on real data on LVIS with Mask R-CNN. In the low-data regime setting on COCO, Gen2Det consistently improves both Box and Mask AP by 2.27 and 1.85 points. In the most general detection setting, Gen2Det still demonstrates robust performance gains, e.g. it improves the Box and Mask AP on COCO by 0.45 and 0.32 points.