Gen2Det：生成以检测

Gen2Det: Generate to Detect

December 7, 2023

作者: Saksham Suri, Fanyi Xiao, Animesh Sinha, Sean Chang Culatana, Raghuraman Krishnamoorthi, Chenchen Zhu, Abhinav Shrivastava

cs.AI

摘要

最近，扩散模型在合成图像质量和生成控制方面显示出了改进。我们提出并展示了Gen2Det，这是一个简单的模块化流程，通过利用最先进的基于图像生成的方法，免费创建用于目标检测的合成训练数据。与现有的生成单个物体实例、需要识别前景然后粘贴到其他图像上的方法不同，我们简化为直接生成以场景为中心的图像。除了合成数据，Gen2Det还提出了一套技术，以最佳方式利用生成的数据，包括图像级过滤、实例级过滤以及更好的训练方案，以应对生成过程中的缺陷。利用Gen2Det，我们展示了在各种设置下以及不考虑检测方法的目标检测和分割任务中的显著改进。在LVIS的长尾检测设置中，Gen2Det显著提高了稀有类别的性能，同时还显著提高了其他类别的性能，例如，相较于仅在LVIS上使用Mask R-CNN训练的真实数据，我们看到Box AP提高了2.13个百分点，Mask AP提高了1.84个百分点。在COCO的低数据量设置中，Gen2Det始终将Box AP和Mask AP分别提高了2.27和1.85个百分点。在最一般的检测设置中，Gen2Det仍然展示出稳健的性能增益，例如，它将COCO上的Box AP和Mask AP分别提高了0.45和0.32个百分点。

English

Recently diffusion models have shown improvement in synthetic image quality as well as better control in generation. We motivate and present Gen2Det, a simple modular pipeline to create synthetic training data for object detection for free by leveraging state-of-the-art grounded image generation methods. Unlike existing works which generate individual object instances, require identifying foreground followed by pasting on other images, we simplify to directly generating scene-centric images. In addition to the synthetic data, Gen2Det also proposes a suite of techniques to best utilize the generated data, including image-level filtering, instance-level filtering, and better training recipe to account for imperfections in the generation. Using Gen2Det, we show healthy improvements on object detection and segmentation tasks under various settings and agnostic to detection methods. In the long-tailed detection setting on LVIS, Gen2Det improves the performance on rare categories by a large margin while also significantly improving the performance on other categories, e.g. we see an improvement of 2.13 Box AP and 1.84 Mask AP over just training on real data on LVIS with Mask R-CNN. In the low-data regime setting on COCO, Gen2Det consistently improves both Box and Mask AP by 2.27 and 1.85 points. In the most general detection setting, Gen2Det still demonstrates robust performance gains, e.g. it improves the Box and Mask AP on COCO by 0.45 and 0.32 points.