CatLIP：在Web规模的图像文本数据上进行2.7倍更快的预训练，实现与CLIP相当的视觉识别准确性

摘要

对比学习已成为通过对齐图像和文本嵌入来学习有效视觉表示的一种革命性方法。然而，在图像和文本对之间的对比损失中进行成对相似度计算会带来计算挑战。本文提出了一种新颖的基于网络规模图像文本数据的弱监督预训练视觉模型的方法。所提出的方法将图像文本数据上的预训练重新构建为一个分类任务。因此，它消除了在对比损失中进行成对相似度计算的需要，相较于在网络规模数据上进行对比学习，实现了训练速度显著提升2.7倍。通过广泛的实验涵盖不同的视觉任务，包括检测和分割，我们证明了所提出的方法保持了高表示质量。我们的源代码以及预训练模型权重和训练配方可在https://github.com/apple/corenet 上获得。

English

Contrastive learning has emerged as a transformative method for learning effective visual representations through the alignment of image and text embeddings. However, pairwise similarity computation in contrastive loss between image and text pairs poses computational challenges. This paper presents a novel weakly supervised pre-training of vision models on web-scale image-text data. The proposed method reframes pre-training on image-text data as a classification task. Consequently, it eliminates the need for pairwise similarity computations in contrastive loss, achieving a remarkable 2.7times acceleration in training speed compared to contrastive learning on web-scale data. Through extensive experiments spanning diverse vision tasks, including detection and segmentation, we demonstrate that the proposed method maintains high representation quality. Our source code along with pre-trained model weights and training recipes is available at https://github.com/apple/corenet.

CatLIP：在Web规模的图像文本数据上进行2.7倍更快的预训练，实现与CLIP相当的视觉识别准确性

CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data

摘要

Support