PaliGemma：一种多功能的3B VLM用于迁移

PaliGemma: A versatile 3B VLM for transfer

July 10, 2024

作者: Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Koppula, Fangyu Liu, Adam Grycner, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong, Julian Eisenschlos, Rishabh Kabra, Matthias Bauer, Matko Bošnjak, Xi Chen, Matthias Minderer, Paul Voigtlaender, Ioana Bica, Ivana Balazevic, Joan Puigcerver, Pinelopi Papalampidi, Olivier Henaff, Xi Xiong, Radu Soricut, Jeremiah Harmsen, Xiaohua Zhai

cs.AI

摘要

PaliGemma是一种基于SigLIP-So400m视觉编码器和Gemma-2B语言模型的开放式视觉-语言模型（VLM）。它经过训练，是一种多才多艺、知识广泛的基础模型，适用于迁移学习。在各种开放世界任务中表现出色。我们对PaliGemma进行了近40项不同任务的评估，包括标准VLM基准测试，以及更专业的任务，如遥感和分割。

English

PaliGemma is an open Vision-Language Model (VLM) that is based on the SigLIP-So400m vision encoder and the Gemma-2B language model. It is trained to be a versatile and broadly knowledgeable base model that is effective to transfer. It achieves strong performance on a wide variety of open-world tasks. We evaluate PaliGemma on almost 40 diverse tasks including standard VLM benchmarks, but also more specialized tasks such as remote-sensing and segmentation.