PaliGemma：転移学習のための汎用3B VLM

PaliGemma: A versatile 3B VLM for transfer

July 10, 2024

著者: Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Koppula, Fangyu Liu, Adam Grycner, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong, Julian Eisenschlos, Rishabh Kabra, Matthias Bauer, Matko Bošnjak, Xi Chen, Matthias Minderer, Paul Voigtlaender, Ioana Bica, Ivana Balazevic, Joan Puigcerver, Pinelopi Papalampidi, Olivier Henaff, Xi Xiong, Radu Soricut, Jeremiah Harmsen, Xiaohua Zhai

cs.AI

要旨

PaliGemmaは、SigLIP-So400mビジョンエンコーダとGemma-2B言語モデルを基盤としたオープンなVision-Language Model（VLM）です。このモデルは、汎用性が高く幅広い知識を持つベースモデルとして訓練されており、転移学習に効果的です。PaliGemmaは、多様なオープンワールドタスクにおいて優れた性能を発揮します。私たちは、標準的なVLMベンチマークに加え、リモートセンシングやセグメンテーションなどより専門的なタスクを含む、約40種類の多様なタスクでPaliGemmaを評価しました。

English

PaliGemma is an open Vision-Language Model (VLM) that is based on the SigLIP-So400m vision encoder and the Gemma-2B language model. It is trained to be a versatile and broadly knowledgeable base model that is effective to transfer. It achieves strong performance on a wide variety of open-world tasks. We evaluate PaliGemma on almost 40 diverse tasks including standard VLM benchmarks, but also more specialized tasks such as remote-sensing and segmentation.