OpenFlamingo：用于训练大型自回归视觉语言模型的开源框架

OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models

August 2, 2023

作者: Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, Ludwig Schmidt

cs.AI

摘要

我们介绍了OpenFlamingo，这是一个参数范围从3B到9B的自回归视觉-语言模型系列。OpenFlamingo是一个持续进行的努力，旨在制作DeepMind的Flamingo模型的开源复制品。在七个视觉-语言数据集上，OpenFlamingo模型的性能平均在80%到89%之间，与相应的Flamingo性能相当。本技术报告描述了我们的模型、训练数据、超参数和评估套件。我们在https://github.com/mlfoundations/open_flamingo 上分享我们的模型和代码。

English

We introduce OpenFlamingo, a family of autoregressive vision-language models ranging from 3B to 9B parameters. OpenFlamingo is an ongoing effort to produce an open-source replication of DeepMind's Flamingo models. On seven vision-language datasets, OpenFlamingo models average between 80 - 89% of corresponding Flamingo performance. This technical report describes our models, training data, hyperparameters, and evaluation suite. We share our models and code at https://github.com/mlfoundations/open_flamingo.