您的预训练模型有所改进吗？一种基于多头后验的方法

摘要

预训练模型的出现对自然语言处理（NLP）和计算机视觉到关系数据集等领域产生了显著影响。传统上，这些模型是通过微调下游任务来评估的。然而，这引发了如何更高效、更有效地评估这些模型的问题。在本研究中，我们探讨了一种新颖的方法，利用与每个实体相关的元特征作为世界知识的来源，并利用模型中的实体表示。我们提出利用这些表示和元特征之间的一致性作为评估预训练模型的度量标准。我们的方法在各个领域展示了其有效性，包括具有关系数据集、大型语言模型和图像模型的模型。

English

The emergence of pretrained models has significantly impacted from Natural Language Processing (NLP) and Computer Vision to relational datasets. Traditionally, these models are assessed through fine-tuned downstream tasks. However, this raises the question of how to evaluate these models more efficiently and more effectively. In this study, we explore a novel approach where we leverage the meta features associated with each entity as a source of worldly knowledge and employ entity representations from the models. We propose using the consistency between these representations and the meta features as a metric for evaluating pretrained models. Our method's effectiveness is demonstrated across various domains, including models with relational datasets, large language models and images models.

您的预训练模型有所改进吗？一种基于多头后验的方法

Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach

摘要

Support