您的預訓練模型有改進嗎？一種基於多頭後驗的方法

摘要

預訓練模型的出現對自然語言處理（NLP）和計算機視覺以及關聯數據集產生了顯著影響。傳統上，這些模型通常通過微調後續任務來評估。然而，這引發了如何更有效地評估這些模型的問題。在本研究中，我們探索了一種新方法，利用與每個實體相關的元特徵作為世界知識的來源，並利用模型中的實體表示。我們提出使用這些表示和元特徵之間的一致性作為評估預訓練模型的度量標準。我們的方法在各個領域展示了有效性，包括具有關聯數據集、大型語言模型和圖像模型的模型。

English

The emergence of pretrained models has significantly impacted from Natural Language Processing (NLP) and Computer Vision to relational datasets. Traditionally, these models are assessed through fine-tuned downstream tasks. However, this raises the question of how to evaluate these models more efficiently and more effectively. In this study, we explore a novel approach where we leverage the meta features associated with each entity as a source of worldly knowledge and employ entity representations from the models. We propose using the consistency between these representations and the meta features as a metric for evaluating pretrained models. Our method's effectiveness is demonstrated across various domains, including models with relational datasets, large language models and images models.

您的預訓練模型有改進嗎？一種基於多頭後驗的方法

Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach

摘要

Support