这个模型能否也识别狗？从权重中进行零样本模型搜索

Can this Model Also Recognize Dogs? Zero-Shot Model Search from Weights

February 13, 2025

作者: Jonathan Kahana, Or Nathan, Eliahu Horwitz, Yedid Hoshen

cs.AI

摘要

随着公开可用模型数量的增加，很可能为用户所需任务提供了预训练的在线模型。然而，当前的模型搜索方法还比较基础，基本上是在文档中进行基于文本的搜索，因此用户无法找到相关的模型。本文提出了ProbeLog，一种用于检索能识别目标概念（如“狗”）的分类模型的方法，而无需访问模型元数据或训练数据。与先前的探测方法不同，ProbeLog通过观察模型对一组固定输入（探针）的响应来计算每个模型的每个输出维度（logit）的描述符。我们的方法支持基于logit的检索（“查找更多类似的logits”）和零样本、基于文本的检索（“查找所有与狗对应的logits”）。由于基于探测的表示需要通过模型进行多次昂贵的前向传递，我们开发了一种基于协同过滤的方法，将编码存储库的成本降低了3倍。我们证明了ProbeLog在现实世界和细粒度搜索任务中均实现了高检索准确性，并且可扩展到全尺寸存储库。

English

With the increasing numbers of publicly available models, there are probably pretrained, online models for most tasks users require. However, current model search methods are rudimentary, essentially a text-based search in the documentation, thus users cannot find the relevant models. This paper presents ProbeLog, a method for retrieving classification models that can recognize a target concept, such as "Dog", without access to model metadata or training data. Differently from previous probing methods, ProbeLog computes a descriptor for each output dimension (logit) of each model, by observing its responses on a fixed set of inputs (probes). Our method supports both logit-based retrieval ("find more logits like this") and zero-shot, text-based retrieval ("find all logits corresponding to dogs"). As probing-based representations require multiple costly feedforward passes through the model, we develop a method, based on collaborative filtering, that reduces the cost of encoding repositories by 3x. We demonstrate that ProbeLog achieves high retrieval accuracy, both in real-world and fine-grained search tasks and is scalable to full-size repositories.