這個模型是否也能識別狗？從權重中進行零樣本模型搜索

Can this Model Also Recognize Dogs? Zero-Shot Model Search from Weights

February 13, 2025

作者: Jonathan Kahana, Or Nathan, Eliahu Horwitz, Yedid Hoshen

cs.AI

摘要

隨著公開模型數量的增加，現在可能有預先訓練的線上模型可滿足使用者所需的大多數任務。然而，目前的模型搜尋方法仍然基本上是在文件中進行基於文本的搜索，因此使用者無法找到相關的模型。本文提出了ProbeLog，一種檢索分類模型的方法，該模型可以識別目標概念，例如"狗"，而無需訪問模型元數據或訓練數據。與以往的探測方法不同，ProbeLog 通過觀察模型對一組固定輸入（探針）的響應，為每個輸出維度（logit）計算描述符。我們的方法支持基於 logit 的檢索（"找到更多像這樣的 logit"）和零樣本、基於文本的檢索（"找到所有與狗相對應的 logit"）。由於基於探測的表示需要通過模型進行多次昂貴的前向傳遞，我們開發了一種基於協同過濾的方法，將編碼存儲庫的成本降低了 3 倍。我們展示了 ProbeLog 在現實世界和細粒度搜索任務中實現了高檢索準確性，並且可擴展到完整大小的存儲庫。

English

With the increasing numbers of publicly available models, there are probably pretrained, online models for most tasks users require. However, current model search methods are rudimentary, essentially a text-based search in the documentation, thus users cannot find the relevant models. This paper presents ProbeLog, a method for retrieving classification models that can recognize a target concept, such as "Dog", without access to model metadata or training data. Differently from previous probing methods, ProbeLog computes a descriptor for each output dimension (logit) of each model, by observing its responses on a fixed set of inputs (probes). Our method supports both logit-based retrieval ("find more logits like this") and zero-shot, text-based retrieval ("find all logits corresponding to dogs"). As probing-based representations require multiple costly feedforward passes through the model, we develop a method, based on collaborative filtering, that reduces the cost of encoding repositories by 3x. We demonstrate that ProbeLog achieves high retrieval accuracy, both in real-world and fine-grained search tasks and is scalable to full-size repositories.