Hypencoder：用於資訊檢索的超網路

摘要

絕大多數的檢索模型依賴向量內積來產生查詢和文件之間的相關性分數。這自然地限制了可用的相關性分數的表達能力。我們提出一種新的範式，不是產生一個向量來代表查詢，而是產生一個小型神經網絡，它作為一個學習到的相關性函數。這個小型神經網絡接收文檔的表示，本文中我們使用一個單一向量，並產生一個標量相關性分數。為了產生這個小型神經網絡，我們使用一個超網絡，一個產生其他網絡權重的網絡，作為我們的查詢編碼器或我們稱之為Hypencoder。在領域內搜索任務上的實驗表明，Hypencoder能夠顯著優於強大的密集檢索模型，並且比重新排序模型和規模大一個數量級的模型具有更高的指標。Hypencoder還表現出對領域外搜索任務的良好泛化能力。為了評估Hypencoder的能力程度，我們在一組困難的檢索任務上進行評估，包括tip-of-the-tongue檢索和instruction-following檢索任務，發現與標準檢索任務相比，性能差距顯著擴大。此外，為了展示我們方法的實用性，我們實現了一個近似搜索算法，並展示我們的模型能夠在不到60毫秒的時間內搜索880萬個文檔。

English

The vast majority of retrieval models depend on vector inner products to produce a relevance score between a query and a document. This naturally limits the expressiveness of the relevance score that can be employed. We propose a new paradigm, instead of producing a vector to represent the query we produce a small neural network which acts as a learned relevance function. This small neural network takes in a representation of the document, in this paper we use a single vector, and produces a scalar relevance score. To produce the little neural network we use a hypernetwork, a network that produce the weights of other networks, as our query encoder or as we call it a Hypencoder. Experiments on in-domain search tasks show that Hypencoder is able to significantly outperform strong dense retrieval models and has higher metrics then reranking models and models an order of magnitude larger. Hypencoder is also shown to generalize well to out-of-domain search tasks. To assess the extent of Hypencoder's capabilities, we evaluate on a set of hard retrieval tasks including tip-of-the-tongue retrieval and instruction-following retrieval tasks and find that the performance gap widens substantially compared to standard retrieval tasks. Furthermore, to demonstrate the practicality of our method we implement an approximate search algorithm and show that our model is able to search 8.8M documents in under 60ms.

Hypencoder：用於資訊檢索的超網路

Hypencoder: Hypernetworks for Information Retrieval

摘要

Summary

Support

Support