噪聲超網絡：在擴散模型中攤銷測試時計算

摘要

測試時縮放的新範式在大型語言模型（LLMs，如推理模型）和生成視覺模型中取得了顯著突破，使模型能在推理過程中分配額外計算資源，有效應對日益複雜的問題。儘管這種方法帶來了改進，但一個重要限制也隨之顯現：計算時間的大幅增加使得該過程變得緩慢，對許多應用來說不切實際。鑑於這一範式的成功及其日益廣泛的應用，我們力求在保留其優勢的同時，避免推理開銷。在本研究中，我們提出了一種解決方案，旨在將測試時縮放的知識整合到模型後訓練階段。具體而言，我們用噪聲超網絡替代了擴散模型中的獎勵引導測試時噪聲優化，該網絡調製初始輸入噪聲。我們提出了一個理論基礎框架，通過一個可處理的噪聲空間目標，為蒸餾生成器學習這種獎勵傾斜分佈，既保持對基礎模型的忠實度，又優化期望特性。我們展示了該方法以極低的計算成本，恢復了顯式測試時優化帶來的大部分質量提升。代碼可在https://github.com/ExplainableML/HyperNoise獲取。

English

The new paradigm of test-time scaling has yielded remarkable breakthroughs in Large Language Models (LLMs) (e.g. reasoning models) and in generative vision models, allowing models to allocate additional computation during inference to effectively tackle increasingly complex problems. Despite the improvements of this approach, an important limitation emerges: the substantial increase in computation time makes the process slow and impractical for many applications. Given the success of this paradigm and its growing usage, we seek to preserve its benefits while eschewing the inference overhead. In this work we propose one solution to the critical problem of integrating test-time scaling knowledge into a model during post-training. Specifically, we replace reward guided test-time noise optimization in diffusion models with a Noise Hypernetwork that modulates initial input noise. We propose a theoretically grounded framework for learning this reward-tilted distribution for distilled generators, through a tractable noise-space objective that maintains fidelity to the base model while optimizing for desired characteristics. We show that our approach recovers a substantial portion of the quality gains from explicit test-time optimization at a fraction of the computational cost. Code is available at https://github.com/ExplainableML/HyperNoise

噪聲超網絡：在擴散模型中攤銷測試時計算

Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models

摘要

Support