ReGuLaR：基於渲染思維鏈引導的變分潛在推理

摘要

儘管思維鏈（CoT）能顯著提升大型語言模型的效能，但顯式推理鏈會引入大量計算冗餘。近期潛在推理方法試圖透過將推理過程壓縮至潛在空間來緩解此問題，卻因缺乏適當的壓縮指引而常出現嚴重效能衰退。本研究提出「具現化 CoT 引導的變分潛在推理」（ReGuLaR），以簡潔新穎的潛在學習範式解決此問題。其核心是將潛在推理建模於變分自編碼器框架內，從基於過往狀態的後驗分布中採樣當前潛在推理狀態。具體而言，在學習此變分潛在推理模型時，我們將顯式推理鏈轉譯為圖像，並從中提取稠密的視覺-語義表徵來正則化後驗分布，從而實現高效壓縮且最小化資訊損失。大量實驗表明，ReGuLaR 在計算效率與推理效能上均顯著優於現有潛在推理方法，更透過多模態推理超越 CoT 表現，為潛在推理提供了兼具創新性與洞察力的解決方案。程式碼：https://github.com/FanmengWang/ReGuLaR。

English

While Chain-of-Thought (CoT) significantly enhances the performance of Large Language Models (LLMs), explicit reasoning chains introduce substantial computational redundancy. Recent latent reasoning methods attempt to mitigate this by compressing reasoning processes into latent space, but often suffer from severe performance degradation due to the lack of appropriate compression guidance. In this study, we propose Rendered CoT-Guided variational Latent Reasoning (ReGuLaR), a simple yet novel latent learning paradigm resolving this issue. Fundamentally, we formulate latent reasoning within the Variational Auto-Encoding (VAE) framework, sampling the current latent reasoning state from the posterior distribution conditioned on previous ones. Specifically, when learning this variational latent reasoning model, we render explicit reasoning chains as images, from which we extract dense visual-semantic representations to regularize the posterior distribution, thereby achieving efficient compression with minimal information loss. Extensive experiments demonstrate that ReGuLaR significantly outperforms existing latent reasoning methods across both computational efficiency and reasoning effectiveness, and even surpasses CoT through multi-modal reasoning, providing a new and insightful solution to latent reasoning. Code: https://github.com/FanmengWang/ReGuLaR.

ReGuLaR：基於渲染思維鏈引導的變分潛在推理

ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thought

摘要

Support