Nanbeige4.2-3B 在 Apple Silicon 上：修復部署錯誤與降低循環 Transformer 記憶體開銷

摘要

Nanbeige4.2-3B 是一個具有30億參數的智能體模型，其核心為迴圈Transformer（LT），透過重複使用同一層堆疊進行第二次前向傳播，在不增加參數的情況下有效提升模型深度。在Apple Silicon（MPS）上進行評估時，我們發現了五個獨立的錯誤，導致已發布的檢查點無法直接透過Hugging Face transformers執行（包括一個靜默歸零的RoPE緩衝區，以及對已移除之transformers快取API的呼叫）。此外，我們證明僅修正這些錯誤仍不足以應付智能體任務，原因在於LT的層重用策略（實際上使峰值注意力記憶體加倍）被用來實現參數效率。因此，我們引入了一種分塊預填充策略，以緩解所產生的記憶體容量損失，在32 GiB共享記憶體上將可容許的上下文寬度延展了2.7倍。然而，即使記憶體開銷降低，我們仍顯示需要修補程式才能使Nanbeige4.2-3B得以使用；同時解決系統提示與MPS原生記憶體錯誤後，最終才能在標準MCP與工具呼叫基準上進行可靠評估。在MCPMark的一個子集上，除錯後的模型能完成多達30%的真實智能體任務（原先為0%）；在BFCL上，它在單一工具呼叫方面近乎完美，但在多工具測試中大多失敗。我們已在https://github.com/johnhalloran321/Nanbeige4.2-3B-mps-fix發布修補後的檢查點、系統提示最佳化器及評估框架。

English

Nanbeige4.2-3B is a 3B-parameter agentic model built around a Looped Transformer (LT) that reuses one stack of layers for a second forward pass, adding effective depth without additional parameters. Evaluated on Apple Silicon (MPS), we identify five independent bugs which prevent the released checkpoint from running via Hugging Face transformers out of the box (including a silently-zeroed RoPE buffer and calls to removed transformers cache APIs). Furthermore, we show that fixing these bugs is still not sufficient for agentic tasks, due to the LT's layer-reuse strategy (which effectively doubles peak attention memory) used to achieve parameter efficiency. We thus introduce a chunked-prefill strategy which alleviates the incurred memory-capacity penalty, extending allowable context width by 2.7 times on 32~GiB shared memory. However, even with the reduced memory overhead, we show that patches are required to render Nanbeige4.2-3B usable; resolving both system prompt and MPS-native memory bugs finally allows reliable evaluation on standard MCP and tool-calling benchmarks. On a subset of MCPMark, the debugged model completes up to 30\% of real agentic tasks (up from the original's 0\%), while, on BFCL, it is near-perfect at single tool calls (yet fails the majority of multi-tool tests). We release the patched checkpoint, system prompt optimizer, and evaluation harnesses at https://github.com/johnhalloran321/Nanbeige4.2-3B-mps-fix.