Nanbeige4.2-3B在Apple Silicon上的部署：修复部署缺陷并降低循环Transformer的内存开销

摘要

Nanbeige4.2-3B是一个30亿参数的智能体模型，基于循环Transformer（LT）架构构建，该架构通过复用一个层堆栈进行第二次前向传播，在不增加参数的情况下增加了有效深度。在Apple Silicon（MPS）上评估时，我们发现了五个独立的缺陷，这些缺陷导致已发布的检查点无法通过Hugging Face transformers开箱即用（包括静默置零的RoPE缓冲区以及对已移除的transformers缓存API的调用）。此外，我们证明即使修复了这些缺陷，由于LT的层复用策略（实际上使峰值注意力内存翻倍）被用于实现参数效率，该模型仍不足以胜任智能体任务。为此，我们引入了一种分块预填充策略，以缓解由此产生的内存容量开销，在32 GiB共享内存上将允许的上下文长度扩展了2.7倍。然而，即使减少了内存开销，我们仍表明需要补丁才能使Nanbeige4.2-3B得以使用；解决系统提示词和MPS原生内存缺陷后，最终可以在标准MCP和工具调用基准上进行可靠的评估。在MCPMark的一个子集上，经过调试的模型成功完成了高达30%的真实智能体任务（原始版本为0%），而在BFCL上，该模型在单工具调用方面近乎完美（但在大多数多工具测试中失败）。我们在https://github.com/johnhalloran321/Nanbeige4.2-3B-mps-fix发布了修补后的检查点、系统提示词优化器和评估套件。

English

Nanbeige4.2-3B is a 3B-parameter agentic model built around a Looped Transformer (LT) that reuses one stack of layers for a second forward pass, adding effective depth without additional parameters. Evaluated on Apple Silicon (MPS), we identify five independent bugs which prevent the released checkpoint from running via Hugging Face transformers out of the box (including a silently-zeroed RoPE buffer and calls to removed transformers cache APIs). Furthermore, we show that fixing these bugs is still not sufficient for agentic tasks, due to the LT's layer-reuse strategy (which effectively doubles peak attention memory) used to achieve parameter efficiency. We thus introduce a chunked-prefill strategy which alleviates the incurred memory-capacity penalty, extending allowable context width by 2.7 times on 32~GiB shared memory. However, even with the reduced memory overhead, we show that patches are required to render Nanbeige4.2-3B usable; resolving both system prompt and MPS-native memory bugs finally allows reliable evaluation on standard MCP and tool-calling benchmarks. On a subset of MCPMark, the debugged model completes up to 30\% of real agentic tasks (up from the original's 0\%), while, on BFCL, it is near-perfect at single tool calls (yet fails the majority of multi-tool tests). We release the patched checkpoint, system prompt optimizer, and evaluation harnesses at https://github.com/johnhalloran321/Nanbeige4.2-3B-mps-fix.