一個主權級、開源的基礎模型，適用於德語和英語

摘要

我們推出 Soofi S 30B-A3B，這是一個主權開源的混合專家（MoE）模型，結合 Mamba 與 Transformer 架構，專為德語與英語設計。其混合設計使每個 token 僅啟動 30B 參數中的 3B，且隨著上下文增長，推論快取大小幾乎保持恆定，這使其在長上下文、高併發部署場景中，相較於密集模型具有顯著的吞吐量優勢。該模型在約 27 兆個 token 上進行預訓練，並特意調高德語資料權重。Soofi S 在英語與英語的綜合基準測試中，表現與 14B 至 27B 參數的密集模型相當，並在 17 個開放基礎模型中，於兩種語言上取得最佳的程式碼綜合表現，同時超越我們比較中的所有歐洲主權基線模型，包含那些活躍參數量更大的模型。在完全開放的模型中，Soofi S 在英語與德語評估中獲得最高分數，領先 Olmo 3 32B 與 Apertus 70B。Soofi S 是基於德國工業 AI 雲端（由德意志電信在慕尼黑營運的主權 HPC 規模 AI 基礎設施）端到端建構而成。Soofi S 將以高度寬容的開放存取條款發布：包含權重、精選的中間檢查點、完整的按來源資料核算、超參數，以及訓練與評估程式碼。在原始碼授權許可的情況下，資料建構產物將以寬容授權發布；商業授權來源則會以彙總統計與精確的混合比例核算方式記錄。

English

We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, giving it a decisive throughput advantage over dense models for long-context, high-concurrency deployment. Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches dense 14 to 27B models on aggregate English and German benchmarks while achieving the best code aggregates in both languages among 17 open base models, and outperforms every European sovereign baseline in our comparison, including ones far larger in active parameters. Among fully open models, Soofi S obtains the highest English and German evaluation scores, ahead of Olmo 3 32B and Apertus 70B. Soofi S was built end-to-end on the German Industrial AI Cloud, a sovereign HPC scale AI infrastructure operated by Deutsche Telekom in Munich. Soofi S will be released under highly permissive, open-access terms: weights, selected intermediate checkpoints, full per-source data accounting, hyperparameters, and training and evaluation code. Where source licenses permit, data-construction artifacts are released under permissive licenses; commercially licensed sources are documented with aggregate statistics and exact mixture accounting.