無損張量壓縮作為程式合成

摘要

模型檢查點的數量與規模持續增長，使得保存、傳輸與部署的成本日益提高。通用型壓縮器雖能降低儲存需求，卻忽略張量結構；而現有的張量專用壓縮器則依賴固定且格式特定的管線。我們提出 Brevis，將無損張量壓縮表述為程式合成問題。我們設計了一個具型別領域特定語言（DSL），透過一組可逆運算子捕捉反覆出現的張量結構，例如重複區域與浮點數欄位。對於給定的張量，Brevis 合成一個自包含的 DSL 程式，以位元級精確地重建該張量。藉由從少量具代表性的張量樣本中學習而得、針對檢查點特定之生成先驗，我們引導有界 A* 搜尋合成精簡程式，之後可直接執行以進行位元級精確解壓縮。在涵蓋語言、音訊與影像生成模型的 10 個公開檢查點上，Brevis 將 2.13 TB 的檢查點資料縮減至 1.41 TB，儲存空間減少 33.93%。其所產生的壓縮檔比四種通用壓縮器（包括 zstd 與 gzip）小最多 30.87%，且產生的壓縮檔亦小於張量專用壓縮器 ZipNN 與 DFloat11。在實際的併發配置下，Brevis 達到 3.60 GB/s 的壓縮速度與 6.61 GB/s 的解壓縮速度，同時完整保留每一個來源位元組。

English

Model checkpoints are growing in both number and size, which makes archival, transfer, and deployment increasingly costly. General-purpose compressors can reduce storage requirements but ignore tensor structure, whereas existing tensor-specific compressors rely on fixed and format-specific pipelines. We present Brevis, which formulates lossless tensor compression as program synthesis. We design a typed domain-specific language (DSL) that captures recurring tensor structures, such as repeated regions and floating-point fields, through a set of reversible operators. Given a tensor, Brevis synthesizes a self-contained DSL program that reconstructs it bit-exactly. A checkpoint-specific production prior, learned from a small representative sample of tensors, guides a bounded A* search to synthesize compact programs, which can later be executed directly for bit-exact decompression. On 10 public checkpoints spanning language, audio, and image generation models, Brevis reduces 2.13 TB of checkpoint data to 1.41 TB, a 33.93% storage reduction. It produces archives up to 30.87% smaller than those of four general-purpose compressors, including zstd and gzip, and smaller archives than the tensor-specific compressors ZipNN and DFloat11. Under a practical concurrency configuration, Brevis achieves 3.60 GB/s compression and 6.61 GB/s decompression while preserving every source byte.