无损张量压缩作为程序合成

摘要

模型检查点在数量和规模上都在不断增长，这使得归档、传输和部署的成本日益增加。通用压缩器能够降低存储需求，但忽略了张量结构，而现有的张量专用压缩器则依赖于固定且格式特定的流水线。我们提出Brevis，将无损张量压缩形式化为程序合成问题。我们设计了一种带类型的领域特定语言（DSL），通过一组可逆算子来捕获重复出现的张量结构，例如重复区域和浮点字段。给定一个张量，Brevis合成一个自包含的DSL程序，以逐位精确的方式重建该张量。从少量具有代表性的张量样本中学习得到的检查点专属先验知识，引导有界A*搜索合成紧凑程序，这些程序随后可直接执行以实现逐位精确解压。在涵盖语言、音频和图像生成模型的10个公开检查点上，Brevis将2.13 TB的检查点数据压缩至1.41 TB，实现了33.93%的存储缩减。与包括zstd和gzip在内的四种通用压缩器相比，它生成的归档文件最多缩小30.87%，并且生成的归档文件也小于张量专用压缩器ZipNN和DFloat11。在实际并发配置下，Brevis实现了3.60 GB/s的压缩速度和6.61 GB/s的解压速度，同时保留每一个源字节。

English

Model checkpoints are growing in both number and size, which makes archival, transfer, and deployment increasingly costly. General-purpose compressors can reduce storage requirements but ignore tensor structure, whereas existing tensor-specific compressors rely on fixed and format-specific pipelines. We present Brevis, which formulates lossless tensor compression as program synthesis. We design a typed domain-specific language (DSL) that captures recurring tensor structures, such as repeated regions and floating-point fields, through a set of reversible operators. Given a tensor, Brevis synthesizes a self-contained DSL program that reconstructs it bit-exactly. A checkpoint-specific production prior, learned from a small representative sample of tensors, guides a bounded A* search to synthesize compact programs, which can later be executed directly for bit-exact decompression. On 10 public checkpoints spanning language, audio, and image generation models, Brevis reduces 2.13 TB of checkpoint data to 1.41 TB, a 33.93% storage reduction. It produces archives up to 30.87% smaller than those of four general-purpose compressors, including zstd and gzip, and smaller archives than the tensor-specific compressors ZipNN and DFloat11. Under a practical concurrency configuration, Brevis achieves 3.60 GB/s compression and 6.61 GB/s decompression while preserving every source byte.