用於設備端語義音頻生成的量化原生運行時

摘要

語義音頻應用日益需要在普通硬體和嵌入式硬體上實現可控生成，而非透過框架繁重的資料中心堆疊。我們提出 aria，一個無依賴的原生執行時，可在普通 GPU、僅 CPU 的機器以及 Raspberry Pi 5 上運行 Stable Audio 3 (SA3) 的完整文字轉音樂流水線，底層無需 Python 或深度學習框架。我們的主要貢獻是對量化的研究：以更低數值精度運行模型以適應緊張的記憶體預算，就地節省記憶體而非增加記憶體。由於執行時擁有所有內部張量，它還暴露了激活引導（activation steering），一種低成本引導模型生成內容的方式。我們透過輸出的三個獨立衡量指標（提示遵循度、整體音頻品質、品味保持度）來評判品質代價，每個指標均與隨機種子間的普通變異進行比較。八位精度在任何指標上均未顯示可測量的品質損失，同時大幅削減記憶體，並且是 GPU 上最快的模式；四位精度則增加了一個較小且有界代價，但足以將參數量達 12 億的模型壓縮至可在 8 GB 的 Pi 上運行。與官方實現相比，aria 的生成速度相當或更快，且啟動速度快約七倍。一個關於引導介面的案例研究生成了帶有品味關聯（聲音調味）的音樂，對部分屬性具有真實但有界的控制。這些結果使得緊湊、量化且內置控制的執行時成為物聯網聲音環境中設備端語義音頻的實用基礎。aria 執行時已在 https://github.com/matteospanio/aria 發布。

English

Semantic audio applications increasingly require controllable generation on commodity and embedded hardware rather than through framework-heavy datacenter stacks. We present aria, a dependency-free native runtime that runs the complete text-to-music pipeline of Stable Audio~3 (SA3) on ordinary GPUs, CPU-only machines, and a Raspberry~Pi~5, with no Python or deep-learning framework underneath. Our main contribution is a study of quantization: running the model at lower numerical precision to fit tight memory budgets, saving memory in place rather than adding to it. Because the runtime owns every internal tensor, it also exposes activation steering, a low-cost way to steer what the model generates. We judge the quality cost with three independent measures of the output (prompt adherence, overall audio quality, taste preservation), each compared against the ordinary variation between random seeds. Eight-bit precision shows no measurable quality loss on any measure while sharply cutting memory, and it is the fastest mode on the GPU; four-bit adds a small, bounded cost but shrinks the footprint enough to run the 1.2-billion-parameter model on an 8\,GB Pi. Against the official implementation, aria matches or exceeds generation speed and starts about seven times faster. A case study of the steering interface generates music carrying taste associations (sonic seasoning), with genuine but bounded control for a subset of attributes. These results make a compact, quantized runtime with built-in control a practical basis for on-device semantic audio in Internet-of-Sounds settings. The aria runtime is released at https://github.com/matteospanio/aria.