PTXBench：以架構特定 PTX 對 GPU 核心優化之大型語言模型進行基準測試與適配

摘要

我們提出 PTXBench，這是一個用於評估與調適大型語言模型（LLMs）運用架構特定 PTX 進行 GPU 核心最佳化的基準測試。PTXBench 衡量功能正確性、所選目標指令是否在執行時期確實被執行，以及在 H100 和 B200 GPU 上，GEMM 與注意力機制工作負載相較於前沿函式庫的加速比。我們的評估顯示，架構特定的 PTX 能力仍參差不齊：在複雜的注意力反向傳播工作負載上，成功率大幅下降，且執行目標指令未必能轉化為具競爭力的效能。在所評估的模型中，沒有任何一個能在整套基準測試中持續匹敵前沿函式庫。我們進一步使用監督式微調來調適 Qwen3.6-27B。修復條件訓練改善了若干任務，但泛化能力仍不均；除了資料集規模之外，資料覆蓋範圍、平衡性以及推理教師模型的品質同樣至關重要。PTXBench 提供了一個可稽核的測試平台，用於衡量並改善 LLM 善用不斷演進之 GPU 架構的能力。

English

We introduce PTXBench, a benchmark for evaluating and adapting large language models (LLMs) to use architecture-specific PTX for GPU kernel optimization. PTXBench measures functional correctness, whether selected target instructions execute at runtime, and speedup over frontier libraries across GEMM and attention workloads on H100 and B200 GPUs. Our evaluation shows that architecture-specific PTX capability remains uneven: success rates fall substantially on complex attention backward workloads, and executing the target instructions does not necessarily translate into competitive performance. No evaluated model consistently matches frontier libraries across the suite. We further adapt Qwen3.6-27B using supervised fine-tuning. Repair-conditioned training improves several tasks, but generalization remains uneven; data coverage, balance, and the quality of the reasoning teacher matter in addition to dataset size. PTXBench provides an auditable testbed for measuring and improving LLMs' ability to exploit evolving GPU architectures.