YOLO-PEFT：針對YOLO系列的參數高效微調

摘要

通用參數高效微調（PEFT）方法自語言模型遷移後，在即時偵測器上可能靜默失效；此類偵測器的異構算子與偵測特定元件施加了放置約束，而一般 Transformer 堆疊中並不存在這些約束。我們提出 YOLO-PEFT，一個結構感知框架，將適配器放置形式化為可稽核的約束規劃問題。給定偵測器圖、PEFT 請求與資源預算，YOLO-PEFT 指派算子角色與語義角色，評估明確的算子有效性、偵測器語義、圖介面與部署謂詞，為每個被排除的模組記錄原因碼，並在訓練前要麼輸出符合預算的目標模組計劃，要麼返回「Refuse」。在官方 VOC07+12 trainval 至 VOC07 測試協議下，規劃器選取的 RS-LoRA 於 YOLO11s 與 YOLO12s 上分別達到 0.7138 與 0.7307 的 mAP50-95，而 Full-SFT 僅分別為 0.6428 與 0.6662。在 RT-DETR-L 上，所有七個受評估的 LoRA 家族配置均跨越預先定義的災難性門檻，支持在所評估覆蓋範圍內做出經校準的「Refuse-to-Full-SFT」決策。一項受控的 YOLO11 稽核進一步顯示，LoRA 將峰值訓練記憶體降低 43.9%，但訓練時間延長為 1.72 倍。在所評估的偵測器家族、放置策略與校準覆蓋範圍內，YOLO-PEFT 以明確且可檢視的規劃取代手動的目標模組試錯，同時保留已驗證的訓練-儲存-合併-匯出路徑；對未見過的偵測器架構進行拒絕仍是一個開放的驗證問題。專案頁面：github.com/Tencent/YOLO-Master

English

Generic parameter-efficient fine-tuning (PEFT) methods transferred from language models can fail silently on real-time detectors, whose heterogeneous operators and detection-specific components impose placement constraints absent from regular Transformer stacks. We propose YOLO-PEFT, a structure-aware framework that formulates adapter placement as an auditable constraint-planning problem. Given a detector graph, a PEFT request, and a resource budget, YOLO-PEFT assigns operator and semantic roles, evaluates explicit operator-validity, detector-semantic, graph-interface, and deployment predicates, records a reason code for each excluded module, and either emits a budgeted target-module plan or returns Refuse before training. Under the official VOC07+12 trainval-to-VOC07 test protocol, planner-selected RS-LoRA reaches 0.7138 and 0.7307 mAP50-95 on YOLO11s and YOLO12s, respectively, compared with 0.6428 and 0.6662 for Full-SFT. On RT-DETR-L, all seven evaluated LoRA-family configurations cross the predefined catastrophic threshold, supporting a calibrated Refuse-to-Full-SFT decision within the evaluated coverage. A controlled YOLO11 audit further shows that LoRA reduces peak training memory by 43.9 percent, although training takes 1.72 times longer. Within the evaluated detector families, placement policies, and calibration coverage, YOLO-PEFT replaces manual target-module trial and error with explicit, inspectable planning while preserving verified train-save-merge-export paths; refusal on unseen detector architectures remains an open validation problem. Project Page: github.com/Tencent/YOLO-Master