YOLO-PEFT：YOLO系列上的参数高效微调

摘要

从语言模型迁移而来的通用参数高效微调（PEFT）方法在实时检测器上可能静默失效，原因在于实时检测器的异构算子与检测专用组件对其施加了常规Transformer堆叠所不具备的放置约束。我们提出YOLO-PEFT，一种结构感知框架，将适配器放置形式化为可审计的约束规划问题。给定检测器计算图、PEFT请求和资源预算，YOLO-PEFT分配算子角色与语义角色，评估显式算子有效性、检测器语义、图接口及部署谓词，为每个被排除的模块记录原因代码，并输出带预算约束的目标模块方案，或在训练前返回拒绝（Refuse）决策。在官方VOC07+12训练验证集到VOC07测试集的协议下，规划器选定的RS-LoRA在YOLO11s和YOLO12s上分别达到0.7138和0.7307的mAP50-95，而全量监督微调（Full-SFT）分别为0.6428和0.6662。在RT-DETR-L上，所有七个被评估的LoRA系列配置均超过预定义的灾难性阈值，这支持在所评估覆盖范围内做出经校准的“拒绝-至-全量监督微调”（Refuse-to-Full-SFT）决策。受控的YOLO11审计进一步表明，LoRA将峰值训练内存降低43.9%，尽管训练耗时增加至1.72倍。在所评估的检测器家族、放置策略与校准覆盖范围内，YOLO-PEFT以显式、可检查的规划替代了手动目标模块试错，同时保留经过验证的训练-保存-合并-导出路径；对未见检测器架构的拒绝决策仍是一个有待验证的开放问题。项目主页：github.com/Tencent/YOLO-Master

English

Generic parameter-efficient fine-tuning (PEFT) methods transferred from language models can fail silently on real-time detectors, whose heterogeneous operators and detection-specific components impose placement constraints absent from regular Transformer stacks. We propose YOLO-PEFT, a structure-aware framework that formulates adapter placement as an auditable constraint-planning problem. Given a detector graph, a PEFT request, and a resource budget, YOLO-PEFT assigns operator and semantic roles, evaluates explicit operator-validity, detector-semantic, graph-interface, and deployment predicates, records a reason code for each excluded module, and either emits a budgeted target-module plan or returns Refuse before training. Under the official VOC07+12 trainval-to-VOC07 test protocol, planner-selected RS-LoRA reaches 0.7138 and 0.7307 mAP50-95 on YOLO11s and YOLO12s, respectively, compared with 0.6428 and 0.6662 for Full-SFT. On RT-DETR-L, all seven evaluated LoRA-family configurations cross the predefined catastrophic threshold, supporting a calibrated Refuse-to-Full-SFT decision within the evaluated coverage. A controlled YOLO11 audit further shows that LoRA reduces peak training memory by 43.9 percent, although training takes 1.72 times longer. Within the evaluated detector families, placement policies, and calibration coverage, YOLO-PEFT replaces manual target-module trial and error with explicit, inspectable planning while preserving verified train-save-merge-export paths; refusal on unseen detector architectures remains an open validation problem. Project Page: github.com/Tencent/YOLO-Master