评估多模态大语言模型作为无人机控制的通用视觉-语言-动作智能体：指令下达、目标接近、目标跟踪与目标搜索

摘要

多模态大语言模型（MLLMs）是图像和视频的强力感知器。我们探究这种能力在多大程度上可以延伸至行动领域：将MLLM直接置入无人机的控制回路，其全部动作空间仅通过提示词声明。近期系统虽已接近这一设定，但日益收窄模型的决策范围。我们将其重新拓宽。我们提出DroneCATS-Agent——一种将MLLM作为可替换组件的架构——以及DroneCATS，一个将模型视为自变量的基准。超越单纯飞向某个像素，我们的智能体将偏航与搜索、不确定时的深思熟虑、以及自主声明到达等任务全部托付给模型——无需微调或函数调用模式。我们在四项核心能力上评估前沿模型与开放模型——接近可见目标、追踪移动目标、在初始视野之外搜索、以及指挥多无人机编队——结果表明，即便最简单的具身设定也远未得到解决。关键在于，为了识别在边缘端率先失效的环节，我们的模型阵容下探至20亿参数。研究结果揭示了一个鲜明的悖论：失效的并非飞行本身。小型开放模型往往比前沿模型更可靠地飞入成功半径，却因过早或根本不声明到达而输掉回合。多机指挥进一步放大了这一差距，小型模型会因盲目地将单一坐标复制到不同视角而失败。从视觉-语言-行动智能体的视角来看，模型的空间感知尚能维持，但其行动协议却难以达标。区分可部署的边缘模型与前沿模型的，并非导航能力，而是持续执行既定协议并发出正确终止动作的纪律性。弥合这一差距所面临的开放问题，在于以机载计算成本实现——产生一个既能持续规划、又能在恰当时刻准确知晓任务完成的快速模型——而DroneCATS正是为度量这一距离而构建。

English

Multimodal Large Language Models (MLLMs) are strong perceivers of images and video. We ask how far that reach extends into acting: dropping an MLLM directly into a drone's control loop, with its entire action space declared solely in the prompt. Recent systems approach this setting but increasingly narrow the model's decision-making. We widen it back. We introduce DroneCATS-Agent, an architecture where the MLLM is a swappable component, and DroneCATS, a benchmark treating the model as the independent variable. Beyond merely flying toward a pixel, our agent entrusts the model to yaw and search, deliberate when unsure, and self-declare arrival---all without fine-tuning or function-calling schemas. Evaluating frontier and open models across four core capabilities---approaching a visible target, tracking a moving one, searching outside the initial view, and commanding a multi-drone fleet---reveals that even the simplest embodied settings are far from solved. Crucially, to identify what breaks first at the edge, our roster scales down to 2B parameters. The findings expose a stark paradox: it is not the flying that fails. Small open models often navigate into the success radius more reliably than frontier models, yet lose the episode by declaring arrival prematurely or not at all. Multi-drone commanding amplifies this divide, with small models failing by blindly copying a single coordinate across distinct views. Viewed as vision-language-action agents, the models' spatial perception holds up, but their action protocol does not. What separates a deployable edge model from a frontier model is not navigation, but the discipline to sustain a declared protocol and emit the correct terminating action. The open problem is closing this gap at onboard compute costs---yielding a fast model that plans persistently and knows exactly when it is done---and DroneCATS is built to measure that distance.