大规模智能体行为分析中的扎根理论应用

摘要

理解智能体行为需要能够扩展到数千条轨迹的方法，并能在冗长且往往陌生的任务中揭示新的模式，而预构建的分类器在这种情况下往往力不从心。我们提出将扎根理论引入智能体轨迹分析——这一源自社会科学、已有六十年历史的定性方法，具有原则性的饱和判据以及从数据到理论的可审计轨迹。我们提出AutoTraceGT（通过扎根理论实现自动化轨迹分析），这是首个在智能体轨迹上自动化扎根理论的多智能体流水线。它迭代执行开放编码、轴心编码和理论编码，直至达到饱和，从而生成针对每项任务量身定制的行为分类体系。在六个轨迹语料库上，AutoTraceGT生成的编码手册能够恢复人工标注分类体系中73%至91%的失败模式，并揭示出这些分类体系所遗漏的额外模式。其涌现出的理论叙事与既有专家论述相一致。作为演绎性特征空间使用时，该编码手册在下游失败预测任务上优于零样本和少样本LLM基线。这些结果表明，扎根理论为机器学习研究者和智能体开发者研究智能体的实际行为提供了一种可扩展的分析工具。

English

Understanding agent behavior requires methods that scale to thousands of trajectories and surface new patterns in long, often unfamiliar tasks where pre-built classifiers fall short. We propose to bring grounded theory into agent trajectory analysis: a six-decade-old qualitative method from the social sciences, with a principled saturation criterion and an auditable trail from data to theory. We propose AutoTraceGT (Automated Trace analysis through Grounded Theory), the first multi-agent pipeline that automates grounded theory on agent trajectories. It iteratively performs open, axial, and theoretical coding until saturation, producing a behavioral taxonomy tailored to each task. Across six trajectory corpora, AutoTraceGT produces codebooks that recover 73-91 percent of the failure modes in human-annotated taxonomies and surface additional patterns that those taxonomies miss. The emergent theoretical narrative aligns with prior expert accounts. Used as a deductive feature space, the codebook outperforms zero-shot and few-shot LLM baselines on downstream failure prediction. These results suggest Grounded Theory offers a scalable analytic tool for ML researchers and agent developers studying what agents actually do.