FoldingAgent: 从演示视频中推断参数化折纸流程

摘要

我们提出了FoldingAgent，一个智能体框架，能够直接从折纸演示视频中推断显式的参数化折叠程序。该框架利用预训练视觉-语言模型（VLM）的推理能力，并配备一系列专用工具，使智能体能够模拟几何变换、验证物理合理性、检索和比较视觉内容，以及评估自身的预测。为了将视觉内容转化为折叠程序，我们定义了一个参数空间，由纸张的几何结构与一组参数化折叠动作组成。与预测静态折痕图的模型不同，我们的智能体以顺序方式运行，并具备重新规划动作的能力，从而有效缓解多步折叠中固有的累积误差。我们的方法朝弥合人类折纸知识与计算方法之间的差距迈出了一步：前者主要通过非结构化视觉演示进行分享，而后者通常依赖结构化、参数化的表示，例如折痕图或可执行的参数化计划。我们在PurelandFold上评估了我们的方法，这是一个新整理的基准数据集，包含多种Pureland折纸视频，并附有真值几何与动作标签。结果表明，通过将VLM推理与一组专用工具及物理模拟相结合，我们能够成功地将非结构化视觉演示转化为可执行且物理上合理的折叠流程。

English

We present FoldingAgent, an agentic framework for inferring explicit parametric folding programs directly from origami demonstration videos. Our framework leverages the reasoning power of a pre-trained Vision-Language Model (VLM) equipped with a suite of specialized tools that enable the agent to simulate geometric transitions, verify physical plausibility, retrieve and compare visual content, and evaluate its own predictions. To translate visual content into folding programs, we define a parametric space that consists of the paper's geometry and a set of parametric folding actions. Unlike models that predict static crease patterns, our agent operates sequentially and possesses the ability to re-plan its actions, effectively mitigating the compounding errors inherent in multi-step folding. Our approach takes a step toward closing the gap between human origami knowledge, which is primarily shared through unstructured visual demonstrations, and computational methods, which typically rely on structured, parametric representations such as a crease pattern or an executable parametric plan. We evaluate our approach on PurelandFold, a newly curated benchmark of diverse Pureland origami videos with ground-truth geometry and action labels. Our results demonstrate that by combining VLM reasoning with a set of specialized tools and physical simulation, we can successfully transform unstructured visual demonstrations into executable, physically plausible folding procedures.