DragMesh：轻松实现交互式3D生成

摘要

尽管生成模型在创建静态3D内容方面表现出色，但如何让系统理解物体运动方式并响应交互仍是根本性挑战。当前关节运动方法正处于十字路口：要么物理一致但速度过慢无法实时应用，要么具备生成能力却违反基本运动学约束。我们提出DragMesh——一个围绕轻量级运动生成核心构建的实时交互式3D关节运动框架。核心创新在于新颖的解耦式运动学推理与运动生成框架：首先通过分离语义意图推理（确定关节类型）和几何回归（使用运动学预测网络KPP-Net确定轴心和原点）来推断潜在关节参数；其次利用对偶四元数表示刚体运动所具有的紧凑、连续、无奇异性特点，开发了新型对偶四元数变分自编码器（DQ-VAE）。该DQ-VAE接收预测先验与原始用户拖拽指令，生成完整合理的运动轨迹。为确保严格遵循运动学规则，我们通过FiLM（特征线性调制）条件化将关节先验注入DQ-VAE非自回归Transformer解码器的每一层。这种持续的多尺度指导辅以数值稳定的叉积损失来保证轴对齐。解耦设计使DragMesh实现实时性能，并能对未见物体进行合理生成式关节运动而无需重新训练，为生成式3D智能迈出实用一步。代码：https://github.com/AIGeeksGroup/DragMesh 项目页面：https://aigeeksgroup.github.io/DragMesh

English

While generative models have excelled at creating static 3D content, the pursuit of systems that understand how objects move and respond to interactions remains a fundamental challenge. Current methods for articulated motion lie at a crossroads: they are either physically consistent but too slow for real-time use, or generative but violate basic kinematic constraints. We present DragMesh, a robust framework for real-time interactive 3D articulation built around a lightweight motion generation core. Our core contribution is a novel decoupled kinematic reasoning and motion generation framework. First, we infer the latent joint parameters by decoupling semantic intent reasoning (which determines the joint type) from geometric regression (which determines the axis and origin using our Kinematics Prediction Network (KPP-Net)). Second, to leverage the compact, continuous, and singularity-free properties of dual quaternions for representing rigid body motion, we develop a novel Dual Quaternion VAE (DQ-VAE). This DQ-VAE receives these predicted priors, along with the original user drag, to generate a complete, plausible motion trajectory. To ensure strict adherence to kinematics, we inject the joint priors at every layer of the DQ-VAE's non-autoregressive Transformer decoder using FiLM (Feature-wise Linear Modulation) conditioning. This persistent, multi-scale guidance is complemented by a numerically-stable cross-product loss to guarantee axis alignment. This decoupled design allows DragMesh to achieve real-time performance and enables plausible, generative articulation on novel objects without retraining, offering a practical step toward generative 3D intelligence. Code: https://github.com/AIGeeksGroup/DragMesh. Website: https://aigeeksgroup.github.io/DragMesh.

DragMesh：轻松实现交互式3D生成

DragMesh: Interactive 3D Generation Made Easy

摘要

Support