GARField：基于辐射场的任意物体分组

摘要

由于可以将场景分解为多个粒度级别，因此分组本质上是模糊的——挖掘机的车轮应该被视为单独的部分还是整体的一部分？我们提出了一种名为Radiance Fields的Group Anything with Radiance Fields (GARField)方法，用于从姿态图像输入中将3D场景分解为语义上有意义的组的层次结构。为了做到这一点，我们通过物理尺度来接受分组的模糊性：通过优化一个尺度条件的3D亲和特征场，世界中的一个点可以属于不同尺寸的不同组。我们通过一组由Segment Anything (SAM)提供的2D掩模来优化这个场，以一种尊重由粗到细层次结构的方式，利用尺度来一致地融合来自不同视点的冲突掩模。通过这个场，我们可以通过自动树构建或用户交互推导可能分组的层次结构。我们在各种野外场景上评估了GARField，并发现它有效地在许多级别提取组：对象群、对象和各种子部分。GARField固有地代表多视角一致的分组，并产生比输入SAM掩模更高保真度的组。GARField的分层分组可能具有令人兴奋的下游应用，如3D资产提取或动态场景理解。请访问项目网站https://www.garfield.studio/。

English

Grouping is inherently ambiguous due to the multiple levels of granularity in which one can decompose a scene -- should the wheels of an excavator be considered separate or part of the whole? We present Group Anything with Radiance Fields (GARField), an approach for decomposing 3D scenes into a hierarchy of semantically meaningful groups from posed image inputs. To do this we embrace group ambiguity through physical scale: by optimizing a scale-conditioned 3D affinity feature field, a point in the world can belong to different groups of different sizes. We optimize this field from a set of 2D masks provided by Segment Anything (SAM) in a way that respects coarse-to-fine hierarchy, using scale to consistently fuse conflicting masks from different viewpoints. From this field we can derive a hierarchy of possible groupings via automatic tree construction or user interaction. We evaluate GARField on a variety of in-the-wild scenes and find it effectively extracts groups at many levels: clusters of objects, objects, and various subparts. GARField inherently represents multi-view consistent groupings and produces higher fidelity groups than the input SAM masks. GARField's hierarchical grouping could have exciting downstream applications such as 3D asset extraction or dynamic scene understanding. See the project website at https://www.garfield.studio/

GARField：基于辐射场的任意物体分组

GARField: Group Anything with Radiance Fields

摘要

Support