GARField：使用輻射場將任何物體分組

摘要

由於可以將場景分解為多個層次，因此分組在本質上是含糊的 - 挖掘機的輪子應該被視為獨立的還是整體的一部分？我們提出了一種名為Radiance Fields的Group Anything with Radiance Fields (GARField)方法，用於從姿態圖像輸入中將3D場景分解為具有語義意義的組的層次結構。為了做到這一點，我們通過物理尺度來接受組模糊性：通過優化一個與尺度相關的3D親和特徵場，世界中的一個點可以屬於不同尺寸的不同組。我們從由Segment Anything (SAM)提供的一組2D遮罩中優化這個場，以一種尊重從粗到細層次結構的方式，利用尺度來一致地融合來自不同視角的衝突遮罩。通過這個場，我們可以通過自動樹構造或用戶交互來推導可能分組的層次結構。我們在各種野外場景上評估了GARField，並發現它有效地提取了許多層次的組：對象的聚集、對象和各種子部分。GARField本質上代表了多視角一致的分組，並且比輸入的SAM遮罩產生了更高保真度的組。GARField的分層分組可能具有令人興奮的下游應用，例如3D資產提取或動態場景理解。請參閱項目網站：https://www.garfield.studio/

English

Grouping is inherently ambiguous due to the multiple levels of granularity in which one can decompose a scene -- should the wheels of an excavator be considered separate or part of the whole? We present Group Anything with Radiance Fields (GARField), an approach for decomposing 3D scenes into a hierarchy of semantically meaningful groups from posed image inputs. To do this we embrace group ambiguity through physical scale: by optimizing a scale-conditioned 3D affinity feature field, a point in the world can belong to different groups of different sizes. We optimize this field from a set of 2D masks provided by Segment Anything (SAM) in a way that respects coarse-to-fine hierarchy, using scale to consistently fuse conflicting masks from different viewpoints. From this field we can derive a hierarchy of possible groupings via automatic tree construction or user interaction. We evaluate GARField on a variety of in-the-wild scenes and find it effectively extracts groups at many levels: clusters of objects, objects, and various subparts. GARField inherently represents multi-view consistent groupings and produces higher fidelity groups than the input SAM masks. GARField's hierarchical grouping could have exciting downstream applications such as 3D asset extraction or dynamic scene understanding. See the project website at https://www.garfield.studio/

GARField：使用輻射場將任何物體分組

GARField: Group Anything with Radiance Fields

摘要

Support