ChatPaper.aiChatPaper

UnMix-NeRF:光谱解混与神经辐射场的融合

UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields

June 27, 2025
作者: Fabian Perez, Sara Rojas, Carlos Hinojosa, Hoover Rueda-Chacón, Bernard Ghanem
cs.AI

摘要

基于神经辐射场(NeRF)的分割方法主要关注物体语义,并仅依赖RGB数据,缺乏内在材质属性。这一局限限制了精确的材质感知,而这对机器人技术、增强现实、仿真及其他应用至关重要。我们提出了UnMix-NeRF,一个将光谱解混技术融入NeRF的框架,实现了联合高光谱新视角合成与无监督材质分割。我们的方法通过漫反射和镜面反射分量建模光谱反射率,其中学习到的全局端元字典代表纯净材质特征,逐点丰度则捕捉其分布。对于材质分割,我们利用沿学习端元的光谱特征预测,实现无监督材质聚类。此外,UnMix-NeRF通过修改学习到的端元字典,支持场景编辑,实现灵活的基于材质的外观操控。大量实验验证了我们的方法,在光谱重建和材质分割方面均优于现有技术。项目页面:https://www.factral.co/UnMix-NeRF。
English
Neural Radiance Field (NeRF)-based segmentation methods focus on object semantics and rely solely on RGB data, lacking intrinsic material properties. This limitation restricts accurate material perception, which is crucial for robotics, augmented reality, simulation, and other applications. We introduce UnMix-NeRF, a framework that integrates spectral unmixing into NeRF, enabling joint hyperspectral novel view synthesis and unsupervised material segmentation. Our method models spectral reflectance via diffuse and specular components, where a learned dictionary of global endmembers represents pure material signatures, and per-point abundances capture their distribution. For material segmentation, we use spectral signature predictions along learned endmembers, allowing unsupervised material clustering. Additionally, UnMix-NeRF enables scene editing by modifying learned endmember dictionaries for flexible material-based appearance manipulation. Extensive experiments validate our approach, demonstrating superior spectral reconstruction and material segmentation to existing methods. Project page: https://www.factral.co/UnMix-NeRF.
PDF101July 8, 2025