UnMix-NeRF:光谱解混与神经辐射场的融合
UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields
June 27, 2025
作者: Fabian Perez, Sara Rojas, Carlos Hinojosa, Hoover Rueda-Chacón, Bernard Ghanem
cs.AI
摘要
基于神经辐射场(NeRF)的分割方法主要关注物体语义,并仅依赖RGB数据,缺乏内在材质属性。这一局限限制了精确的材质感知,而这对机器人技术、增强现实、仿真及其他应用至关重要。我们提出了UnMix-NeRF,一个将光谱解混技术融入NeRF的框架,实现了联合高光谱新视角合成与无监督材质分割。我们的方法通过漫反射和镜面反射分量建模光谱反射率,其中学习到的全局端元字典代表纯净材质特征,逐点丰度则捕捉其分布。对于材质分割,我们利用沿学习端元的光谱特征预测,实现无监督材质聚类。此外,UnMix-NeRF通过修改学习到的端元字典,支持场景编辑,实现灵活的基于材质的外观操控。大量实验验证了我们的方法,在光谱重建和材质分割方面均优于现有技术。项目页面:https://www.factral.co/UnMix-NeRF。
English
Neural Radiance Field (NeRF)-based segmentation methods focus on object
semantics and rely solely on RGB data, lacking intrinsic material properties.
This limitation restricts accurate material perception, which is crucial for
robotics, augmented reality, simulation, and other applications. We introduce
UnMix-NeRF, a framework that integrates spectral unmixing into NeRF, enabling
joint hyperspectral novel view synthesis and unsupervised material
segmentation. Our method models spectral reflectance via diffuse and specular
components, where a learned dictionary of global endmembers represents pure
material signatures, and per-point abundances capture their distribution. For
material segmentation, we use spectral signature predictions along learned
endmembers, allowing unsupervised material clustering. Additionally, UnMix-NeRF
enables scene editing by modifying learned endmember dictionaries for flexible
material-based appearance manipulation. Extensive experiments validate our
approach, demonstrating superior spectral reconstruction and material
segmentation to existing methods. Project page:
https://www.factral.co/UnMix-NeRF.