SHINOBI：通过BRDF优化在野外使用神经对象分解进行形状和光照

摘要

我们提出了SHINOBI，这是一个端到端的框架，用于从以不同光照、姿势和背景捕获的物体图像中重建形状、材质和光照。基于无约束图像集合的物体的逆渲染是计算机视觉和图形领域长期以来的挑战，需要对形状、辐射和姿势进行联合优化。我们展示了基于多分辨率哈希编码的隐式形状表示，可以实现更快速、更稳健的形状重建，通过联合相机对准优化，优于先前的工作。此外，为了实现对光照和物体反射（即材质）的编辑，我们联合优化BRDF和光照以及物体的形状。我们的方法是无类别的，并且适用于野外图像集合中的物体，可用于生成可重新照明的3D资产，适用于AR/VR、电影、游戏等多种用例。项目页面：https://shinobi.aengelhardt.com 视频：https://www.youtube.com/watch?v=iFENQ6AcYd8&feature=youtu.be

English

We present SHINOBI, an end-to-end framework for the reconstruction of shape, material, and illumination from object images captured with varying lighting, pose, and background. Inverse rendering of an object based on unconstrained image collections is a long-standing challenge in computer vision and graphics and requires a joint optimization over shape, radiance, and pose. We show that an implicit shape representation based on a multi-resolution hash encoding enables faster and robust shape reconstruction with joint camera alignment optimization that outperforms prior work. Further, to enable the editing of illumination and object reflectance (i.e. material) we jointly optimize BRDF and illumination together with the object's shape. Our method is class-agnostic and works on in-the-wild image collections of objects to produce relightable 3D assets for several use cases such as AR/VR, movies, games, etc. Project page: https://shinobi.aengelhardt.com Video: https://www.youtube.com/watch?v=iFENQ6AcYd8&feature=youtu.be

SHINOBI：通过BRDF优化在野外使用神经对象分解进行形状和光照

SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild

摘要

Support