SHINOBI：通過BRDF優化在野外使用神經對象分解的形狀和照明

摘要

我們提出了SHINOBI，一個端到端的框架，用於從捕獲具有不同照明、姿勢和背景的物體圖像中重建形狀、材質和照明。基於無限制圖像集合的物體的反渲染是計算機視覺和圖形領域中一個長期存在的挑戰，需要對形狀、輻射和姿勢進行聯合優化。我們展示了基於多分辨率哈希編碼的隱式形狀表示，可以實現更快速、更穩健的形狀重建，並通過聯合相機對齊優化來超越先前的工作。此外，為了實現對照明和物體反射（即材質）的編輯，我們聯合優化BRDF和照明，以及物體的形狀。我們的方法是與類別無關的，可應用於野外物體圖像集合，以生成可重新照明的3D資產，適用於AR/VR、電影、遊戲等多種用例。專案頁面：https://shinobi.aengelhardt.com 影片：https://www.youtube.com/watch?v=iFENQ6AcYd8&feature=youtu.be

English

We present SHINOBI, an end-to-end framework for the reconstruction of shape, material, and illumination from object images captured with varying lighting, pose, and background. Inverse rendering of an object based on unconstrained image collections is a long-standing challenge in computer vision and graphics and requires a joint optimization over shape, radiance, and pose. We show that an implicit shape representation based on a multi-resolution hash encoding enables faster and robust shape reconstruction with joint camera alignment optimization that outperforms prior work. Further, to enable the editing of illumination and object reflectance (i.e. material) we jointly optimize BRDF and illumination together with the object's shape. Our method is class-agnostic and works on in-the-wild image collections of objects to produce relightable 3D assets for several use cases such as AR/VR, movies, games, etc. Project page: https://shinobi.aengelhardt.com Video: https://www.youtube.com/watch?v=iFENQ6AcYd8&feature=youtu.be

SHINOBI：通過BRDF優化在野外使用神經對象分解的形狀和照明

SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild

摘要

Support