光速：輕巧快速的神經光場在行動裝置上

摘要

由於移動設備的有限計算能力和存儲容量，實時在移動設備上進行新視角圖像合成是困難的。在移動設備上使用體積渲染方法，如 NeRF 及其衍生物，並不適合，因為體積渲染的計算成本高。另一方面，最近神經光場表示法的進步展示了在移動設備上有前景的實時視角合成結果。神經光場方法學習從射線表示到像素顏色的直接映射。目前的射線表示選擇為分層射線採樣或 Pl\"{u}cker 坐標，忽略了經典的光板（雙平面）表示，這是在光場視角之間插值的首選表示。在這項工作中，我們發現使用光板表示對於學習神經光場是一種高效的表示。更重要的是，這是一種低維度的射線表示，使我們能夠使用特徵網格學習 4D 射線空間，這樣訓練和渲染速度顯著更快。儘管主要設計用於正面視圖，我們展示了光板表示可以通過分治策略進一步擴展到非正面場景。我們的方法提供了比以前的光場方法更優越的渲染質量，並實現了在渲染質量和速度之間顯著改進的折衷。

English

Real-time novel-view image synthesis on mobile devices is prohibitive due to the limited computational power and storage. Using volumetric rendering methods, such as NeRF and its derivatives, on mobile devices is not suitable due to the high computational cost of volumetric rendering. On the other hand, recent advances in neural light field representations have shown promising real-time view synthesis results on mobile devices. Neural light field methods learn a direct mapping from a ray representation to the pixel color. The current choice of ray representation is either stratified ray sampling or Pl\"{u}cker coordinates, overlooking the classic light slab (two-plane) representation, the preferred representation to interpolate between light field views. In this work, we find that using the light slab representation is an efficient representation for learning a neural light field. More importantly, it is a lower-dimensional ray representation enabling us to learn the 4D ray space using feature grids which are significantly faster to train and render. Although mostly designed for frontal views, we show that the light-slab representation can be further extended to non-frontal scenes using a divide-and-conquer strategy. Our method offers superior rendering quality compared to previous light field methods and achieves a significantly improved trade-off between rendering quality and speed.

光速：輕巧快速的神經光場在行動裝置上

LightSpeed: Light and Fast Neural Light Fields on Mobile Devices

摘要

Support