LightSpeed：轻巧快速的神经光场在移动设备上

摘要

由于移动设备的有限计算能力和存储空间，实时在移动设备上进行新视角图像合成是困难的。在移动设备上使用体积渲染方法，如NeRF及其衍生物，由于体积渲染的高计算成本，不太适用。另一方面，最近神经光场表示方法的进展展示了在移动设备上有希望的实时视角合成结果。神经光场方法学习了从射线表示到像素颜色的直接映射。目前射线表示的选择要么是分层射线采样，要么是Pl\"{u}cker坐标，忽视了经典的光板（双平面）表示，这是在光场视角之间插值的首选表示。在这项工作中，我们发现使用光板表示是学习神经光场的有效表示。更重要的是，这是一个低维射线表示，使我们能够使用特征网格学习4D射线空间，这样训练和渲染速度显著更快。尽管大多数设计用于前视图，我们展示了光板表示可以通过分而治之策略进一步扩展到非前景场景。我们的方法相比先前的光场方法提供了更优质的渲染质量，并实现了在渲染质量和速度之间显著改进的折衷。

English

Real-time novel-view image synthesis on mobile devices is prohibitive due to the limited computational power and storage. Using volumetric rendering methods, such as NeRF and its derivatives, on mobile devices is not suitable due to the high computational cost of volumetric rendering. On the other hand, recent advances in neural light field representations have shown promising real-time view synthesis results on mobile devices. Neural light field methods learn a direct mapping from a ray representation to the pixel color. The current choice of ray representation is either stratified ray sampling or Pl\"{u}cker coordinates, overlooking the classic light slab (two-plane) representation, the preferred representation to interpolate between light field views. In this work, we find that using the light slab representation is an efficient representation for learning a neural light field. More importantly, it is a lower-dimensional ray representation enabling us to learn the 4D ray space using feature grids which are significantly faster to train and render. Although mostly designed for frontal views, we show that the light-slab representation can be further extended to non-frontal scenes using a divide-and-conquer strategy. Our method offers superior rendering quality compared to previous light field methods and achieves a significantly improved trade-off between rendering quality and speed.

LightSpeed：轻巧快速的神经光场在移动设备上

LightSpeed: Light and Fast Neural Light Fields on Mobile Devices

摘要

Support