神经脉冲响应场的声学体绘制

摘要

为了在虚拟和增强现实中创造沉浸式体验，捕捉准确的声学现象的真实音频合成至关重要。合成在任何位置接收到的声音依赖于脉冲响应（IR）的估计，IR表征了声音在一个场景中沿不同路径传播到达听者位置之前的方式。在本文中，我们提出声学体积渲染（AVR），这是一种将体积渲染技术应用于建模声学脉冲响应的新方法。虽然体积渲染在建模图像和神经场景表示的辐射场方面取得了成功，但IR作为时间序列信号具有独特的挑战。为了解决这些挑战，我们引入了频域体积渲染，并使用球形积分来拟合IR测量。我们的方法构建了一个脉冲响应场，从根本上编码了波传播原理，并在合成新姿势的脉冲响应方面实现了最先进的性能。实验证明AVR在很大程度上超越了当前领先方法。此外，我们开发了一个声学模拟平台AcoustiX，提供比现有模拟器更准确和逼真的IR模拟。AVR和AcoustiX的代码可在https://zitonglan.github.io/avr 上找到。

English

Realistic audio synthesis that captures accurate acoustic phenomena is essential for creating immersive experiences in virtual and augmented reality. Synthesizing the sound received at any position relies on the estimation of impulse response (IR), which characterizes how sound propagates in one scene along different paths before arriving at the listener's position. In this paper, we present Acoustic Volume Rendering (AVR), a novel approach that adapts volume rendering techniques to model acoustic impulse responses. While volume rendering has been successful in modeling radiance fields for images and neural scene representations, IRs present unique challenges as time-series signals. To address these challenges, we introduce frequency-domain volume rendering and use spherical integration to fit the IR measurements. Our method constructs an impulse response field that inherently encodes wave propagation principles and achieves state-of-the-art performance in synthesizing impulse responses for novel poses. Experiments show that AVR surpasses current leading methods by a substantial margin. Additionally, we develop an acoustic simulation platform, AcoustiX, which provides more accurate and realistic IR simulations than existing simulators. Code for AVR and AcoustiX are available at https://zitonglan.github.io/avr.

神经脉冲响应场的声学体绘制

Acoustic Volume Rendering for Neural Impulse Response Fields

摘要

Support