Human101：在100秒內從一個視角訓練100+FPS的人體高斯模型

摘要

從單視角視頻重建人體在虛擬現實領域中發揮著關鍵作用。一個普遍的應用場景需要快速重建高保真度的3D數字人類，同時確保實時渲染和交互。現有方法往往難以滿足這兩個要求。本文介紹了Human101，一個新穎的框架，能夠從單視角視頻中訓練3D高斯模型並在100秒內進行渲染，達到100+ FPS。我們的方法利用3D高斯飛濺的優勢，提供了對3D人體的明確且高效的表示。Human101與先前基於NeRF的流程有所不同，巧妙地應用了以人為中心的前向高斯動畫方法來變形3D高斯模型的參數，從而提高渲染速度（即以令人印象深刻的60+ FPS渲染1024分辨率圖像，以及以100+ FPS渲染512分辨率圖像）。實驗結果表明，我們的方法顯著超越了當前方法，每秒幀數增加了多達10倍，並提供可比擬或更優質的渲染質量。代碼和演示將在https://github.com/longxiang-ai/Human101 上發布。

English

Reconstructing the human body from single-view videos plays a pivotal role in the virtual reality domain. One prevalent application scenario necessitates the rapid reconstruction of high-fidelity 3D digital humans while simultaneously ensuring real-time rendering and interaction. Existing methods often struggle to fulfill both requirements. In this paper, we introduce Human101, a novel framework adept at producing high-fidelity dynamic 3D human reconstructions from 1-view videos by training 3D Gaussians in 100 seconds and rendering in 100+ FPS. Our method leverages the strengths of 3D Gaussian Splatting, which provides an explicit and efficient representation of 3D humans. Standing apart from prior NeRF-based pipelines, Human101 ingeniously applies a Human-centric Forward Gaussian Animation method to deform the parameters of 3D Gaussians, thereby enhancing rendering speed (i.e., rendering 1024-resolution images at an impressive 60+ FPS and rendering 512-resolution images at 100+ FPS). Experimental results indicate that our approach substantially eclipses current methods, clocking up to a 10 times surge in frames per second and delivering comparable or superior rendering quality. Code and demos will be released at https://github.com/longxiang-ai/Human101.

Human101：在100秒內從一個視角訓練100+FPS的人體高斯模型

Human101: Training 100+FPS Human Gaussians in 100s from 1 View

摘要

Support