WildCity: 一个面向渲染、仿真与空间智能的真实世界城市级测试平台

摘要

人类能够在陌生城市中穿行，并逐步形成覆盖数十平方公里范围的连贯空间心智地图。人工智能能否构建同等规模的空间表征？尽管近期的基础模型在场景重建与具身智能领域取得了进展，但扩展到整个城市规模仍是一个尚未解决的挑战，主要障碍在于缺乏城市级数据。为弥合这一差距，我们提出WildCity——一个由自动驾驶车队在复杂城市环境中采集的真实世界多模态数据集。该数据集包含18条轨迹，平均每条长度达83.7公里，并保留了野外感知的核心挑战，例如动态物体、光照变化以及不完美的相机位姿。我们进一步构建了面向城市场景的重建基准，并将重建环境转化为闭环仿真器。除数据集与基准外，我们系统分析了迈向可仿真城市数字孪生过程中的关键挑战：可扩展性、外推能力与不确定性。最终，WildCity旨在不仅推动城市级渲染的进步，更广泛地促进AI在感知、记忆与空间推理方面达到与人类认知相媲美的规模。项目主页：https://han-xiangyu.github.io/Wild-City/

English

Humans can navigate an unfamiliar city and gradually form a coherent spatial mental map spanning tens of square kilometers. Can AI build spatial representations at a comparable scale? Although recent foundation models have advanced scene reconstruction and embodied intelligence, scaling to entire cities remains an open challenge, primarily due to the lack of city-scale data. To bridge the gap, we introduce WildCity, a real-world multimodal dataset collected by autonomous fleets traversing complex urban environments. Our dataset includes 18 trajectories, each averaging 83.7 kilometers in length, and preserves the core challenges of in-the-wild perception, e.g., dynamic objects, lighting variations, and imperfect camera poses. We further establish an urban-tailored reconstruction baseline and convert the reconstructed environments into a closed-loop simulator. Beyond the dataset and baseline, we systematically analyze the key challenges on the path to simulation-ready urban digital twins: scalability, extrapolation, and uncertainty. Ultimately, WildCity aims to catalyze progress not only in city-scale rendering, but more broadly in the pursuit of AI that can perceive, remember, and reason across space at a scale comparable to human cognition. Project page: https://han-xiangyu.github.io/Wild-City/