Zebra：用於解決參數化偏微分方程的上下文和生成預訓練

Zebra: In-Context and Generative Pretraining for Solving Parametric PDEs

October 4, 2024

作者: Louis Serrano, Armand Kassaï Koupaï, Thomas X Wang, Pierre Erbacher, Patrick Gallinari

cs.AI

摘要

解決時間相依性參數偏微分方程（PDEs）是具有挑戰性的，因為模型必須適應參數的變化，如係數、強制項和邊界條件。基於數據的神經求解器通常通過訓練來自PDE參數分佈的數據，希望模型能推廣到新實例，或依賴基於梯度的適應和元學習來從觀察中隱式編碼動態性。然而，這通常會帶來增加的推論複雜性。受大型語言模型（LLMs）在上下文學習能力的啟發，我們引入了Zebra，一種新穎的生成自回歸變壓器，旨在解決參數化PDEs，而無需在推論時進行梯度適應。通過在預訓練和推論期間利用上下文信息，Zebra通過條件化於包含上下文軌跡或前狀態的輸入序列，動態適應新任務。這種方法使Zebra能靈活處理任意大小的上下文輸入，並通過多個解軌跡的抽樣支持不確定性量化。我們在各種具有挑戰性的PDE情境中評估了Zebra，展示了其適應性、穩健性和優越性能，相較於現有方法。

English

Solving time-dependent parametric partial differential equations (PDEs) is challenging, as models must adapt to variations in parameters such as coefficients, forcing terms, and boundary conditions. Data-driven neural solvers either train on data sampled from the PDE parameters distribution in the hope that the model generalizes to new instances or rely on gradient-based adaptation and meta-learning to implicitly encode the dynamics from observations. This often comes with increased inference complexity. Inspired by the in-context learning capabilities of large language models (LLMs), we introduce Zebra, a novel generative auto-regressive transformer designed to solve parametric PDEs without requiring gradient adaptation at inference. By leveraging in-context information during both pre-training and inference, Zebra dynamically adapts to new tasks by conditioning on input sequences that incorporate context trajectories or preceding states. This approach enables Zebra to flexibly handle arbitrarily sized context inputs and supports uncertainty quantification through the sampling of multiple solution trajectories. We evaluate Zebra across a variety of challenging PDE scenarios, demonstrating its adaptability, robustness, and superior performance compared to existing approaches.