NOAH：學習完整的病患旅程。一個用於表徵與預測的縱貫性多模態時間感知模型

摘要

醫療照護數位化已產生病患一生中龐大、縱貫性且多模態的病患紀錄，然而，充分運用這些資料以表徵並預測病患狀態軌跡仍是一項關鍵挑戰。當前人工智慧模型往往難以捕捉真實世界多模態病患資料中複雜、不規則的時間動態及固有隨機性。現有針對縱貫性病患紀錄建模的人工智慧方法主要為判別式，僅限於少數模態，受封閉式類別詞彙所限，將時間視為單調歸納偏誤，或其預測未來病患狀態的能力有限。我們提出 NOAH，一個時間感知、任務無關的生成式 Transformer 模型，用以表徵並預測完整的多模態病患歷程。NOAH 具備新穎的雙向時間整合與變分潛在空間，以捕捉病患狀態的連續演變及臨床軌跡的隨機性。NOAH 建構自橫跨 MIMIC 資料集家族、來自 299,000 名病患 431,000 次住院的超過 5.59 億筆臨床事件，能原生處理醫學影像、時間序列與數值訊號、類別事件，以及結構化與非結構化臨床紀錄。NOAH 是該領域首個真正整體性的生成式模型，能進行具可選時間控制的自迴歸預測、零樣本分類與反事實介入模擬。它產生具高度資訊量與預測力的病患狀態表徵，在探測臨床結果、15 個 ICD 章節與 29 種共病，以及事件發生時間預測上展現優異效能。無縫處理多元模態與複雜時間動態，NOAH 為個人化臨床照護與數位醫療中的智慧預測系統提供多用途、任務無關且可擴展的基礎。

English

The digitization of healthcare has generated vast, longitudinal, and multimodal patient records over a lifetime, yet fully exploiting these data to represent and predict patient state trajectories remains a critical challenge. Current AI models often struggle to capture the complex, irregular temporal dynamics and inherent stochasticity of real-world multimodal patient data. Existing AI approaches for modeling longitudinal patient records are predominantly discriminative, limited to a few modalities, constrained by closed categorical vocabularies, treating time as a monotonic inductive bias, or they are limited in forecasting future patient states. We introduce NOAH, a time-aware, task-agnostic, generative transformer model representing and forecasting the full multimodal patient journey. NOAH features a novel bidirectional time integration and a variational latent space to capture the continuous evolution of patient states and the stochasticity of clinical trajectories. Built from over 559 million clinical events from 431,000 hospital visits of 299,000 patients across the MIMIC dataset family, NOAH natively processes medical images, time-series and numeric signals, categorical events, as well as structured and unstructured clinical records. NOAH is the first truly holistic generative model in its field, enabling autoregressive forecasting with optional time control, zero-shot classification, and counterfactual intervention simulation. It generates highly informative and predictive patient state representations that demonstrate strong performance in probing for clinical outcomes, 15 ICD chapters, and 29 comorbidities, as well as in time-to-event prediction. Seamlessly handling diverse modalities and complex temporal dynamics, NOAH provides a versatile, task-agnostic, scalable foundation for intelligent predictive systems in personalized clinical care and digital medicine.