跳到正文
原文
Nature 健康护理主题·· 29 天前AI 评分37

大语言模型用于多模态临床预测建模:系统综述

Large language models for multimodal clinical predictive modeling: a systematic review

AI 导读

一项系统综述筛选641项研究后纳入57项,评估大语言模型(LLM)在电子健康记录(EHR)多模态临床预测建模中的应用。该综述识别出嵌入、提取和文本融合三种LLM多模态整合策略,均较单模态模型带来性能提升(平均6.4%)。作者指出,相关建模需关注样本量、可解释性和计算成本。

正文

npj Digital Medicine (2026) Cite this article

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

The electronic health record (EHR) comprises structured and unstructured data, both of which are integral for comprehensively representing patient history. The emergence of large language models (LLMs) has enabled the incorporation of unstructured data modalities into machine learning models that leverage EHR data. In this systematic review, we characterize studies that utilize LLM-enabled multimodal predictive modeling with EHR data. Following a screening of 641 studies, 57 met inclusion criteria. We identified three strategies for LLM-based multimodal integration: embedding, extraction, and text fusion. Delineated by the format of the unstructured data at the time of fusion, all strategies yielded performance gains (average 6.4% increase) over single modality models. LLMs facilitated modeling by generating note-embeddings for downstream prediction, extracting clinically informative variables, and summarizing long text documents prior to integration. Building on our review, we identified LLM-enhanced predictive modeling considerations that include sample size, interpretability, and computational costs. This study supports the potential value of LLMs for EHR-based predictive modeling and provides a framework for future multimodal modeling tools.

Acknowledgements

We thank Drs. Linying Zhang, Laura Wiley, and Mackenzie Hofford (Institute for Informatics, Data Science, and Biostatistics at Washington University in St. Louis) for their valuable comments and suggestions.

Funding

This research received no specific grant funding. LK is supported by the Gilbert Family Foundation for related work.

Author information

Authors and Affiliations

  1. Institute for Informatics, Data Science and Biostatistics, Washington University School of Medicine, St. Louis, MO, USA

    Levi Kaster, Abigail E. Lewis, Philip R. O. Payne & Aditi Gupta

Authors

  1. Levi Kaster
  2. Abigail E. Lewis
  3. Philip R. O. Payne
  4. Aditi Gupta

Corresponding author

Correspondence to Aditi Gupta.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kaster, L., Lewis, A.E., Payne, P.R.O. et al. Large language models for multimodal clinical predictive modeling: a systematic review. npj Digit. Med. (2026). https://doi.org/10.1038/s41746-026-03349-z

Download citation

  • Received: 07 May 2026

  • Accepted: 21 September 2026

  • Published: 22 October 2026

  • DOI: https://doi.org/10.1038/s41746-026-03349-z

来源:Nature 健康护理主题 · nature.com