跳到正文
原文
Nature 健康护理主题·· 8 天前AI 评分40

多模态强化学习助力医学影像报告生成

Scaling medical imaging report generation with multimodal reinforcement learning

AI 导读

该研究引入了Universal Report Generation (UNIRG)框架,通过强化学习优化医学影像报告的临床意义质量标准。UNIRG-CXR模型在四个公开数据集上超越了先前模型,特别是在两个测试集上的表现提升了50%以上,同时减少了临床显著错误,保持了对未见过数据源和不同亚组的性能。

正文

Nature Communications (2026) Cite this article

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

In medical imaging report generation, supervised fine-tuning can overfit to superficial patterns and generalize poorly across institutions. We introduce Universal Report Generation (UNIRG), which adopts reinforcement learning to directly optimize clinically meaningful quality criteria. With this framework, we developed UNIRG-CXR, an 8-billion-parameter vision–language model trained on four public datasets (MIMIC-CXR, CheXpert Plus, ReXGradient-160k and IU-Xray) by supervised fine-tuning followed by reinforcement learning (group-relative policy optimization) against a composite lexical, semantic and clinical-error reward. We evaluated held-out test sets and an external proprietary dataset, with RadCliQ as primary metric and expert radiologist validation. Here we show that our method improves report quality and generalization: UNIRG-CXR surpasses prior models on the ReXrank benchmark across four datasets and two generation settings, exceeding the previous best score by over 50% on two test sets, and produces fewer clinically significant errors while maintaining performance on unseen data sources and across demographic subgroups.

Similar content being viewed by others

Acknowledgements

The authors acknowledge the patients and institutions whose contributions made the datasets used in this study possible. We also thank Harvard Medical School for hosting the ReXrank Radiology Report Generation Challenge and for supporting the evaluation of our model.

Funding

This research received no specific grant from any funding agency.

Author information

Author notes

  1. These authors contributed equally: Qianchu Liu, Sheng Zhang, Guanghui Qin.

Authors and Affiliations

  1. Microsoft Research, Redmond, WA, USA

    Qianchu Liu, Sheng Zhang, Guanghui Qin, Yu Gu, Ying Jin, Sam Preston, Yanbo Xu, Sid Kiblawi, Wen-wai Yim, Timothy Ossowski, Tristan Naumann, Mu Wei & Hoifung Poon

Authors

  1. Qianchu Liu
  2. Sheng Zhang
  3. Guanghui Qin
  4. Yu Gu
  5. Ying Jin
  6. Sam Preston
  7. Yanbo Xu
  8. Sid Kiblawi
  9. Wen-wai Yim
  10. Timothy Ossowski
  11. Tristan Naumann
  12. Mu Wei
  13. Hoifung Poon

Corresponding authors

Correspondence to Mu Wei or Hoifung Poon.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Q., Zhang, S., Qin, G. et al. Scaling medical imaging report generation with multimodal reinforcement learning. Nat Commun (2026). https://doi.org/10.1038/s41467-026-78767-4

Download citation

  • Received: 03 June 2026

  • Accepted: 05 October 2026

  • Published: 12 November 2026

  • DOI: https://doi.org/10.1038/s41467-026-78767-4

来源:Nature 健康护理主题 · nature.com