多模态强化学习助力医学影像报告生成
Scaling medical imaging report generation with multimodal reinforcement learning
该研究引入了Universal Report Generation (UNIRG)框架,通过强化学习优化医学影像报告的临床意义质量标准。UNIRG-CXR模型在四个公开数据集上超越了先前模型,特别是在两个测试集上的表现提升了50%以上,同时减少了临床显著错误,保持了对未见过数据源和不同亚组的性能。
- Article
- Open access
- Published: 12 November 2026
- Qianchu Liu ORCID: orcid.org/0009-0000-2610-08681 na1,
- Sheng Zhang1 na1,
- Guanghui Qin ORCID: orcid.org/0000-0002-3009-86141 na1,
- Yu Gu1,
- Ying Jin1,
- Sam Preston1,
- Yanbo Xu1,
- Sid Kiblawi ORCID: orcid.org/0000-0001-6183-33541,
- Wen-wai Yim1,
- Timothy Ossowski1,
- Tristan Naumann ORCID: orcid.org/0000-0003-2150-17471,
- Mu Wei ORCID: orcid.org/0009-0000-4119-64901 &
- …
- Hoifung Poon ORCID: orcid.org/0000-0002-9067-09181
Nature Communications (2026) Cite this article
We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.
Abstract
In medical imaging report generation, supervised fine-tuning can overfit to superficial patterns and generalize poorly across institutions. We introduce Universal Report Generation (UNIRG), which adopts reinforcement learning to directly optimize clinically meaningful quality criteria. With this framework, we developed UNIRG-CXR, an 8-billion-parameter vision–language model trained on four public datasets (MIMIC-CXR, CheXpert Plus, ReXGradient-160k and IU-Xray) by supervised fine-tuning followed by reinforcement learning (group-relative policy optimization) against a composite lexical, semantic and clinical-error reward. We evaluated held-out test sets and an external proprietary dataset, with RadCliQ as primary metric and expert radiologist validation. Here we show that our method improves report quality and generalization: UNIRG-CXR surpasses prior models on the ReXrank benchmark across four datasets and two generation settings, exceeding the previous best score by over 50% on two test sets, and produces fewer clinically significant errors while maintaining performance on unseen data sources and across demographic subgroups.
Similar content being viewed by others



Acknowledgements
The authors acknowledge the patients and institutions whose contributions made the datasets used in this study possible. We also thank Harvard Medical School for hosting the ReXrank Radiology Report Generation Challenge and for supporting the evaluation of our model.
Funding
This research received no specific grant from any funding agency.
Author information
Author notes
These authors contributed equally: Qianchu Liu, Sheng Zhang, Guanghui Qin.
Authors and Affiliations
Microsoft Research, Redmond, WA, USA
Qianchu Liu, Sheng Zhang, Guanghui Qin, Yu Gu, Ying Jin, Sam Preston, Yanbo Xu, Sid Kiblawi, Wen-wai Yim, Timothy Ossowski, Tristan Naumann, Mu Wei & Hoifung Poon
Authors
- Qianchu Liu
- Sheng Zhang
- Guanghui Qin
- Yu Gu
- Ying Jin
- Sam Preston
- Yanbo Xu
- Sid Kiblawi
- Wen-wai Yim
- Timothy Ossowski
- Tristan Naumann
- Mu Wei
- Hoifung Poon
Corresponding authors
Correspondence to Mu Wei or Hoifung Poon.
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Liu, Q., Zhang, S., Qin, G. et al. Scaling medical imaging report generation with multimodal reinforcement learning. Nat Commun (2026). https://doi.org/10.1038/s41467-026-78767-4
Received: 03 June 2026
Accepted: 05 October 2026
Published: 12 November 2026
DOI: https://doi.org/10.1038/s41467-026-78767-4
来源:Nature 健康护理主题 · nature.com