跳到正文
原文
Nature 健康护理主题·· 7 天前AI 评分48

多中心评估大型语言模型生成膀胱癌mpMRI报告管理建议的效果

Multicenter benchmarking of large language models for bladder cancer management recommendations from mpMRI reports

AI 导读

研究评估了九种大型语言模型生成膀胱癌mpMRI报告管理建议的能力,发现翻译后的英文报告比原始中文报告在某些模型中表现更好,但也有模型表现不佳。研究强调了模型特定输入条件验证的重要性。

正文

npj Digital Medicine (2026) Cite this article

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

Large language models (LLMs) can interpret radiology reports, but their ability to generate guideline-concordant recommendations from bladder cancer multiparametric MRI (mpMRI) reports remains uncertain. Following TRIPOD-LLM, we retrospectively evaluated nine API-accessed LLMs using 590 mpMRI reports from three hospitals under two input conditions: original Chinese reports and clinician-translated, quality-controlled English reports. Two physicians independently rated each model-generated recommendation on a 1–5 ordinal scale. We analyzed 21,240 ratings using patient-level random-intercept cumulative link mixed models, with paired Wilcoxon tests as secondary exploratory analyses. The association between input condition and physician rating differed across LLMs (likelihood-ratio chi-square = 683.05, df = 8, P < 0.001). Compared with original Chinese input, clinician-translated English input was associated with higher ratings for chatgpt-o3-pro (OR 2.79, 95% CI 2.29–3.41), deepseek-v4-flash (OR 1.46, 95% CI 1.19–1.81), claude-opus-4-7 (OR 3.07, 95% CI 2.47–3.82), and gpt-5.5 (OR 1.88, 95% CI 1.53–2.32), but with lower ratings for claude-sonnet-4-5 (OR 0.29, 95% CI 0.23–0.36; all Holm-adjusted P < 0.001). Four LLMs showed no clear input-condition difference. Center-specific contrasts also depended on input condition and were materially unchanged after adjustment for age, sex, pathological T stage, tumor grade, tumor number, and maximum lesion diameter. Inter-rater agreement was substantial (quadratic weighted kappa range, 0.621–0.743). Readability did not track physician-rated recommendation quality. These findings argue against clinician-translated English as a universal optimization strategy and support model-specific validation of input conditions before clinical deployment. LLM-generated recommendations from mpMRI reports are best positioned as clinician-supervised, imaging-based decision support rather than standalone treatment advice.

Funding

This research was supported by the National Natural Science Foundation of China (No. 82303623, No. 82373330) and the Noncommunicable Chronic Diseases-National Science and Technology Major Project (2024ZD0525700). The study's funder was not involved in the study design, data collection, data analysis, data interpretation, and decision to write or publish the manuscript.

Author information

Author notes

  1. These authors contributed equally: Zongyu Li, Chengzhang Zhang, Xiaoni Chen, Liangqi Kang, Qinliang Si.

Authors and Affiliations

  1. Department of Urology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, PR China

    Zongyu Li, Chengzhang Zhang, Yiting Liu, Yaxin Hou, Zhenghao Liu, Yuhong Ding, Yang Li, Pengjie Shi, Yingchun Kuang, Yixian Li, Junyi Hu, Zheng Liu, Ke Chen & Lilong Liu

  2. Department of Neurobiology, School of Basic Medicine, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, PR China

    Xiaoni Chen

  3. Department of Urology, Hubei Cancer Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, PR China

    Liangqi Kang

  4. Department of Urology, The Second Hospital of Tianjin Medical University, Tianjin, PR China

    Qinliang Si, Hailong Hu & Chong Shen

  5. Department of Urology, Zhongnan Hospital of Wuhan University, Wuhan, PR China

    Lina Yu & Guang Zeng

  6. Hubei Key Laboratory of Urological Diseases, Zhongnan Hospital of Wuhan University, Wuhan, PR China

    Lina Yu & Guang Zeng

  7. Department of Biological Repositories, Human Genetic Resources Preservation Center of Hubei Province, Zhongnan Hospital of Wuhan University, Wuhan, PR China

    Hongjie Shi

Authors

  1. Zongyu Li
  2. Chengzhang Zhang
  3. Xiaoni Chen
  4. Liangqi Kang
  5. Qinliang Si
  6. Yiting Liu
  7. Yaxin Hou
  8. Zhenghao Liu
  9. Yuhong Ding
  10. Yang Li
  11. Pengjie Shi
  12. Lina Yu
  13. Hongjie Shi
  14. Yingchun Kuang
  15. Yixian Li
  16. Junyi Hu
  17. Hailong Hu
  18. Zheng Liu
  19. Guang Zeng
  20. Chong Shen
  21. Ke Chen
  22. Lilong Liu

Corresponding authors

Correspondence to Hailong Hu, Zheng Liu, Guang Zeng, Chong Shen, Ke Chen or Lilong Liu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Z., Zhang, C., Chen, X. et al. Multicenter benchmarking of large language models for bladder cancer management recommendations from mpMRI reports. npj Digit. Med. (2026). https://doi.org/10.1038/s41746-026-03469-6

Download citation

  • Received: 28 March 2026

  • Accepted: 19 October 2026

  • Published: 13 November 2026

  • DOI: https://doi.org/10.1038/s41746-026-03469-6

来源:Nature 健康护理主题 · nature.com