多中心评估大型语言模型生成膀胱癌mpMRI报告管理建议的效果
Multicenter benchmarking of large language models for bladder cancer management recommendations from mpMRI reports
研究评估了九种大型语言模型生成膀胱癌mpMRI报告管理建议的能力,发现翻译后的英文报告比原始中文报告在某些模型中表现更好,但也有模型表现不佳。研究强调了模型特定输入条件验证的重要性。
- Article
- Open access
- Published: 13 November 2026
- Zongyu Li ORCID: orcid.org/0009-0009-3867-80721 na1,
- Chengzhang Zhang1 na1,
- Xiaoni Chen2 na1,
- Liangqi Kang3 na1,
- Qinliang Si4 na1,
- Yiting Liu1,
- Yaxin Hou1,
- Zhenghao Liu ORCID: orcid.org/0000-0003-1185-598X1,
- Yuhong Ding1,
- Yang Li ORCID: orcid.org/0000-0003-4135-43341,
- Pengjie Shi1,
- Lina Yu5,6,
- Hongjie Shi7,
- Yingchun Kuang1,
- Yixian Li1,
- Junyi Hu1,
- Hailong Hu4,
- Zheng Liu1,
- Guang Zeng5,6,
- Chong Shen4,
- Ke Chen ORCID: orcid.org/0000-0002-2098-19211 &
- …
- Lilong Liu ORCID: orcid.org/0000-0001-5459-52041
npj Digital Medicine (2026) Cite this article
We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.
Abstract
Large language models (LLMs) can interpret radiology reports, but their ability to generate guideline-concordant recommendations from bladder cancer multiparametric MRI (mpMRI) reports remains uncertain. Following TRIPOD-LLM, we retrospectively evaluated nine API-accessed LLMs using 590 mpMRI reports from three hospitals under two input conditions: original Chinese reports and clinician-translated, quality-controlled English reports. Two physicians independently rated each model-generated recommendation on a 1–5 ordinal scale. We analyzed 21,240 ratings using patient-level random-intercept cumulative link mixed models, with paired Wilcoxon tests as secondary exploratory analyses. The association between input condition and physician rating differed across LLMs (likelihood-ratio chi-square = 683.05, df = 8, P < 0.001). Compared with original Chinese input, clinician-translated English input was associated with higher ratings for chatgpt-o3-pro (OR 2.79, 95% CI 2.29–3.41), deepseek-v4-flash (OR 1.46, 95% CI 1.19–1.81), claude-opus-4-7 (OR 3.07, 95% CI 2.47–3.82), and gpt-5.5 (OR 1.88, 95% CI 1.53–2.32), but with lower ratings for claude-sonnet-4-5 (OR 0.29, 95% CI 0.23–0.36; all Holm-adjusted P < 0.001). Four LLMs showed no clear input-condition difference. Center-specific contrasts also depended on input condition and were materially unchanged after adjustment for age, sex, pathological T stage, tumor grade, tumor number, and maximum lesion diameter. Inter-rater agreement was substantial (quadratic weighted kappa range, 0.621–0.743). Readability did not track physician-rated recommendation quality. These findings argue against clinician-translated English as a universal optimization strategy and support model-specific validation of input conditions before clinical deployment. LLM-generated recommendations from mpMRI reports are best positioned as clinician-supervised, imaging-based decision support rather than standalone treatment advice.
Funding
This research was supported by the National Natural Science Foundation of China (No. 82303623, No. 82373330) and the Noncommunicable Chronic Diseases-National Science and Technology Major Project (2024ZD0525700). The study's funder was not involved in the study design, data collection, data analysis, data interpretation, and decision to write or publish the manuscript.
Author information
Author notes
These authors contributed equally: Zongyu Li, Chengzhang Zhang, Xiaoni Chen, Liangqi Kang, Qinliang Si.
Authors and Affiliations
Department of Urology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, PR China
Zongyu Li, Chengzhang Zhang, Yiting Liu, Yaxin Hou, Zhenghao Liu, Yuhong Ding, Yang Li, Pengjie Shi, Yingchun Kuang, Yixian Li, Junyi Hu, Zheng Liu, Ke Chen & Lilong Liu
Department of Neurobiology, School of Basic Medicine, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, PR China
Xiaoni Chen
Department of Urology, Hubei Cancer Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, PR China
Liangqi Kang
Department of Urology, The Second Hospital of Tianjin Medical University, Tianjin, PR China
Qinliang Si, Hailong Hu & Chong Shen
Department of Urology, Zhongnan Hospital of Wuhan University, Wuhan, PR China
Lina Yu & Guang Zeng
Hubei Key Laboratory of Urological Diseases, Zhongnan Hospital of Wuhan University, Wuhan, PR China
Lina Yu & Guang Zeng
Department of Biological Repositories, Human Genetic Resources Preservation Center of Hubei Province, Zhongnan Hospital of Wuhan University, Wuhan, PR China
Hongjie Shi
Authors
- Zongyu Li
- Chengzhang Zhang
- Xiaoni Chen
- Liangqi Kang
- Qinliang Si
- Yiting Liu
- Yaxin Hou
- Zhenghao Liu
- Yuhong Ding
- Yang Li
- Pengjie Shi
- Lina Yu
- Hongjie Shi
- Yingchun Kuang
- Yixian Li
- Junyi Hu
- Hailong Hu
- Zheng Liu
- Guang Zeng
- Chong Shen
- Ke Chen
- Lilong Liu
Corresponding authors
Correspondence to Hailong Hu, Zheng Liu, Guang Zeng, Chong Shen, Ke Chen or Lilong Liu.
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Li, Z., Zhang, C., Chen, X. et al. Multicenter benchmarking of large language models for bladder cancer management recommendations from mpMRI reports. npj Digit. Med. (2026). https://doi.org/10.1038/s41746-026-03469-6
Received: 28 March 2026
Accepted: 19 October 2026
Published: 13 November 2026
DOI: https://doi.org/10.1038/s41746-026-03469-6
来源:Nature 健康护理主题 · nature.com