跳到正文
原文
Nature 健康护理主题·· 2026-10-14AI 评分43

Dr.Copilot 多语言检索增强型临床 AI 框架的实证评估

Real-world evaluation of a multilingual agentic retrieval-augmented clinical AI framework for evidence-grounded medical decision support

AI 导读

Dr.Copilot 多语言检索增强型临床 AI 框架在 70 天部署中记录 956 个会话线程及 4,226 次工具调用事件。在独立 75 例外部验证中,Dr.Copilot 加权得分 4.557,显著优于 GPT-5.5 的 4.217(p = 0.00023),且临床重要错误率更低(5/75 vs. 16/75,p = 0.00098)。

正文

Scientific Reports (2026) Cite this article

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

Large language models may support clinical information tasks, but hallucination, limited evidence traceability, and inconsistent workflow integration remain barriers to clinical deployment. We evaluated Dr.Copilot, a multilingual retrieval-augmented clinical AI framework combining hybrid evidence retrieval, specialized tools, and model-directed orchestration. During a 70-day deployment, 835 registered users generated 956 conversation threads; the analytics log contained 22,574 events and 4,226 tool-use events. In the original 64-prompt expert benchmark, Dr.Copilot and GPT-5.5 had comparable weighted scores (4.607 vs. 4.561; paired difference 0.046, 95% CI -0.064 to 0.140; p = 0.103). In a new independent 75-case, fixed-backbone external validation, Dr.Copilot scored 4.557 versus 4.217 for GPT-5.5 (difference 0.340, 95% CI 0.181 to 0.519; p = 0.00023), with fewer clinically important errors (5/75 vs. 16/75; p = 0.00098). A 30-case four-arm GPT-5.4 ablation found the highest mean score with standard RAG (4.813), while additional prespecified-tool and model-directed routing conditions did not yield uniform further gains. Targeted safety, pediatric, multilingual, citation, temporal, and abstention analyses identified both strengths and important failure modes. These findings support retrieval as the clearest incremental contributor in this evaluation, while underscoring that agentic tool orchestration requires direct validation and does not itself establish clinical benefit or patient-safety improvement.

Acknowledgements

The authors sincerely thank the internal medicine specialist and clinical pharmacist who independently and blindly evaluated the benchmark responses.

Funding

This study was supported by the Vice-Chancellorship for Research and Technology, Shiraz University of Medical Sciences (SUMS).

Author information

Authors and Affiliations

  1. Student Research Committee, School of Medicine, Shiraz University of Medical Sciences, Shiraz, Iran

    Mohammad Hossein Imani

  2. Student Research Committee, School of Pharmacy, Shiraz University of Medical Sciences, Shiraz, Iran

    Mohammad Hasannejad

  3. Department of Radiology, Shiraz University of Medical Sciences, Shiraz, Iran

    Amir Hossein Imani

  4. Department of Clinical Pharmacy, Faculty of Pharmacy, Shiraz University of Medical Sciences, Shiraz, Iran

    Mojtaba Shafiekhani

Authors

  1. Mohammad Hossein Imani
  2. Mohammad Hasannejad
  3. Amir Hossein Imani
  4. Mojtaba Shafiekhani

Corresponding author

Correspondence to Mojtaba Shafiekhani.

Ethics declarations

Competing interests

Mohammad Hossein Imani and Mohammad Hasannejad are members of the Dr.Copilot development team. Amir Hossein Imani and Mojtaba Shafiekhani declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Imani, M.H., Hasannejad, M., Imani, A.H. et al. Real-world evaluation of a multilingual agentic retrieval-augmented clinical AI framework for evidence-grounded medical decision support. Sci Rep (2026). https://doi.org/10.1038/s41598-026-75369-4

Download citation

  • Received: 26 June 2026

  • Accepted: 06 October 2026

  • Published: 14 October 2026

  • DOI: https://doi.org/10.1038/s41598-026-75369-4

Keywords

来源:Nature 健康护理主题 · nature.com