跳到正文
原文
Nature 健康护理主题·· 19 天前AI 评分51

研究评估ChatGPT、Gemini、Copilot、Claude与DeepSeek对股骨头坏死患者提问的回答质量

Evaluation of large language models’ responses to patient questions about avascular necrosis of the femoral head

AI 导读

四名骨科医生对ChatGPT、Gemini、Copilot、Claude与DeepSeek回答15个股骨头坏死患者常见问题共75条回答评分,Claude的JAMA与DISCERN得分最高,DeepSeek的DISCERN优于ChatGPT、Gemini和Copilot。

正文

Scientific Reports (2026) Cite this article

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

Artificial intelligence chatbots are rapidly gaining popularity for acquiring health information, but the quality of responses seems to differ between platforms. This study investigated the responses of ChatGPT, Gemini, Copilot, Claude, and DeepSeek to 15 frequently asked questions regarding femoral head avascular necrosis. Each platform answered each question once, for a total of 75 responses. Four orthopedic surgeons assessed the response using the JAMA benchmark criteria, DISCERN, an adapted five-point scale of clinical accuracy, and a binary safety score. Readability was measured by Flesch reading ease score (FRES) and Flesch-Kincaid grade level (FKGL). Models were compared by using Friedman tests and Bonferroni-corrected Wilcoxon signed-rank tests. Claude ranked the highest by JAMA and DISCERN scores and that DeepSeek outperformed ChatGPT, Gemini, and Copilot in terms of DISCERN. Clinical accuracy differed significantly among the models (p < 0.001). Claude and DeepSeek were rated higher than ChatGPT, Gemini, and Copilot, with no significant difference between Claude and DeepSeek. All responses were rated safe by all four raters. Readability performance scores of the models achieved significant differences on FRES (p = 0.043) and FKGL (p < 0.001) measures. The average FKGL score (10.85–13.93) was higher than the grade scores recommended for patient education information. There was heterogeneity in the platforms ability to report on aspects of transparency, information quality and clinical accuracy while consistently failing on readability. Patient comprehension was not evaluated.

Abbreviations

ACL:

Anterior cruciate ligament

AI:

Artificial intelligence

ANFH:

Avascular necrosis of the femoral head

DISCERN:

Quality criteria for consumer health information on treatment choices (DISCERN Instrument)

FKGL:

Flesch–kincaid grade level

FRES:

Flesch reading ease score

JAMA:

Journal of the American medical association

LLM:

Large language model

Acknowledgements

The authors gratefully acknowledge the support and contributions of all individuals who assisted in the completion of this study.

Author information

Authors and Affiliations

  1. Department of Orthopedics, Nigde Omer Halisdemir Research and Training Hospital, Nigde, Türkiye

    Alp Yekta Gokalp, Hasan Issiz & Talha Tepeoglu

  2. Department of Orthopedics, Mersin City Hospital, Mersin, Türkiye, Turkey

    Adnan Cakar

  3. Department of Biophysics, Faculty of Medicine, Nigde Omer Halisdemir University, Nigde, Türkiye

    Sefa Erturk

Authors

  1. Alp Yekta Gokalp
  2. Adnan Cakar
  3. Hasan Issiz
  4. Talha Tepeoglu
  5. Sefa Erturk

Corresponding author

Correspondence to Sefa Erturk.

Ethics declarations

Competing interests

The authors declare no competing interests.

Consent to publish

The authors have read and approved the manuscript and consent to its publication.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

This article is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gokalp, A.Y., Cakar, A., Issiz, H. et al. Evaluation of large language models’ responses to patient questions about avascular necrosis of the femoral head. Sci Rep (2026). https://doi.org/10.1038/s41598-026-77078-4

Download citation

  • Received: 19 June 2026

  • Accepted: 16 October 2026

  • Published: 01 November 2026

  • DOI: https://doi.org/10.1038/s41598-026-77078-4

Keywords

来源:Nature 健康护理主题 · nature.com