大型语言模型驱动的聊天机器人在肺部霉菌病信息生成中的比较评估
Comparative evaluation of pulmonary aspergillosis information generated by large language model-based chatbots
一项比较研究显示,Microsoft Copilot和Perplexity生成的关于肺部霉菌病的信息在质量、透明度和可读性方面优于ChatGPT和Gemini 3.6 Flash,但所有平台生成的信息均较为复杂,需专业验证后使用。
We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.
Abstract
Chatbots powered by large language models are increasingly used for medical information retrieval, yet pulmonary aspergillosis presents complex diagnostic and therapeutic information needs. This comparative cross-sectional study evaluated the information quality, transparency, and readability of responses generated by four chatbot platforms. On 6 August 2026, 31 search terms derived from five-year worldwide Google Trends data were submitted to Microsoft Copilot, ChatGPT (GPT-5.5), Perplexity, and Gemini 3.6 Flash, yielding 124 first-response texts. Quality and transparency were assessed using JAMA, GQS, EQIP, and DISCERN, and readability using SMOG, Coleman-Liau, FKGL, GFI, FRES, and ARI. Friedman tests accounting for matched queries showed significant platform differences for all four quality and transparency measures. Median DISCERN scores were 43 for Copilot and Perplexity, 36 for ChatGPT, and 33 for Gemini. None of the 124 default responses met the prespecified sixth-grade threshold for any of the six readability measures. Copilot and Perplexity achieved higher scores on several measures of written-information quality and transparency, while responses from all four platforms remained textually complex. Chatbot-generated information should serve as a supplementary resource following professional verification.
Funding
No specific funding was received.
Author information
Authors and Affiliations
Department of Respiratory Medicine, Xinfeng County People’s Hospital, Ganzhou, 341600, Jiangxi, China
Yanmao Wang, Jing Li & Jin Zhong
Authors
- Yanmao Wang
- Jing Li
- Jin Zhong
Corresponding author
Correspondence to Jin Zhong.
Ethics declarations
Competing interests
The authors declare no competing interests.
Ethics Declarations
Ethical review and approval were not required for this study because it analyzed publicly accessible AI-generated information and involved no human participants, human biological specimens, or identifiable personal data.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Wang, Y., Li, J. & Zhong, J. Comparative evaluation of pulmonary aspergillosis information generated by large language model-based chatbots. Sci Rep (2026). https://doi.org/10.1038/s41598-026-79335-y
Received: 24 August 2026
Accepted: 30 October 2026
Published: 04 November 2026
DOI: https://doi.org/10.1038/s41598-026-79335-y
Keywords
来源:Nature 健康护理主题 · nature.com