跳到正文
原文
Nature 健康护理主题·· 2026-10-02AI 评分39

糖尿病检测机器学习模型的多维度预测安全性综合分析

A comprehensive study on multi-dimensional predictive safety analysis of machine learning models in diabetes detection

AI 导读

一项在Scientific Reports发表的研究通过基准测试评估了不同机器学习模型在糖尿病风险检测中的预测安全性。研究发现,即使模型区分性能(PR-AUC)相近,其校准质量、不确定性有效性和子群体可靠性仍存在显著差异。该工作表明仅依赖区分指标不足以保障临床决策可靠性,需纳入校准、公平性及保形预测等维度进行综合评估。

正文

Scientific Reports (2026) Cite this article

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

High discrimination performance is often interpreted as evidence of model reliability in clinical machine learning (ML). However, strong predictive accuracy does not guarantee safe decision behavior. In safety-critical domains such as healthcare, miscalibrated probabilities, ineffective uncertainty estimation, and overconfident errors can lead to harmful decisions despite high area-under-curve metrics. This study presents a comprehensive multi-dimensional predictive safety analysis demonstrating that models with comparable discrimination performance can exhibit substantially different reliability and risk characteristics. We conduct a rigorous benchmarking evaluation across two heterogeneous diabetes risk datasets, a large population-scale BRFSS cohort and a small-sample early-stage clinical dataset. A diverse set of classical, ensemble, and deep tabular models are evaluated under a strict leakage-free cross-validation protocol with dedicated calibration splits. Beyond conventional discrimination metrics (PR-AUC, ROC-AUC), we assess probabilistic calibration (ECE, Brier score, NLL), epistemic uncertainty using a unified bootstrap-based framework, uncertainty–error correlation, selective prediction safety, high-confidence error rates, subgroup fairness across age and sex, and conformal prediction with finite-sample marginal coverage guarantees under the standard exchangeability assumption. Our results reveal a consistent accuracy–safety gap that models achieving near-identical PR-AUC display markedly different calibration quality, uncertainty effectiveness, subgroup reliability, and overconfidence behavior. On the early-stage clinical dataset, near-ceiling discrimination performance is observed across models, but due to the limited sample size, these results should be interpreted cautiously, as variability across folds may affect the stability of comparative performance across models. The findings demonstrate that discrimination metrics alone are insufficient to characterize reliable clinical decision-making. A comprehensive evaluation incorporating calibration, uncertainty quality, subgroup fairness, and formal coverage guarantees is essential for the trustworthy pre-deployment assessment of machine learning in healthcare. Rather than introducing a new predictive algorithm, this work contributes a unified empirical audit of safety-relevant model behavior, addressing reliability dimensions that are commonly evaluated separately or omitted in conventional diabetes prediction benchmarks.

Acknowledgements

This Project was funded by the Deanship of Scientific Research (DSR) at King Abdulaziz University, Jeddah, under grant no.(IPP: 675-611-2025). The authors, therefore, acknowledge DSR’s technical and financial support with thanks.

Funding

Deanship of Scientific Research (DSR) at King Abdulaziz University, Jeddah, under grant no.(IPP: 675-611-2025).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Bangladesh University of Business and Technology, Dhaka, Bangladesh

    Nasirul Mumenin

  2. Department of Information and Communication Technology, Bangladesh University of Professionals, Dhaka, Bangladesh

    Md. Appel Mahmud Pranto

  3. Institute of Information Technology, Jahangirnagar University, Dhaka, Bangladesh

    Mohammad Abu Yousuf

  4. Department of Information Technology, Faculty of Computing and Information Technology, King Abdulaziz University, 21589, Jeddah, Kingdom of Saudi Arabia

    Muhammad Mostafa Monowar, Md Abdul Hamid & Madini O. Alassafi

  5. Department of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, 21589, Jeddah, Kingdom of Saudi Arabia

    Fahd S. Alotaibi

Authors

  1. Nasirul Mumenin
  2. Md. Appel Mahmud Pranto
  3. Mohammad Abu Yousuf
  4. Muhammad Mostafa Monowar
  5. Md Abdul Hamid
  6. Fahd S. Alotaibi
  7. Madini O. Alassafi

Corresponding author

Correspondence to Nasirul Mumenin.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mumenin, N., Pranto, M.A.M., Yousuf, M.A. et al. A comprehensive study on multi-dimensional predictive safety analysis of machine learning models in diabetes detection. Sci Rep (2026). https://doi.org/10.1038/s41598-026-73801-3

Download citation

  • Received: 28 February 2026

  • Accepted: 25 September 2026

  • Published: 02 October 2026

  • DOI: https://doi.org/10.1038/s41598-026-73801-3

来源:Nature 健康护理主题 · nature.com