跳到正文
原文
Nature 健康护理主题·· 6 天前AI 评分57

OASIS与KAAD MRI基准中患者级及重复图像数据泄漏致报告准确率虚高

Patient-level and duplicate-image data leakage inflate reported accuracy in OASIS and KAAD MRI benchmarks

AI 导读

该研究针对OASIS与Kaggle-Augmented Alzheimer's Dataset(KAAD)两个常用于阿尔茨海默病MRI深度学习的基准进行分析,发现两者均存在显著数据泄漏。

正文

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

Deep learning studies of Alzheimer’s disease classification often use two Kaggle-hosted MRI datasets: OASIS, and a widely used Kaggle Alzheimer’s MRI export sometimes falsely referred to as ADNI, here termed the Kaggle-Augmented Alzheimer’s Dataset (KAAD) to reflect its documented Kaggle provenance (Methods). Both are split into training and test sets at the image level without accounting for image provenance, creating an opportunity for identity leakage. In OASIS, patient identifiers recoverable from filenames show that 100% of test images share a patient with the training set under naive random splitting, across all 86,437 images. In KAAD, whose filenames are anonymized, exact-hash and validated perceptual-hash analyses estimate that roughly one-fifth of images are near-duplicates of another image in the set. Using three classifiers of increasing capacity, removing this leakage lowers mean test accuracy from 100.0% to about 46% for OASIS (p < = 0.00005) and from 68.0% to 58.9–62.8% for KAAD (p = 0.0028-0.09); the OASIS collapse was essentially unchanged by a five-fold capacity increase, indicating it is not an artifact of an under-capacity baseline. A cross-dataset test across five seeds shows a classifier trained on OASIS performs at chance level on KAAD (25.0%) despite perfect training accuracy. These findings, consistent with prior reports, indicate that much published accuracy on these benchmarks reflects identity memorization rather than transferable diagnostic signal. Leakage-free, patient- and duplicate-aware split manifests are released for both datasets.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Karpagam Academy of Higher Education, Coimbatore, India

    V. Santhoshkumar & B. Lanitha

Authors

  1. V. Santhoshkumar
  2. B. Lanitha

Corresponding authors

Correspondence to V. Santhoshkumar or B. Lanitha.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethical considerations and data use

This study is a secondary, pixel- and file-structure-level analysis of two pre-existing, third-party MRI image collections; no new human-subjects data were collected. All methods were carried out in accordance with relevant institutional guidelines and regulations and with the Declaration of Helsinki. The original OASIS cross-sectional study (Washington University in St. Louis) obtained institutional ethical approval and informed consent from all participants as part of its primary data collection, and its data are released by the OASIS project in de-identified form for research use¹⁸. The base images of KAAD correspond to the same OASIS cross-sectional cohort (Data availability), whose ethical approval and consent therefore also apply to them; the KAAD Kaggle compiler holds no ethics documentation and is not relied on for this. This secondary, retrospective analysis of pre-existing, publicly available data was reviewed by the Institutional Ethics Committee of Karpagam Academy of Higher Education, which determined that this study did not require additional ethical approval or separate participant consent. Neither the authors nor this manuscript recruited participants, performed MRI acquisition, or assigned new clinical diagnoses; the diagnostic categories reported throughout this study are inherited, unmodified, from the respective source datasets. This manuscript does not redistribute any MRI images, only derived, non-identifying manifests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Santhoshkumar, V., Lanitha, B. Patient-level and duplicate-image data leakage inflate reported accuracy in OASIS and KAAD MRI benchmarks. Sci Rep (2026). https://doi.org/10.1038/s41598-026-77108-1

Download citation

  • Received: 24 July 2026

  • Accepted: 17 October 2026

  • Published: 14 November 2026

  • DOI: https://doi.org/10.1038/s41598-026-77108-1

Keywords

来源:Nature 健康护理主题 · nature.com