跳到正文
原文
Nature 健康护理主题·· 17 天前AI 评分45

RETBind: 一种用于统一视网膜多模态表示学习的患者级多模态基础模型

RETBind: a patient-level multimodal foundation model for unified cross-modal retinal representation learning

AI 导读

RETBind 是一种患者级多模态基础模型,通过统一编码器学习视网膜光学相干断层扫描(OCT)和超广视野(UWF)影像的跨模态对应关系,显著提升了眼科疾病分类和分割任务的性能,特别是在临床相关不平衡设置中增强了AUPR值。

正文

npj Digital Medicine (2026) Cite this article

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

Medical artificial intelligence (AI) holds considerable promise across a wide range of ophthalmic tasks, offering a practical path to patient-level diagnostic support. Foundation models further extend this potential by learning generalizable representations from large-scale unlabeled data, alleviating the burden of extensive manual annotation. However, existing ophthalmic foundation models largely rely on single-modality or modality-specific encoders, failing to capture patient-level cross-modal correspondence in real-world multimodal clinical workflows. To address these gaps, we propose RETBind, a patient-level multimodal foundation model with a unified encoder that explicitly learns cross-modal correspondence from paired clinical imaging using patient-level paired optical coherence tomography (OCT) and ultra-widefield (UWF) fundus imaging. This patient-level paired pretraining strategy enables the model to learn unified cross-modal representations of complementary information, improv-ing performance and robustness beyond single-modality approaches. Across nine benchmark datasets spanning ocular disease classification and segmentation tasks, RETBind achieved consistent improvements, particularly enhancing AUPR in clinically relevant imbalanced settings. Exploratory analyses further examined the potential of RETBind for systemic disease prediction from retinal imaging, collectively highlighting its ability to learn unified cross-modal representations within a single shared encoder, enabling direct interaction between modalities and supporting transfer across diverse retinal imaging tasks.

Acknowledgments

Not applicable.

Funding

This research was supported by the MSIT(Ministry of Science, ICT), Korea, under the Global Scholars Invitation Program [RS-2024-00459638] supervised by the IITP(Institute for Information & Communications Technology Planning & Evaluation) and the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) [No. 2023R1A2C2007625]. The funding organizations had no role in the design, conduct, analysis, interpretation, or reporting of this study.

Author information

Author notes

  1. These authors contributed equally: Jian Park, Jinyoung Han.

Authors and Affiliations

  1. Department of Applied Artificial Intelligence, Sungkyunkwan University, Seoul, Republic of Korea

    Jian Park, Jinyoung Han, Hyunseon Won, JeeEun Kim & Sihyeon Yang

  2. Seoul Plus Eye Clinic, Seoul, Republic of Korea

    Joon Seo Hwang

  3. Seoul Bombit Eye Clinic, Sejong, Republic of Korea

    Jeong Mo Han

  4. Department of Internal Medicine, Kangwon National University Hospital, Kangwon National University School of Medicine, Chuncheon, Republic of Korea

    Ji In Park

  5. Hangil Eye Hospital, Incheon, Republic of Korea

    Daniel Duck-Jin Hwang

  6. Lux Mind, Incheon, Republic of Korea

    Daniel Duck-Jin Hwang

  7. Department of Ophthalmology, Catholic Kwandong University College of Medicine, Incheon, Republic of Korea

    Daniel Duck-Jin Hwang

Authors

  1. Jian Park
  2. Jinyoung Han
  3. Hyunseon Won
  4. JeeEun Kim
  5. Sihyeon Yang
  6. Joon Seo Hwang
  7. Jeong Mo Han
  8. Ji In Park
  9. Daniel Duck-Jin Hwang

Corresponding author

Correspondence to Daniel Duck-Jin Hwang.

Ethics declarations

Competing interests

DDH is the founder and Chief Executive Officier (CEO) of Lux Mind. The remaining authors declare no competing financial or non-financial interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Park, J., Han, J., Won, H. et al. RETBind: a patient-level multimodal foundation model for unified cross-modal retinal representation learning. npj Digit. Med. (2026). https://doi.org/10.1038/s41746-026-03426-3

Download citation

  • Received: 21 April 2026

  • Accepted: 07 October 2026

  • Published: 03 November 2026

  • DOI: https://doi.org/10.1038/s41746-026-03426-3

来源:Nature 健康护理主题 · nature.com