RETBind: 一种用于统一视网膜多模态表示学习的患者级多模态基础模型
RETBind: a patient-level multimodal foundation model for unified cross-modal retinal representation learning
RETBind 是一种患者级多模态基础模型,通过统一编码器学习视网膜光学相干断层扫描(OCT)和超广视野(UWF)影像的跨模态对应关系,显著提升了眼科疾病分类和分割任务的性能,特别是在临床相关不平衡设置中增强了AUPR值。
- Article
- Open access
- Published: 03 November 2026
- Jian Park1 na1,
- Jinyoung Han1 na1,
- Hyunseon Won1,
- JeeEun Kim1,
- Sihyeon Yang1,
- Joon Seo Hwang2,
- Jeong Mo Han3,
- Ji In Park4 &
- …
- Daniel Duck-Jin Hwang ORCID: orcid.org/0000-0003-1808-31695,6,7
npj Digital Medicine (2026) Cite this article
We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.
Abstract
Medical artificial intelligence (AI) holds considerable promise across a wide range of ophthalmic tasks, offering a practical path to patient-level diagnostic support. Foundation models further extend this potential by learning generalizable representations from large-scale unlabeled data, alleviating the burden of extensive manual annotation. However, existing ophthalmic foundation models largely rely on single-modality or modality-specific encoders, failing to capture patient-level cross-modal correspondence in real-world multimodal clinical workflows. To address these gaps, we propose RETBind, a patient-level multimodal foundation model with a unified encoder that explicitly learns cross-modal correspondence from paired clinical imaging using patient-level paired optical coherence tomography (OCT) and ultra-widefield (UWF) fundus imaging. This patient-level paired pretraining strategy enables the model to learn unified cross-modal representations of complementary information, improv-ing performance and robustness beyond single-modality approaches. Across nine benchmark datasets spanning ocular disease classification and segmentation tasks, RETBind achieved consistent improvements, particularly enhancing AUPR in clinically relevant imbalanced settings. Exploratory analyses further examined the potential of RETBind for systemic disease prediction from retinal imaging, collectively highlighting its ability to learn unified cross-modal representations within a single shared encoder, enabling direct interaction between modalities and supporting transfer across diverse retinal imaging tasks.
Acknowledgments
Not applicable.
Funding
This research was supported by the MSIT(Ministry of Science, ICT), Korea, under the Global Scholars Invitation Program [RS-2024-00459638] supervised by the IITP(Institute for Information & Communications Technology Planning & Evaluation) and the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) [No. 2023R1A2C2007625]. The funding organizations had no role in the design, conduct, analysis, interpretation, or reporting of this study.
Author information
Author notes
These authors contributed equally: Jian Park, Jinyoung Han.
Authors and Affiliations
Department of Applied Artificial Intelligence, Sungkyunkwan University, Seoul, Republic of Korea
Jian Park, Jinyoung Han, Hyunseon Won, JeeEun Kim & Sihyeon Yang
Seoul Plus Eye Clinic, Seoul, Republic of Korea
Joon Seo Hwang
Seoul Bombit Eye Clinic, Sejong, Republic of Korea
Jeong Mo Han
Department of Internal Medicine, Kangwon National University Hospital, Kangwon National University School of Medicine, Chuncheon, Republic of Korea
Ji In Park
Hangil Eye Hospital, Incheon, Republic of Korea
Daniel Duck-Jin Hwang
Lux Mind, Incheon, Republic of Korea
Daniel Duck-Jin Hwang
Department of Ophthalmology, Catholic Kwandong University College of Medicine, Incheon, Republic of Korea
Daniel Duck-Jin Hwang
Authors
- Jian Park
- Jinyoung Han
- Hyunseon Won
- JeeEun Kim
- Sihyeon Yang
- Joon Seo Hwang
- Jeong Mo Han
- Ji In Park
- Daniel Duck-Jin Hwang
Corresponding author
Correspondence to Daniel Duck-Jin Hwang.
Ethics declarations
Competing interests
DDH is the founder and Chief Executive Officier (CEO) of Lux Mind. The remaining authors declare no competing financial or non-financial interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Park, J., Han, J., Won, H. et al. RETBind: a patient-level multimodal foundation model for unified cross-modal retinal representation learning. npj Digit. Med. (2026). https://doi.org/10.1038/s41746-026-03426-3
Received: 21 April 2026
Accepted: 07 October 2026
Published: 03 November 2026
DOI: https://doi.org/10.1038/s41746-026-03426-3
来源:Nature 健康护理主题 · nature.com