跳到正文
原文
Nature 健康护理主题·· 2026-10-07AI 评分58

多模态AI评估系统MAPLES在医学OSCE考核中的首次前瞻性部署

Prospective deployment of multimodal AI grading for medical student OSCEs

AI 导读

UT Southwestern开发MAPLES多模态AI评分系统,于2025年秋季对222名医学生进行OSCE考核,涵盖笔记、音频和视频。系统在低分复核中AI与标准化患者评价者一致率达85.7–92.4%,医生复核616例分歧时与AI一致率76.0%。AI初筛使人工评分环节减少92.3%。

正文

npj Digital Medicine (2026) Cite this article

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

Objective Structured Clinical Examinations (OSCEs) are central to assessing medical student clinical competence, but human grading imposes substantial burden. We report, to our knowledge, the first prospective single-center deployment of an integrated multimodal artificial intelligence (AI) system for OSCE grading in undergraduate medical education, spanning notes, audio, and video. We present MAPLES (Multimodal Assessment Pipeline for Learning Encounter Scoring), a rubric-driven zero-shot multimodal LLM system deployed in Fall 2025 for 222 students (72,907 retained item-level scores). In the routed low-scoring review set, tolerant AI–standardized patient evaluator (SPE) agreement was 85.7–92.4% by modality. In a selected set of 616 disagreements adjudicated with visible score provenance, physician scores matched AI on 76.0% of items and the SPE on 19.0%. Human scoring passes fell by 92.3% versus a modeled single-pass manual comparator. Together, these findings show that multimodal AI first-pass scoring can be embedded in routine OSCE operations, with human review on the low-scoring tail and physician adjudication of selected disagreements.

Acknowledgements

We are grateful to the standardized patient evaluators  and Simulation Center staff who conducted and reviewed the OSCEs and the AI supervisory committee for oversight. Support for this work was provided by UT Southwestern institutional funds (Office of the President).

Author information

Author notes

  1. These authors contributed equally: Huong-Tra Ngo, Ameer Hamza Shakur.

Authors and Affiliations

  1. Lyda Hill Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX, USA

    Huong-Tra Ngo, Ameer Hamza Shakur, Michael Holcomb, Shinyoung Kang, David Hein, Hunter Schuler & Andrew R. Jamieson

  2. Department of Internal Medicine, UT Southwestern Medical Center, Dallas, TX, USA

    Judah Gruen & Thomas Dalton

  3. Department of Emergency Medicine, UT Southwestern Medical Center, Dallas, TX, USA

    Philip Jarrett & Krystle Campbell

  4. Simulation Center, UT Southwestern Medical Center, Dallas, TX, USA

    Krystle Campbell & Daniel Scott

  5. Department of Surgery, UT Southwestern Medical Center, Dallas, TX, USA

    Daniel Scott

Authors

  1. Huong-Tra Ngo
  2. Ameer Hamza Shakur
  3. Michael Holcomb
  4. Shinyoung Kang
  5. David Hein
  6. Judah Gruen
  7. Hunter Schuler
  8. Philip Jarrett
  9. Thomas Dalton
  10. Krystle Campbell
  11. Daniel Scott
  12. Andrew R. Jamieson

Corresponding author

Correspondence to Andrew R. Jamieson.

Ethics declarations

Competing interests

The authors declare the following competing interests. The Board of Regents of The University of Texas System is the applicant for four pending patent applications related to this manuscript: (i) U.S. non-provisional patent application 19/699,629, filed 5 June 2026 and claiming priority to provisional application 63/819,115 (inventors: Andrew Jamieson, Michael Holcomb, Gaudenz Danuser, Daniel Scott, Krystle Campbell, and Thomas Dalton), covering context-aware grading of learner responses; (ii) PCT/US2026/022629, filed 7 April 2026 (inventors: Andrew Jamieson and Michael Holcomb), covering audio-visual segmentation and rubric-based OSCE grading; (iii) U.S. provisional patent application 64/140,783, filed 25 August 2026 (inventors: Andrew Jamieson and Shinyoung Kang), covering AI-assisted rubric authoring and refinement; and (iv) U.S. provisional patent application 64/136,854, filed 19 August 2026 (inventors: Andrew Jamieson, Michael Holcomb, Thomas Dalton, and Ngoc Le Van Horn), covering multimodal clinical-skills assessment and review workflows. The remaining authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ngo, HT., Shakur, A.H., Holcomb, M. et al. Prospective deployment of multimodal AI grading for medical student OSCEs. npj Digit. Med. (2026). https://doi.org/10.1038/s41746-026-03352-4

Download citation

  • Received: 24 July 2026

  • Accepted: 21 September 2026

  • Published: 07 October 2026

  • DOI: https://doi.org/10.1038/s41746-026-03352-4

来源:Nature 健康护理主题 · nature.com