跳到正文
原文
Nature 健康护理主题·· 2026-10-07AI 评分38

通过人群数据配对比较评分提升基于AI的语音表现评估

Boosting AI-based speech performance assessment through pairwise comparative scoring of crowd data

AI 导读

Scientific Reports 发表的一项研究让122名参与者完成4项语音任务(每项3个难度),提出配对比较评分框架以提升AI语音表现评估可靠性。配对比较评分(绝对分与组内相对分)的信度高于单独评分,达到多名人工评分者平均分的水平。工作记忆关联提供效度证据,支持该框架用于复杂开放作答评估。

正文

Scientific Reports (2026) Cite this article

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

Open-ended performance assessments can capture intellectual status that is difficult to measure using highly constrained response formats. However, their practical use is limited by the difficulty of obtaining reliable, scalable, and interpretable scores. Recent advances in large language models provide new opportunities for automated scoring, but the measurement quality of AI-generated scores remains an important empirical question. We evaluated AI models for complex speech-based assessment, and proposed a pairwise comparative scoring framework designed to improve reliability, while examining validity evidence through associations with working memory. Four speech tasks, each with three difficulty levels, were administered to 122 participants. We applied three AI scoring approaches: (a) individual scoring, where each speech response was assessed separately; (b) pairwise scoring with absolute scores, where paired responses were presented together and each participant received an absolute score; and (c) pairwise scoring with relative scores, where each score was converted into a within-pair difference. Human ratings provided a conventional benchmark. Working memory was measured to examine theoretically relevant validity evidence. Pairwise comparative scoring showed higher reliability than individual scoring, reaching levels comparable to human scores averaged across multiple raters, while providing theoretically consistent validity evidence. These results support AI-based comparative scoring for complex open-ended assessment.

Funding

This work was partially supported by Seed Funds from the University of Hong Kong (2402101405, 2502251399, 2503251476). The funders have/had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Complex Neural Signals Decoding Lab, Faculty of Education, The University of Hong Kong, Pokfulam, Hong Kong Island, Hong Kong

    Yingzhe Li & Guang Ouyang

Authors

  1. Yingzhe Li
  2. Guang Ouyang

Corresponding author

Correspondence to Guang Ouyang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Ouyang, G. Boosting AI-based speech performance assessment through pairwise comparative scoring of crowd data. Sci Rep (2026). https://doi.org/10.1038/s41598-026-74639-5

Download citation

  • Received: 03 June 2026

  • Accepted: 30 September 2026

  • Published: 07 October 2026

  • DOI: https://doi.org/10.1038/s41598-026-74639-5

Keywords

来源:Nature 健康护理主题 · nature.com