跳到正文
原文
Nature 健康护理主题·· 5 天前AI 评分37

基于DAIC-WOZ数据的预印本研究:一种用于抑郁症检测的泄漏控制多模态框架(音频/视觉/文本)

A leakage-controlled multimodal framework for depression detection using audio, visual, and text features

AI 导读

该框架结合Mel频率倒谱系数等声学特征、约束局部神经场地标面部行为特征和BERT嵌入文本语义,在DAIC-WOZ数据集上以参与者独立的分层五折交叉验证实现91.2%准确率、宏观-F1 89.5%、ROC AUC 0.94,消融分析显示文本表征最有效,音声与面部特征提供互补增益。该研究为预印本,尚未经同行评审,外部验证须在临床应用前完成。

正文

Scientific Reports (2026) Cite this article

We’re sharing this article early to provide faster access to peer-reviewed, accepted research. It is citable and carries a permanent DOI. This version is subject to further edits and will be replaced automatically by the final Version of Record. All legal disclaimers apply.

Abstract

Depression is one of the most prevalent mental health disorders that can be addressed using objective computational screening that may be used in conjunction with clinical screening, but should not be used to replace it. The present research proposes a clear multimodal approach combining the acoustic, visual and textual behavioural data of a structured interview. The speech is represented by Mel-frequency cepstral coefficients and COVAREP descriptors, the facial behaviour is represented by constrained local neural fields landmarks and the semantics of the text is represented by BERT embeddings. The standardized training-fold statistics, the fixed weighted feature-level concatenation, the principal component analysis and the L2-regularized Logistic Regression are used to standardize, concatenate, reduce and classify modality specific features. The accuracy when evaluated on the DAIC-WOZ corpus, using a participant-independent, stratified five-fold cross validation was 91.2%, macro-precision 88.5%, macro-recall 90.7%, macro-F1 score 89.5%, specificity 91.7%, depressed-class sensitivity 89.7%, and ROC AUC 0.94. Improvements of unimodal and multimodal representations were found in the ablation analysis, with text proving the most effective representation, and acoustic and facial features adding complementary improvements in the multimodal context. The framework focuses on the aspects of methodological transparency, training-fold leakage control and low-parameter classifiers. External validation must be done prior to clinical use.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. School of CS & AI, SR University, Warangal, Telangana, India

    Manisha Kallem & Srinivas Aluvala

  2. School of Computer Science, Bennett University, Greater Noida, Uttar Pradesh, India

    Prabhishek Singh

Authors

  1. Manisha Kallem
  2. Srinivas Aluvala
  3. Prabhishek Singh

Corresponding author

Correspondence to Manisha Kallem.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethics approval and informed consent

The initial data collection for the DAIC-WOZ data set was approved by the University of Southern California Institutional Review Board. All participants gave informed consent at the time the data was collected. Only secondary analysis of controlled access data was conducted with no recruitment of new participants, intervention, or direct contact with participants. All approaches were conducted in compliance with guidelines, regulations and the requirements of the corpus data.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kallem, M., Aluvala, S. & Singh, P. A leakage-controlled multimodal framework for depression detection using audio, visual, and text features. Sci Rep (2026). https://doi.org/10.1038/s41598-026-79352-x

Download citation

  • Received: 12 September 2026

  • Accepted: 30 October 2026

  • Published: 15 November 2026

  • DOI: https://doi.org/10.1038/s41598-026-79352-x

Keywords

来源:Nature 健康护理主题 · nature.com