跳到正文
原文
Nature 健康护理主题·· 14 天前AI 评分55

CHARS:开发和验证一种新的评估健康和社会护理聊天机器人的质量工具

Chatbot rating scale (CHARS): Development and validation of a new tool for assessing the quality of health and social care chatbots

AI 导读

研究开发了一种新的评估工具 CHARS,用于全面评估健康和社会护理聊天机器人的质量,包括设计、个性化、参与度、可用性和功能五个维度,填补了现有评估工具的空白。

正文

Introduction

Chatbots are AI-powered conversational agents that can understand and respond to user queries through text, voice, and/or images, enabling efficient and accessible interactions1. These technologies have expanded rapidly, with over a million chatbots serving more than 1.4 billion users globally2. In healthcare, chatbots have been applied to health promotion, disease prevention, patient monitoring, mental health support, and administrative tasks3,4. They offer potential advantages such as personalized guidance, streamlined workflows, and support for clinical decision-making5,6.

Despite these potential benefits, evidence regarding their effectiveness remains mixed. Challenges include limited accuracy, potential breaches of patient privacy, variable patient trust, and difficulties integrating chatbots into established clinical workflows7,8. While applications in mental health and sexual health have demonstrated some improvements in patient engagement and cost savings9,10,11,12, other studies have reported limited or non-significant effects on anxiety reduction, psychological well-being, therapeutic alliance, long-term engagement, and sustained adherence to chatbot interventions13,14. Concerns have also been raised regarding the reliability, consistency, and safety of chatbot-generated responses. These inconsistencies may reflect variations in chatbot design, intervention duration, target populations, implementation settings, and evaluation methodologies, highlighting the need for rigorous and standardized evaluation frameworks for health and social care chatbots.

Chatbots can be grouped based on whether they are unconstrained (i.e. allows free text or speech) or constrained (allow only pre-programmed responses in the chat), or both. Healthcare chatbots may also operate on-premise systems (hosted locally within an organization’s infrastructure) or cloud-based solutions (hosted on remote servers and accessed via the internet), and can utilize artificial intelligence, artificial generative intelligence, or other emerging technologies to allow patients and the populace to assess their symptoms, get support, locate clinics, and schedule appointments15,16. Conversational health and care chatbots are developed for real-world use, and their quality in healthcare contexts is defined not only by alignment with user needs and the system’s intended domain but also by their ability to deliver accurate, evidence-based information, ensure patient safety, and adhere to clinical and regulatory standards8,17,18,19.

A chatbot’s domain refers to the specific subject area or healthcare context in which the system operates, including the scope of knowledge, tasks, and user interactions for which it is designed15,20. Its usability and functionality must therefore be assessed in light of this domain. Usability refers to the extent to which users can effectively, efficiently, and satisfactorily achieve their goals when interacting with a chatbot within a specific context of use19,20. Functionality refers to the chatbot’s capability to correctly perform intended tasks within its domain, including providing relevant responses, supporting task completion, and delivering appropriate system outputs19,21. Engagement refers to the degree of users’ cognitive, emotional, and behavioural involvement during interaction with the chatbot, including attention, interest, and sustained participation over time22. Personalization refers to the system’s ability to adapt its responses, interaction style, or recommendations based on individual user characteristics, preferences, or prior interactions to enhance relevance and user experience23. Design encompasses the structural and interface-related features of the chatbot, including conversational flow, visual presentation, and interaction layout that shape how users engage with the system24,25. From prior research, chatbot quality encompasses design, engagement, functionality, personalization, and usability dimensions, capturing user-centred measures of system performance17,18,26,27,28,29. More importantly, quality is dynamic rather than static, reflecting the evolving nature of chatbot interactions. For example, a chatbot that ranks high on functionality today may become outdated if its content, design, or conversational capabilities are not updated to meet evolving user needs30.

Despite the exponential rise of chatbots, their quality remains challenging to assess for end users, developers, and health professionals. Most users rely on popularity metrics such as “star ratings” on app stores, which provide little meaningful evidence of quality. The 10-item System Usability Scale (SUS), developed in 1986 by John Brooke, is one of the earliest and most widely applied tools for system usability31. While the SUS has good face validity, broad applicability, and strong internal consistency, it is unidimensional, highly subjective, and not well-suited for advanced technologies such as mobile apps and chatbots. It excludes critical elements such as aesthetics, personalization, and engagement, which are central to modern interactive systems. Other approaches to rating digital systems, including website evaluation tools, have been too technical, checklist-based, or focused primarily on aesthetics and structure32,33,34.

More recently, the Mobile App Rating Scale (MARS)17 was developed specifically for health mobile apps, covering engagement, functionality, aesthetics, and information quality. While MARS is reliable and has good face validity, it was designed for static mobile applications and does not fully account for chatbot-specific features such as natural language interaction, trustworthiness, and personalization. Moreover, MARS was not originally validated through confirmatory factor analysis, limiting evidence for its factorial structure. In contrast, CHARS address these limitations by evaluating the dynamic and interactive qualities of health and mental health chatbots.

Few peer-reviewed studies have proposed chatbot-specific rating scales. For example, Borsci et al.18 developed a chatbot usability scale, but this tool focuses on usability and functionality while neglecting domains like aesthetics and personalization. Lewis & Hardzinski19 and Hone & Graham27 targeted speech-based interfaces rather than conversational health chatbots. Hence, there is a need to build on the strengths of existing tools while addressing their limitations as current approaches to evaluating chatbots are often fragmented, with some focusing narrowly on technical performance while neglecting broader aspects such as trust, engagement, and accessibility. This makes it difficult to compare tools, ensure accountability, or guide developers toward improvements.

To address this gap, the Chatbot Rating Scale (CHARS) was designed as a concise and multidimensional instrument that captures key domains of chatbot quality, including usability, functionality, personalization, design, and aesthetics. The scale was developed to be simple, brief, and practical, enabling use with minimal training while maintaining objectivity and reliability. By providing a standardized framework, CHARS offer a way to systematically assess and compare digital health chatbots, ultimately supporting safer and more effective deployment in health and social care.

The primary objective of this study is to develop and validate the Chatbot Rating Scale (CHARS), a concise, reliable, and multidimensional instrument for evaluating the quality of health and social care chatbots. The scale was designed to be simple, easily interpretable, and applicable with minimal training, ensuring its practical utility for researchers, developers, and healthcare practitioners in assessing chatbot quality and performance.

Methods

This study employed a sequential multi-method design for the development and validation of the Chatbot Rating Scale (CHARS)35. The first phase involved a two-stage literature review process (rapid review and comprehensive literature review) and a modified Delphi process to identify and refine initial scale items and dimensions of chatbot quality. Phase 2 comprised psychometric evaluation, including pilot testing, construct validity assessment, reliability analysis, and factor structure determination.

Chatbot rating scale development of item pool

The development of the Chatbot Rating Scale (CHARS) item pool involved a systematic approach comprising three key steps. First, an initial rapid review to identify existing items and dimensions. Second, a comprehensive literature search to generate the pool of candidate items while ensuring content breadth, identification of other relevant dimensions and theoretical grounding. Third, the application of the modified Delphi method to achieve expert consensus and refine the proposed items for clarity, relevance, and comprehensiveness.

Rapid review

A preliminary rapid review was conducted during the early phase of scale development to identify commonly reported dimensions and items as well as broadly relevant assessment domains relevant to chatbot quality evaluation. Searches were performed across major databases such as PubMed, Scopus, Google Scholar, and PsycINFO using keywords including “chatbot,” “bot,” “scale,” “chatbot scale,” “assessing quality of chatbots,” and related terms. Findings from prior chatbot and digital health evaluation studies17,18,36,37,38 were used to generate an initial pool of candidate items and conceptual domains. This preliminary review served as an exploratory scoping exercise to support the early item generation and to confirm that no similar study has been developed prior to formal evidence synthesis and expert refinement.

Comprehensive literature review

A comprehensive literature review was conducted to provide a more exhaustive assessment of the literature on quality assessment criteria and evaluation approaches relevant to conversational health chatbots to identify additional items and domains for the CHARS development. The review involved database searching, record de-duplication, and sequential title, abstract, and full-text screening to identify eligible studies. A structured search was performed across six electronic databases (PubMed, Scopus, PsycINFO, Web of Science, ScienceDirect, and IEEE Xplore) for studies published between January 2001 and December 2024. Search terms included two groups of keywords: chatbot/online-related terms (“chatbot,” “bot,” “website,” “App*,” “Chat*”) and assessment-related terms (“rating,” “quality,” “assess,” “evaluate*,” “scale”), combined using Boolean operators. Eligible studies included those reporting evaluation frameworks, rating scales, or assessment approaches for chatbots and related digital health technologies.

Records were screened independently by two reviewers, with disagreements resolved by a third reviewer. Reference lists of included studies were manually searched to identify additional sources, and grey literature, including conference proceedings and relevant websites were reviewed. Relevant studies were analysed to extract evaluation domains and criteria applicable to conversational health chatbots. Items not directly relevant to chatbot functionality were excluded. The synthesized evidence informed the development and refinement of domains and the initial item pool for the present scale (See Table 1).

Table 1 Mapping of extracted items to the initial 5-dimension framework.

Full size table

Modified Delphi method

The content validity of the Chatbot Rating Scale (CHARS) was established using a modified Delphi technique, an iterative structured method designed to facilitate systematic expert evaluation of measurement items in terms of relevance, clarity, and appropriateness. A purposive panel of 24 participants was recruited from professional networks and institutions, comprising digital health experts, IT specialists, psychologists, medical doctors, user interface and experience designers, chatbot developers, researchers, graduate and undergraduate students, and end users, ensuring a multidisciplinary evaluation of the scale from clinical, technical, methodological, and user-centred perspectives. An initial pool of 67 items generated through an extensive literature review served as the basis for evaluation (See Table 1). The Delphi process was conducted over three rounds across four months, combining asynchronous online surveys with synchronous virtual discussions to support structured feedback and iterative refinement. Three rounds were conducted based on Delphi standards39, which supports that two to three rounds are usually sufficient to stable consensus while minimizing fatigue. All asynchronous surveys were administered via online survey, with each round open for 14 days and regular email reminders on the 7th, 10th, 12th, 13th and 14th day sent to non-respondents. The a priori consensus threshold was set at 80% agreement among panel members for item retention. Items achieving 80% or higher agreement on relevance, clarity and appropriateness were retained; items below 60% were excluded; and items between 60 and 80% were discussed and re-rated in subsequent rounds. Synchronous virtual discussions were conducted after each asynchronous round specifically to resolve ambiguous items (those with 60–80% agreement) and clarify qualitative feedback. Our modified Delphi approach was chosen to balance quantitative consensus with qualitative depth, with voting remaining independent during asynchronous rounds. In Round 1, experts independently rated each item for relevance, clarity and appropriateness on a 5-point scale and provided qualitative feedback to improve the items. In Round 2, revised items were re-evaluated by the panel to further narrow item list, refine wording and conceptual clarity. In Round 3, a final review was conducted to consolidate expert input on the refined item set.

Chatbot rating scale pilot testing on health and mental health chatbots

A systematic process was followed to identify, screen, and select chatbots for rating using the Chatbot Rating Scale (CHARS). The search for eligible chatbots was conducted between June and July 2025, before commencement of the validation phase, across major platforms including the Google Play Store, Apple App Store, and verified web-based directories for digital health technologies. Searches were performed using predefined keywords related to conversational health technologies, including “chatbot,” “AI chatbot,” “health chatbot,” “mental health chatbot,” “therapy chatbot,” “health assistant,” and “virtual health assistant.” The aim was to identify chatbots providing health or mental health services that were accessible to end users.

For consistency in the chatbot rating, we developed a “prompt chaining” approach, which involves a sequence of documented questions and instructions (Supplement 1), which was first piloted, tested, and further refined. We then asked CHARS participants to evaluate one randomly selected chatbot (from our final 36 chatbots) using the CHARS scale via the KoboCollect interface.

Eligibility criteria for chatbots

Chatbots were eligible for inclusion if they provided health or mental health services, were currently active and accessible to end-users, and were available on major platforms such as the Google Play Store or the Apple App Store. Only chatbots offering content in English and free to use were considered, while those still in pilot testing or not publicly available were excluded. This ensured that the selection focused on functional, widely accessible health chatbots, forming a reliable basis for testing the CHARS scale.

Population and sampling technique

A total of 979 participants, comprising health experts, mental health professionals, undergraduates, and end-users aged 20–35 years, were recruited for the quantitative validation of the Chatbot Rating Scale (CHARS). Participants were recruited using a convenience sampling approach, targeting individuals who were accessible through professional networks, online communities, or interest in health and mental health digital tools. Eligibility criteria for participation included: (i) age between 20–35 years, (ii) willingness to participate voluntarily, (iii) ability to access and use digital devices such as smartphones, tablets, or computers, and (iv) basic familiarity with or interest in digital health or chatbot-based applications. Individuals who were unable to independently complete the digital assessment procedure were excluded. Each participant rated a subset of 36 health and mental health chatbots identified through the systematic app store search using a 5-point Likert scale (1 = “Inadequate,” 2 = “Poor,” 3 = “Acceptable,” 4 = “Good,” 5 = “Excellent”). Prior to the main validation, a pilot test was conducted on 10 chatbots to refine the CHARS rating process and confirm its applicability across participant groups.

Procedure for chatbot rating

The final set of 36 chatbots identified through systematic screening (Fig. 1) was compiled and prepared for evaluation. Data collection sessions were conducted both physically (in supervised group settings) and virtually (through guided online sessions). During these sessions, participants were provided with standardized instructions for interacting with each chatbot. To ensure uniformity in chatbot exposure and minimise variability in user interaction, a standardised prompt-chain protocol (See Supplement 1) was developed and used by all participants when engaging with the chatbots. This prompt chain consisted of a structured sequence of predefined questions and tasks designed to elicit comparable interactions across participants and chatbot systems. The same interaction structure was applied consistently across all evaluations to ensure comparability of responses and rating conditions.

Fig. 1

Flow diagram of the process utilized to identify Chatbots for piloting the Chatbot Rating Scale (CHARS).

Each participant interacted with each assigned chatbot for a minimum of ten minutes to fifteen minutes (and a maximum of thirty minutes) to ensure adequate exposure to functionality and content. Following each interaction, participants completed the Chatbot Rating Scale (CHARS), which was electronically administered via the KoboCollect platform. The CHARS instrument was fully programmed into the system to ensure uniform item presentation, standardised response formatting, and automated data capture. Each participant independently completed ratings on their own device immediately after chatbot interaction. Participants rated a randomized subset of the 36 chatbots to reduce order effects and respondent fatigue. Across the full sample, each chatbot received multiple independent evaluations (more than 25 ratings per chatbot), allowing for reliable aggregation and comparative psychometric analysis.

Data analysis

Data were collected using a highly restricted electronic data capture platform (KoboCollect), which was configured to require responses for all items, thereby ensuring complete datasets with no missing data or incomplete values, which strengthened data integrity and analytical rigor. The completed dataset was exported and analyzed using R version 4.4.2. Descriptive statistics, including frequency counts and percentages, were used to summarize participant characteristics. The psychometric validation of the Chatbot Rating Scale (CHARS) followed a sequential analytic approach beginning with parallel analysis to determine the appropriate number of latent factors. Exploratory factor analysis (EFA) was then conducted to identify the underlying factor structure, with items retained based on factor loadings of ≥ 0.40 and removal of weak or cross-loading items to improve construct clarity. Confirmatory factor analysis (CFA) was subsequently performed to validate the factor structure using maximum likelihood estimation, and model fit was evaluated using standard indices, including Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) values ≥ 0.90, and Root Mean Square Error of Approximation (RMSEA) and Standardized Root Mean Square Residual (SRMR) values ≤ 0.0840,41. In addition, measurement invariance testing was conducted across gender groups to assess the stability and equivalence of the CHARS factor structure between male and female participants, ensuring that the scale measures the same constructs in the same way across demographic groups, which is essential for supporting meaningful group comparisons. Internal consistency reliability was assessed using Cronbach’s alpha to determine the coherence of items within each factor42. All analyses were conducted in R (version 4.4.2) using established psychometric procedures, and collectively supported the validity, reliability, and cross-group comparability of the CHARS instrument for evaluating health and mental health chatbots.

Results

Chatbot rating scale development

We found 3,678 publications after initial search across selected databases and grey literature. After de-duplication, title, abstract and full text review, 32 eligible publications met the eligibility criteria (More details are in Figure S1 in the Supplementary Material 2). Content analysis of these studies generated an initial pool of 67 chatbot evaluation items, which were subsequently iteratively mapped into a five-dimensional framework (Table 1).

Delphi study result

The Delphi process resulted in systematic refinement of the initial 67-item pool through iterative expert evaluation conducted over three rounds. Expert feedback and voting were used to enhance item clarity, relevance, and appropriateness, leading to progressive revision and consolidation of the scale items. The final Delphi evaluation demonstrated strong content validity, with all retained items achieving at least 80% agreement among expert reviewers on relevance, clarity, and appropriateness, indicating a high level of consensus on the adequacy of the final item set. Further, discussions and the 24-member panel review also found CHARS and its items to be theoretically grounded, relevant, clear, and suitable for evaluating the quality of health and social care chatbots across diverse user contexts.

The process produced a final 25-item scale organized into five conceptually coherent constructs: Design, Personalization, Engagement, Usability, and Functionality. These 25 validated items constitute the final CHARS measurement scale which was then validated (Appendix 1).

Systematic review of chatbot and chatbot rating scale piloting

Following the systematic search (Fig. 1) across major app stores, a total of 314 chatbots were initially identified, screened, and assessed for eligibility. After applying predefined inclusion criteria and conducting functionality checks, eligible chatbots were compiled into a final sampling frame. From this pool, 36 eligible health and mental health chatbots were selected for inclusion in the validation study (See Supplement 3 for full list of selected chatbots). Before the main validation, 10 chatbots were used in a pilot test to refine and confirm the applicability of the CHARS rating process. The final 36 chatbots were subsequently evaluated using the CHARS scale by independent experts, technology developers, researchers, and end users.

CHARS psychometric validation

Participants information

Table 2 presents the socio-demographic characteristics of the study participants. The majority were female (69.6%), while males accounted for 30.4%. Nearly half of the respondents (48.8%) identified as beginners in chatbot use, followed by 38.0% with intermediate experience, and only 10.1% reported advanced proficiency. In terms of usage frequency, 31.6% used chatbots daily and most participants (87.2%) accessed chatbots primarily via smartphones.

Table 2 Socio-demographic characteristics of study participants.

Full size table

KMO and Bartlett’s test

The Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy was 0.98, which is considered superb (values above 0.90 indicate excellent suitability for factor analysis). All individual items also demonstrated very high measures of sampling adequacy (MSA) values (ranging from 0.96 to 0.99), confirming that each item contributes adequately to the factor structure. Bartlett’s test of sphericity was significant (χ2 = 21,132.85, df = 300, p < 0.001), indicating that the correlation matrix is not an identity matrix and that the variables are sufficiently correlated to justify factor analysis. Together, these results confirm the appropriateness of proceeding with factor extraction.

Parallel analysis

The factor loadings from the parallel analysis revealed a clear three-factor structure aligning with the conceptual dimensions of the CHARS scale. Factor 1 (MR1), explaining the largest portion of variance (32.9%), is dominated by items related to Usability and Engagement (e.g., USA13–USA20, ENG9–ENG12) with strong loadings ranging from 0.576 to 0.901, indicating a well-defined usability-engagement construct. Factor 3 (MR3), accounting for 12.6% of the variance, represents Functionality, with FUN22–FUN25 loading strongly (0.716–0.881). Factor 2 (MR2), explaining 10.0% of the variance, corresponds to Design, with DES1–DES4 loading between 0.631 and 0.806 (See Appendix 2). Factor 2 (Design) included all four design items (DES1–DES4; loadings 0.631–0.806). Together, these three factors explain 55.5% of the total variance, which is consistent with accepted thresholds in psychometric research on technology acceptance and user experience measures and supports a multidimensional structure of CHARS with distinct but related components. Cross-loadings were minimal, suggesting a clean factor solution.

The scree plot in Fig. 2 from the parallel analysis shows that the eigenvalues of the actual data (blue line) are greater than those of the simulated and resampled data (red lines) only for the first three factors. After the third factor, the actual data line falls below the simulated and resampled lines, indicating that additional factors do not explain more variance than would be expected by chance. This suggests that retaining three factors is appropriate because only these three factors have eigenvalues exceeding those of random data, confirming the three-factor solution identified in your analysis.

Fig. 2

Parallel analysis scree plot indicating three retained factors.

Exploratory factor analysis

Table 3 shows the exploratory factor analysis (EFA) using the minimum residual (minres) method with oblimin rotation confirmed a three-factor solution, which accounted for 68% of the total variance (Factor 1, 38%; Factor 2, 17%; Factor 3, 13%). Factor 1 (MR1) comprised all Usability (USA13–USA21), and Engagement (ENG9–ENG12) items, along with most Personalization (PER5–PER8) items, with strong standardized loadings ranging from 0.36 to 0.90, indicating a consolidated user experience construct. Factor 2 (MR3) represented Functionality (FUN22–FUN25), with high loadings ranging from 0.72 to 0.88, while Factor 3 (MR2) reflected Design (DES1–DES4), loading from 0.63 to 0.81, thus preserving its conceptual distinctiveness. Factor correlations were substantial (MR1–MR3 = 0.77; MR1–MR2 = 0.71), suggesting that these dimensions are related but not redundant. Model fit indices indicated a good fit (RMSEA = 0.055 [90% CI: 0.050–0.061], TLI = 0.958, RMSR = 0.02), and the chi-square test was significant, as expected with large samples. These results confirm that while the initial theoretical model proposed five dimensions, the empirical structure supports three interrelated factors, reflecting how users holistically perceive chatbot quality through design, functionality, and an integrated usability-engagement-personalization experience as shown in Fig. 3.

Table 3 Exploratory factor analysis factor loading result.

Full size table

Fig. 3

Exploratory factor analysis model showing three-factor structure of CHARS.

Confirmatory factor analysis

The results of the Confirmatory Factor Analysis (CFA), provide strong evidence supporting the three-factor model structure identified during the exploratory phase. First, the model fit indices demonstrate an overall acceptable to excellent fit: the Comparative Fit Index (CFI) was 0.998 (scaled = 0.978) and the Tucker-Lewis Index (TLI) was 0.998 (scaled = 0.975), both well above the commonly recommended threshold of 0.90, indicating excellent fit. The Root Mean Square Error of Approximation (RMSEA) was 0.049 (scaled = 0.072), with a 90% confidence interval between 0.043 and 0.054, suggesting a close approximate fit to the data. Additionally, the Standardized Root Mean Square Residual (SRMR) was 0.033, far below the 0.08 cut-off, indicating that residual differences between the observed and model-implied correlations were minimal. Although the chi-square test was significant (χ2 = 589.09, df = 272, p < 0.001), this is expected in large samples and does not indicate poor fit.

Table 4 shows that the factor loadings for all items were high and statistically significant (p < 0.001), with standardized loadings ranging from 0.77 to 0.87 for Factor 1 (which combines items originally from personalization, engagement, and usability), from 0.79 to 0.87 for Factor 2 (Functionality), and from 0.83 to 0.87 for Factor 3 (Design). These high loadings indicate that each item strongly reflects its underlying construct. The corrected item–total correlations for all items across the three factors were consistently high, ranging from 0.723 to 0.814 for Factor 1, 0.705 to 0.788 for Factor 2, and 0.720 to 0.772 for Factor 3. These values exceed the commonly recommended threshold of 0.30, indicating strong internal consistency and that each item contributes meaningfully to the overall scale reliability, suggesting that while distinct, the factors share substantial common variance, which is theoretically plausible given that all dimensions measure aspects of user interaction with chatbots. The results show that the latent factors are highly and positively correlated, with standardized covariances ranging from 0.73 to 0.88, indicating strong conceptual overlap among the three constructs. All correlations are statistically significant (p < 0.001), suggesting that the factors share substantial common variance (See Appendix 2).

Table 4 Pattern matrix of standardized factor loadings (Oblimin rotation).

Full size table

In summary, the CFA confirms that the revised three-factor structure fits the data well and that the items are robust indicators of their respective latent constructs. This finding validates the dimensionality reduction observed during exploratory analysis and provides strong support for using this three-factor model in subsequent analyses and practical applications. The results also suggest that, although the initial conceptualization included five dimensions (design, functionality, usability, engagement, and personalization), these dimensions empirically cluster into three broader, strongly related factors, reflecting a more parsimonious representation of chatbot quality perceptions.

Reliability

The reliability analysis result indicated strong internal consistency for all three extracted factors of the CHARS scale based on Cronbach’s alpha coefficients. Factor 1 (Interactive User Experience) achieved an alpha value of 0.88, reflecting high reliability. Factor 2 (Design) demonstrated excellent consistency with an alpha of 0.90, while Factor 3 (Functionality) also showed strong reliability with an alpha of 0.90. All values exceed the recommended minimum threshold of 0.70, confirming that the items within each factor are internally consistent and reliable. The intraclass correlation coefficient (ICC) for average raters was excellent at 0.97 (95% CI: 0.97–0.98), indicating very high inter-rater reliability for the CHARS scale.

Measurement invariance assessment

Table 5 presents baseline models for males (χ2 = 975.842, df = 272, CFI = 0.976, RMSEA = 0.073) and females (χ2 = 942.163, df = 272, CFI = 0.980, RMSEA = 0.071), both demonstrated good fit, confirming the stability of the hypothesized factor structure across gender. Configural invariance (CFI = 0.979, RMSEA = 0.071) indicated that the same pattern of factor–item relationships held for both groups. Further, metric invariance (CFI = 0.983, RMSEA = 0.063; ΔCFI = 0.004, ΔRMSEA = 0.008) supported the equivalence of factor loadings, while scalar invariance (CFI = 0.981, RMSEA = 0.063; ΔCFI = –0.002) confirmed that intercepts were comparable across groups. Lastly, the strict (structural/residual) invariance model (χ2 = 972.469, df = 647, CFI = 0.990, TLI = 0.991, RMSEA = 0.045, SRMR = 0.039) fit the data excellently. Although chi-square difference testing was significant (Δχ2 = 279.263, Δdf = 9, p < 0.001), the changes in incremental fit indices were negligible (ΔCFI = 0.009, ΔRMSEA = –0.018), supporting strict invariance. This indicates that measurement errors and residual variances are also equivalent across gender.

Table 5 Measurement invariance assessment.

Full size table

Model fit indices (CFI, TLI, RMSEA, SRMR, χ2)

The overall model fit, which was assessed using multiple indices, each providing evidence of good model adequacy. The Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) were 0.95 and 0.94, respectively, which exceed the recommended threshold of 0.90 for acceptable fit and approach the stricter 0.95 standard for excellent fit. The Root Mean Square Error of Approximation (RMSEA) was 0.045, with a 90% confidence interval within the acceptable range (< 0.06), indicating a close fit to the data (Table 6). Similarly, the Standardized Root Mean Square Residual (SRMR) was 0.041, well below the cut-off of 0.08, suggesting minimal residual variance between the observed and predicted covariances.

Table 6 The final chatbot rating scale (CHARS).

Full size table

Discussion

The study presented evidence of the development and validation of the Chatbot Rating Scale (CHARS), a novel instrument for comprehensively evaluating the quality of chatbots. Initial conceptualization of CHARS, grounded in a rigorous modified Delphi process, yielded five theoretically distinct dimensions (Design, Personalization, Engagement, Usability, and Functionality) reflecting expert consensus on chatbot quality and usability. However, subsequent exploratory and confirmatory factor analyses revealed a more parsimonious three-factor structure, in which the EFA merged the original dimensions of personalization, engagement, and usability into a single dimension, subsequently renamed Interactive User Experience, alongside Design (Factor 2) and Functionality (Factor 3). This collapse reflects that these dimensions are conceptually intertwined in end-user’s perception, representing an integrated experiential construct rather than separate qualities. Users appear to evaluate aspects such as visual appeal, conversational flow, and system responsiveness not in isolation but as part of a unified judgment of system fluency and effortlessness, capturing the dynamic and interactive characteristics of chatbot quality, which evolve during real-time user interaction43. This convergence is consistent with theories of cognitive load and information integration, suggesting that end-users form holistic judgments of chatbot quality by combining multiple interactional cues such as visual design, conversational flow, and responsiveness into an overall perception of system effort and ease of interaction44,45.

To our knowledge, CHARS represents the first comprehensive scale developed to assess the multidimensional quality of health and mental health chatbots, advancing beyond existing narrow instruments that focus primarily on single domains46,47,48. In contrast to the Mobile App Rating Scale (MARS)12, which was designed for general mobile health apps and lacks specificity for conversational agents, CHARS offer a more holistic evaluation framework that captures the unique interactive, design, and functional features of chatbots. This broader coverage emphasizes the contribution of CHARS as a psychometrically robust and conceptually comprehensive tool for evaluating the quality of digital conversational agents.

Unlike the few existing digital health scales, CHARS offer a more comprehensive and psychometrically rigorous assessment tool. It improves upon earlier instruments such as the Chatbot Usability Scale (BUS-15), which primarily emphasizes usability and omits critical domains such as design and functionality49, and the Mobile App Rating Scale (MARS), which does not capture chatbot-specific quality features and was not validated through confirmatory factor analysis17. Similarly, broader chatbot service quality measures such as the AI Chatbot Service Quality Scale (AICSQ) focused specifically on functionality and service interactions47, while user experience measures such as the Conversational Agent Scale for User Experience (CASUX) emphasize experiential aspects without addressing performance or technical robustness50. In contrast, CHARS integrate interactive user experience, design, and functionality into a single validated instrument, making it a leading multidimensional quality scale for health and mental health chatbots. Importantly, our study also introduced methodological innovations such as a “prompt chaining” procedure to standardize user engagement with chatbots, ensuring more objective and reliable assessments, an approach not explicitly reported in previous psychometric studies. Our “prompt chaining” technique also contributes to methodological advances in the field. Beyond healthcare, the adaptability of CHARS positions it as a versatile tool for evaluating chatbots in diverse domains such as education, finance, and customer service, highlighting its broader potential impact.

The findings of this study carry notable theoretical and practical implications. First, the observed multidimensional structure of CHARS indicates that end-user assessments of chatbot quality encompass multiple interrelated dimensions, highlighting the complexity of user perceptions in evaluating conversational AI systems. Second, by consolidating overlapping dimensions, the final CHARS structure offers a parsimonious yet comprehensive assessment tool, enhancing its utility for both research and industry applications. For practitioners, these results highlight the need to prioritize integrated design strategies, where aesthetics, interaction flow, and system performance are optimized simultaneously to deliver seamless user experiences.

The development of CHARS adheres to best-practice psychometric standards, including content validation, factor-analytic refinement, and robustness checks using CFA. Nonetheless, the restructuring of dimensions warrants further investigation across diverse cultural, linguistic, and technological contexts to ensure generalizability. Future studies should examine predictive validity, including how CHARS dimensions relate to behavioural outcomes such as continued use, trust formation, and user satisfaction. Longitudinal studies could elucidate whether perceptions of these dimensions remain stable over time or evolve with advancements in conversational AI design. Additionally, while the Interactive User Experience dimension integrates multiple subdomains, this consolidation may mask nuanced differences in user perceptions, suggesting potential refinements in future iterations. While CHARS provide a robust and multidimensional framework for evaluating health and mental health chatbots, its applicability to diverse populations, emerging chatbot technologies, and longitudinal user experiences remains an important area for further evaluation. These considerations complement the strengths of the scale and highlights the need for ongoing refinement as conversational AI continues to evolve.

In conclusion, the CHARS scale is a novel, psychometrically sound, theoretically grounded, and empirically validated instrument for assessing conversational and AI chatbot quality. While grounded in expert consensus, its final structure reflects users’ holistic perception of interactional quality, consolidating overlapping domains into a streamlined three-factor model without compromising content coverage. This evolution from five conceptual dimensions to three empirical factors not only reinforces the primacy of user-centric validation in scale development but also provides actionable insights for designing next-generation conversational agents that balance functionality, personalization, and experiential fluency.

Data availability

The datasets generated and/or analysed during the current study are not publicly available due to the need to protect participant confidentiality, but are available from the corresponding author on reasonable request.

References

  1. Rane, N., Choudhary, S. & Rane, J. Artificial intelligence (AI), Internet of Things (IoT), and blockchain-powered chatbots for improved customer satisfaction, experience, and loyalty. SSRN Electron. J.https://doi.org/10.2139/ssrn.4847274 (2024).

    Article  Google Scholar 

  2. Jeevanandam N. Seven interesting chatbots and AI assistants for 2023 IndiaAI [cited 2023 Nov 5]. Available from: https://indiaai.gov.in/article/seven-interesting-chatbots-ai-assistants-for-2023 (2023).

  3. Singh, B. et al. Systematic review and meta-analysis of the effectiveness of chatbots on lifestyle behaviours. NPJ. Digit. Med.6(1), 118 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Kavitha, B. R. & Murthy, C. R. Chatbot for healthcare system using artificial intelligence. Int. J. Adv. Res. Ideas Innov. Technol.5, 1304–1307 (2019).

    Google Scholar 

  5. Abd-Alrazaq, A. A., Rababeh, A., Alajlani, M., Bewick, B. M. & Househ, M. Effectiveness and safety of using chatbots to improve mental health: Systematic review and meta-analysis. J. Med. Internet Res.22(7), e16021. https://doi.org/10.2196/16021 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Pereira, J. & Díaz, Ó. Using health chatbots for behavior change: A mapping study. J. Med. Syst.43, 41 (2019).

    Article  Google Scholar 

  7. Chow, J. C. L. & Li, K. Large language models in medical chatbots: Opportunities, challenges, and the need to address AI risks. Information16(7), 549. https://doi.org/10.3390/info16070549 (2025).

    Article  Google Scholar 

  8. Laranjo, L. et al. Conversational agents in healthcare: A systematic review. J. Am. Med. Inform. Assoc.25(9), 1248–1258. https://doi.org/10.1093/jamia/ocy072 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Greer, S. et al. Use of the chatbot “Vivibot” to deliver positive psychology skills and promote well-being among young people after cancer treatment: Randomized controlled feasibility trial. JMIR Mhealth Uhealth7(10), e15018 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  10. Rahman, R., Rahman, M.R., Tripto, N.I., Ali, M.E., Apon, S.H., Shahriyar, R. AdolescentBot: understanding opportunities for chatbots in combating adolescent sexual and reproductive health problems in Bangladesh. In Proc. 2021 CHI Conference on Human Factors in Computing Systems. 1–15 (New York, Association for Computing Machinery, 2021).

  11. Vozna, A. Healthcare chatbot development: transforming modern patient care. Glorium Technologies. [cited 2023 Nov 5]. Available from: https://gloriumtech.com/how-to-build-healthcare-chatbots/ (2025).

  12. Precedence Research. Healthcare chatbots market—global industry analysis, size, share, growth, trends, regional outlook, and forecast 2023–2032. Ottawa: Precedence Research; [cited 2023 Nov 5]. Available from: https://www.precedenceresearch.com/healthcare-chatbots-market (2022).

  13. Vaidyam, A. N., Linggonegoro, D. & Torous, J. Changes to the psychiatric chatbot landscape: a systematic review of conversational agents in serious mental illness. Can. J. Psychiatr.66(4), 339–348. https://doi.org/10.1177/0706743720966429 (2021).

    Article  Google Scholar 

  14. Hussain, S., Sianaki, O. A. & Ababneh, N. A survey on conversational agents/chatbots classification and design techniques. In Web, Artificial Intelligence and Network Applications 946–956 (Springer, 2019). https://doi.org/10.1007/978-3-030-15035-8_93.

    Chapter  Google Scholar 

  15. Stoyanov, S. R. et al. Mobile app rating scale: A new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth3(1), e27. https://doi.org/10.2196/mhealth.3422 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  16. Borsci, S., Schmettow, M., Malizia, A., Chamberlain, A. & Van der Velde, F. A confirmatory factorial analysis of the Chatbot Usability Scale: a multilanguage validation. Pers. Ubiquitous Comput.27(2), 317–330 (2023).

    Article  Google Scholar 

  17. Blease, C. et al. Artificial intelligence and the future of primary care: Exploratory qualitative study of UK general practitioners’ views. J. Med. Internet Res.21(3), e12802. https://doi.org/10.2196/12802 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  18. Adamopoulou, E. & Moussiades, L. An overview of chatbot technology. In Artificial Intelligence Applications and Innovations 373–383 (Springer, 2020). https://doi.org/10.1007/978-3-030-49186-4_31.

    Chapter  Google Scholar 

  19. International Organization for Standardization. ISO/IEC 25010: Systems and Software Quality Models (International Organization for Standardization, 2011).

    Google Scholar 

  20. Nielsen, J. Usability Engineering (Morgan Kaufmann, 1994).

    Google Scholar 

  21. Borsci, S. et al. The chatbot usability scale: The design and pilot of a usability scale for interaction with AI-based conversational agents. Pers. Ubiquitous Comput.26(1), 95–119. https://doi.org/10.1007/s00779-021-01582-9 (2022).

    Article  Google Scholar 

  22. O’Brien, H. L. & Toms, E. G. What is user engagement? A conceptual framework for defining user engagement with technology. J. Am. Soc. Inf. Sci. Technol.59(6), 938–955. https://doi.org/10.1002/asi.20801 (2008).

    Article  Google Scholar 

  23. Følstad, A. & Brandtzaeg, P. B. Chatbots and the new world of HCI. Interactions24(4), 38–42. https://doi.org/10.1145/3085558 (2017).

    Article  Google Scholar 

  24. Ischen, C., Araujo, T., Voorveld, H., van Noort, G., Smit, E. Privacy concerns in chatbot interactions. In Chatbot research and design: conversations 2019 Lecture Notes in Computer Science. (eds Følstad, A., Araujo, T., Brandtzaeg, P.B.) https://doi.org/10.1007/978-3-030-39540-7_3 (Cham, Springer; 2020).

  25. Følstad, A., Araujo, T., Papadopoulos, S., Law, E.L.C., Luger, E., Goodwin. M et al. Chatbot research and design. In 6th International Workshop, CONVERSATIONS 2022, Amsterdam, The Netherlands, November 22–23, revised selected papers. Lecture Notes in Computer Sciencehttps://doi.org/10.1007/978-3-031-25581-6 (Cham, Springer, 2023).

  26. Lewis, J. R. & Hardzinski, M. L. Investigating the psychometric properties of the speech user interface service quality questionnaire. Int. J. Speech Technol.18(3), 479–487 (2015).

    Article  Google Scholar 

  27. Hone, K.S., Graham, R. Towards a tool for the subjective assessment of speech system interfaces (SASSI). In Proc. Natural Language Processing Engineering Conference 93–101 (2000).

  28. Koman, J., Fauvelle, K., Schuck, S., Texier, N. & Mebarki, A. Physicians’ perceptions of the use of a chatbot for information seeking: Qualitative study. J. Med. Internet Res.22(11), e15185. https://doi.org/10.2196/15185 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Montenegro, J. L. Z., da Costa, C. A. & da Rosa Righi, R. Survey of conversational agents in health. Expert Syst. Appl.129, 56–67. https://doi.org/10.1016/j.eswa.2019.03.054 (2019).

    Article  Google Scholar 

  30. Zhang, J., Oh, Y. J., Lange, P., Yu, Z. & Fukuoka, Y. Artificial intelligence chatbot behavior change model for designing artificial intelligence chatbots to promote physical activity and a healthy diet: Viewpoint. J. Med. Internet Res.22(9), e22845. https://doi.org/10.2196/22845 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Brooke, J. SUS: A quick and dirty usability scale. In Usability Evaluation in Industry (eds Jordan, P. W. et al.) 189–194 (Taylor & Francis, 1996).

    Google Scholar 

  32. Seethamraju, R. Measurement of user perceived web quality. ECIS Proc.176, (2004).

  33. Aladwani, A. M. & Palvia, P. C. Developing and validating an instrument for measuring user-perceived web quality. Inf. Manag.39(6), 467–476 (2002).

    Article  Google Scholar 

  34. Olsina, L. & Rossi, G. Measuring web application quality with WebQEM. IEEE Multimed.9(4), 20–29 (2002).

    Article  Google Scholar 

  35. Zhou, Y. A mixed methods model of scale development and validation analysis. Meas. Interdiscip. Res. Perspect.17(1), 38–57. https://doi.org/10.1080/15366367.2018.1479088 (2019).

    Article  Google Scholar 

  36. Ashton, L. M. et al. Exploring the design and utility of an integrated web-based chatbot for young adults to support healthy eating: A qualitative study. Int. J. Behav. Nutr. Phys. Act.20(1), 119. https://doi.org/10.1186/s12966-023-01511-4 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  37. Borsci, S. et al. The Chatbot Usability Scale: The design and pilot of a usability scale for interaction with AI-based conversational agents. Personal Ubiquitous Comput.26, 95–119 (2022).

    Article  Google Scholar 

  38. Oh, Y. J., Zhang, J., Fang, M. L. & Fukuoka, Y. A systematic review of artificial intelligence chatbots for promoting physical activity, healthy diet, and weight loss. Int. J. Behav. Nutr. Phys. Act.18(1), 160 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  39. Hasson, F., Keeney, S. & McKenna, H. Research guidelines for the Delphi survey technique. J. Adv. Nurs.32(4), 1008–1015 (2000).

    Article  CAS  PubMed  Google Scholar 

  40. Tavakol, M. & Wetzel, A. Factor analysis: A means for theory and instrument development in support of construct validity. Int. J. Med. Educ.11, 245–247. https://doi.org/10.5116/ijme.5f96.0f4a (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  41. Husain, H. & Aziz, H. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to measure the validity and reliability constructs of historical thinking skills, TPACK and application of historical thinking skills. Int. J. Educ. Psychol. Couns.7(46), 608–623. https://doi.org/10.35631/ijepc.746046 (2022).

    Article  Google Scholar 

  42. Izah, S. C., Sylva, L. & Hait, M. Cronbach’s alpha: A cornerstone in ensuring reliability and validity in environmental health assessment. ES Energy Environ.https://doi.org/10.30919/esee1057 (2023).

    Article  Google Scholar 

  43. Sutcliffe, A. Designing for user engagement. Synth. Lect. Hum. Cent. Inform.https://doi.org/10.2200/S00253ED1V01Y201004HCI008 (2010).

    Article  Google Scholar 

  44. Plass, J. L. et al. (eds) Cognitive Load Theory (Cambridge University Press, 2010).

    Google Scholar 

  45. Fayad, R. & Paper, D. The technology acceptance model e-commerce extension: A conceptual framework. Procedia Econ. Finance26, 1000–1006. https://doi.org/10.1016/S2212-5671(15)00922-3 (2015).

    Article  Google Scholar 

  46. Lewis, J. R. The system usability scale: Past, present, and future. Int. J. Hum. Comput. Interact.34(7), 577–590. https://doi.org/10.1080/10447318.2018.1455307 (2018).

    Article  Google Scholar 

  47. Chen, Q., Gong, Y., Lu, Y. & Tang, J. Classifying and measuring the service quality of AI chatbot in frontline service. J. Bus. Res.145, 552–568. https://doi.org/10.1016/j.jbusres.2022.02.088 (2022).

    Article  Google Scholar 

  48. Faruk, L. I. D., Pal, D., Funilkul, S., Perumal, T. & Mongkolnam, P. Introducing CASUX: A standardized scale for measuring the user experience of artificial intelligence-based conversational agents. Int. J. Hum. Comput. Interact.40(2), 123–140. https://doi.org/10.1080/10447318.2024.2359206 (2024).

    Article  Google Scholar 

  49. Kocielnik, R., Xiao, L., Avrahami, D. & Hsieh, G. BUS-15: a chatbot usability scale. Pers. Ubiquitous Comput.26(5), 1005–1018. https://doi.org/10.1007/s00779-022-01690-0 (2022).

    Article  Google Scholar 

  50. Følstad, A., Nordheim, C. B. & Brandtzaeg, P. B. Conversational Agent Scale for User Experience (CASUX): development and validation of a multidimensional measure for chatbot interactions. Int. J. Hum. Comput. Interact.40(2), 123–140. https://doi.org/10.1080/10447318.2024.2359206 (2024).

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. AI for Low-Resource Public Health Application (ALPHA) Center, Slum and Rural Health Initiative (SRHIN), Ibadan, Nigeria

    Isaac Iyinoluwa Olufadewa, Enoch Olabode Olayori, Miracle Ayomikun Adesina, Blessing Oladokun, Ruth Ifeoluwa Oladele & Toluwase Ayobola Olufadewa

  2. Social Justice and Rural Health Initiative, Chapel Hill, NC, 27514, USA

    Blessing Oladokun & Toluwase Ayobola Olufadewa

  3. University of North Carolina at Charlotte, Charlotte, NC, 28223, USA

    Isaac Iyinoluwa Olufadewa

  4. College of Medicine, University of Ibadan, Ibadan, Nigeria

    Miracle Ayomikun Adesina & Ruth Ifeoluwa Oladele

  5. Institute of Education, University of Ibadan, Ibadan, Nigeria

    Enoch Olabode Olayori

  6. Assessment and Quality Assurance Unit, Department of Science, Mathematics and Technology Education, Faculty of Education, University of Pretoria, Groenkloof, Pretoria, South Africa

    Musa Adekunle Ayanwale

  7. Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, 27599, USA

    Oluwabusolami Ale

  8. Department of Health Behavior, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, 27599, USA

    Maame Araba Oduro

Authors

  1. Isaac Iyinoluwa Olufadewa
  2. Enoch Olabode Olayori
  3. Miracle Ayomikun Adesina
  4. Musa Adekunle Ayanwale
  5. Blessing Oladokun
  6. Ruth Ifeoluwa Oladele
  7. Oluwabusolami Ale
  8. Toluwase Ayobola Olufadewa
  9. Maame Araba Oduro

Contributions

IIO conceptualized the study. IIO, EOO, MAA, MA, BO, RIO, and TAO contributed to methodology and tool development. IIO, EOO, MAA, MA, BO, RIO, OA, TAO, and MAO contributed to the literature review and synthesis. IIO, EOO, MAA, and RIO were involved in data collection. IIO, EOO, MA, and MAA performed data analysis and interpretation. IIO, MAA, and RIO drafted the initial manuscript. IIO, EOO, and MA contributed to the second draft of the manuscript. IIO, EOO, MAA, MA, BO, OA, TAO, and MAO reviewed and edited the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Musa Adekunle Ayanwale.

Ethics declarations

Ethics

Ethical approval for this study was obtained from the Federal Capital Territory (FCT) Health Research Ethics Committee (Approval No: FHREC/2025/01/290/15–09-25). Informed consent was obtained from all participants from eligible individuals as required. All data collected were treated with strict confidentiality to ensure participant anonymity. The study was conducted in accordance with ethical principles that prioritized the welfare, rights, and dignity of all participants while minimizing potential risks or harm.

Consent to participate

All participants provided informed consent to participate in this study.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Olufadewa, I.I., Olayori, E.O., Adesina, M.A. et al. Chatbot rating scale (CHARS): Development and validation of a new tool for assessing the quality of health and social care chatbots. Sci Rep 16, 37213 (2026). https://doi.org/10.1038/s41598-026-68018-3

Download citation

  • Received: 04 November 2025

  • Accepted: 19 August 2026

  • Published: 06 November 2026

  • Version of record: 06 November 2026

  • DOI: https://doi.org/10.1038/s41598-026-68018-3

Keywords

来源:Nature 健康护理主题 · nature.com