跳到正文
原文
Nature 健康护理主题·· 4 天前AI 评分51

生成式AI教师视角量表(GAIE-PM)在医学教育中的开发与心理测量学验证

Development and psychometric validation of the generative AI Educator PerspectoMeter Scale (GAIE-PM) in medical education: a descriptive scale development study

AI 导读

本研究开发并验证了生成式AI教师视角量表(GAIE-PM),用于评估医学教育工作者对生成式AI局限性的认知。研究团队通过文献综述、焦点小组访谈及180名伊朗医学教育工作者的问卷数据,最终形成包含教学局限性、技术/可访问性局限性及伦理与质量局限性三个维度的14条目量表。

正文

Introduction

As technology advances, artificial intelligence (AI) has become increasingly influential in education, healthcare, and health professions education by automating and mimicking human cognitive abilities1,2,3. Within this field, natural language processing has advanced rapidly through generative artificial intelligence and large language models (LLMs). Generative Pretrained Transformer technologies, including widely used applications such as ChatGPT, have changed how learners, educators, researchers, clinicians, and patients interact with digital systems4,5. Although these systems can generate fluent text, they also present limitations, including factual errors, hallucinations, incomplete reasoning, biased outputs, and awkward phrasing that may signal automation to readers. Therefore, while generative AI tools offer valuable support for language processing and educational tasks, their strengths and weaknesses must be considered when evaluating their effectiveness in medical education across teaching, learning, assessment, and medical educational policy contexts6.

Generative AI tools can have advantages in medical education. They may automate parts of scoring, provide detailed feedback on students’ work and assignments, and support interactive learning experiences. They can also assist with exercises, quizzes, translation, question generation, and formative feedback7. Students and researchers may use generative AI to support literature reviews, data analysis, question answering, and concise summarization8. In clinical and educational contexts, AI-supported systems may assist with producing structured clinical notes, organizing patient data, and supporting evidence-informed recommendations under expert supervision9. These tools may also enhance patient engagement through automated communication related to appointment scheduling, medication management, and health information10. In assessment, AI may enhance existing models by generating diverse questions, adapting difficulty levels, supporting grading, and strengthening feedback-oriented assessment practices11,12.

However, the rapid use of generative AI tools in medical education has generated controversy and critical attention. Institutional responses such as bans, restrictions, and policy discussions reflect concerns about academic integrity, assessment validity, reliability, transparency, and possible risks. Although generative AI has demonstrated promising performance in examinations and research-related tasks, concerns remain regarding the accuracy, reproducibility, and transparency of its outputs13,14,15. Scholars have also highlighted challenges related to data quality, knowledge limitations, privacy, ethics, bias, overreliance, and the need for systematic evaluation and collaboration with human teachers. Accordingly, generative AI should complement, rather than replace, human educators and healthcare professionals. Despite its benefits, understanding its limitations and ethical implications is essential for responsible implementation16,17,18.

Recent studies have moved beyond general discussions of generative AI and have increasingly focused on its educational risks, limitations, and implications for medical educators. Parente emphasized that the use of generative AI in medical education should be interpreted cautiously because inaccurate information in this context may influence future clinical reasoning and patient care; key limitations include hallucination, bias, cost, and security concerns19. In a survey of medical educators, Cervantes et al. reported that faculty recognized the potential value of generative AI for research efficiency, task automation, and content accessibility, but also expressed major concerns about assessment cheating, false information, lack of contextual understanding, reduced human interaction in feedback processes, privacy, and the absence of clear institutional guidelines20. These findings indicate that the educational value of generative AI cannot be judged only by technical performance. Rather, its responsible integration into medical education depends on how educators perceive, manage, and respond to its limitations.

Despite the existing body of research on the promise and challenges of generative AI, there is a methodological gap in standardizing how medical educators perceive these limitations. Most current instruments are either too technically focused or tethered to specific, transient software interfaces, which risks rapid obsolescence. Furthermore, there is no validated, multifactoral tool that captures the multifaceted pedagogical, ethical, and clinical concerns of faculty members in a product-agnostic manner. To address this gap, this study aimed to develop and psychometrically validate the Generative AI Educator PerspectoMeter (GAIE-PM) Scale.

Methods

Design

To develop this questionnaire titled Generative AI Educator PerspectoMeter Scale (GAIE-PM), we followed the modified approach indicated in AMEE guide No. 87: “Development of Questionnaires for Educational Research”21. An extensive literature review, qualitative research, the formulation of questionnaire items, expert validation, and a pilot study were conducted during this process.

Questionnaire development

Literature review

A literature review was conducted to identify studies relevant to the development and psychometric validation of instruments assessing medical educators’ perceptions of the limitations, risks, and challenges of generative artificial intelligence in medical education.

The review was designed to inform the conceptual framework, item generation, and content domain of the Generative AI Educator PerspectoMeter Scale (GAIE-PM). The search covered PubMed/MEDLINE, Scopus, Web of Science, ERIC, PsycINFO, IEEE Xplore, and ACM Digital Library. The search strategy was organized around four conceptual clusters: generative AI and large language models, medical educators and health professions faculty, perceived limitations and concerns, and psychometric instrument development and validation. The following Boolean search strategy was adapted and applied across the selected databases.

(“generative AI” OR “generative artificial intelligence” OR “large language model*” OR “LLM*” OR “GPT-3” OR “GPT-4” OR “conversational AI” OR “AI Chatbot*” OR “ChatGPT”) AND (“medical education” OR “health professions education” OR “medical educator*” OR “health professions educator*” OR “health professions faculty” OR “faculty member*” OR “clinical teacher*” OR “medical teacher*”) AND (perception* OR perspective* OR attitude* OR belief* OR acceptance OR readiness OR trust OR concern* OR limitation* OR challenge* OR barrier* OR risk* OR “ethical concern*” OR bias OR privacy OR security OR accuracy OR hallucination* OR reliability OR transparency) AND (scale OR instrument OR questionnaire OR survey OR measure* OR psychometric* OR validation OR “scale development” OR “instrument development” OR “factor analysis” OR “exploratory factor analysis” OR “confirmatory factor analysis” OR “content validity” OR “construct validity”).

The literature review followed predefined eligibility criteria during the screening process. Articles were included if they were peer-reviewed empirical studies, systematic reviews, or conceptual frameworks focusing on generative artificial intelligence or large language models (LLMs) within higher education or health professions education, studies explicitly examining medical educators, health professions faculty members, or clinical teachers, articles investigating perceptions, attitudes, concerns, risks, or perceived limitations regarding AI integration in education, and articles published in English. Conversely, studies focusing exclusively on student perspectives without evaluating faculty members or medical educators, articles centered solely on clinical diagnostic AI applications or general computing without an educational factor, non-peer-reviewed commentaries, general editorials, opinion pieces, and conference abstracts lacking full methodological data, as well as publications in languages other than English, were excluded.

The search yielded 1,468 records. After duplicate removal, 1,120 records remained for title and abstract screening. Following this initial screening, 975 records were excluded, leaving 145 articles for full-text assessment. Of these, 90 articles were excluded because they did not meet the inclusion criteria, and 55 articles were retained for the final synthesis, as shown in Fig. 1.

Fig. 1

PRISMA flow diagram.

Qualitative study

This qualitative study employed conventional content analysis to explore Iranian experts’ perceptions of the limitations of generative AI in medical education and to inform the conceptual domains and item generation process of the Generative AI Educator PerspectoMeter Scale (GAIE-PM).

The Technology Acceptance Model (TAM)22 was used as a sensitizing conceptual lens for considering that may influence educators’ acceptance of educational technologies, particularly perceived usefulness, and perceived ease of use, attitudes, and intention to use. However, the analysis was not restricted to predefined TAM categories and remained open to themes emerging from participants’ experiences and expert discussions. This approach enabled the identification of broader limitation domains relevant to medical education, including pedagogical, technical, ethical, professional, assessment-related, and implementation concerns.

A purposeful sampling strategy was employed to recruit 25 experts from three key domains: medical education (10 participants, including professors and curriculum designers), computer engineering/artificial intelligence (8 participants, including developers and researchers with natural language processing expertise), and healthcare specialists (7 participants, including clinicians and ethicists with experience in technology integration). Participants were selected based on their extensive experience in artificial intelligence, medical education, educational technology, or technology-driven educational reforms. This approach ensured that the sample included diverse perspectives relevant to the integration of generative AI into medical education.

Three virtual focus groups were conducted on the Clubhouse platform to gather experts’ views on the limitations, risks, and educational implications of generative AI tools. Each session was moderated by two researchers and followed a structured format. The sessions began with a 10-minute introduction to the role of generative AI in medical education, followed by a 45-minute guided discussion based on questions derived from the literature review. These questions focused on topics such as the impact of accuracy limitations and hallucinations on medical training, ethical safeguards needed to address bias and data privacy concerns, the influence of generative AI on medical students’ critical thinking, and the implications of AI-generated content for assessment integrity and feedback quality. A 20-minute open discussion then allowed participants to discuss possible strategies for addressing language-specific limitations, including challenges related to Persian-language performance, and to explore potential applications of generative AI in educational exercises, formative assessment, and faculty support.

The qualitative analysis followed the approach developed by Graneheim and Lundman23, which emphasizes the identification of both manifest and latent content within textual data. This method involves several key concepts: meaning units, condensed meaning units, codes, subcategories, categories, and themes. Recorded focus group sessions were transcribed verbatim for accuracy, and MAXQDA software was used to organize and analyze the data systematically. Open coding was first employed to identify meaning units, which were then condensed and labelled as codes. Related codes were compared and grouped into subcategories, which were further abstracted into main categories and overarching themes. This process resulted in 170 subcategories, 40 categories, and 8 themes addressing various factors of generative AI limitations in medical education.

To ensure the trustworthiness of the qualitative findings, several strategies were applied. Credibility was supported through member checking by sharing preliminary findings with participants for verification. Triangulation with existing literature on generative AI limitations was performed to compare the emerging findings with broader research evidence. Dependability was strengthened by involving three researchers in the coding and analysis process, and discrepancies during coding were resolved through discussion and consensus. Confirmability was enhanced by maintaining an audit trail of coding decisions, category development, and theme refinement. The qualitative findings provided the empirical foundation for developing the initial item pool of the GAIE-PM Scale.

Development of items

The item development process was guided by the convergence between the limitation domains identified in the qualitative study and those extracted from the literature review. The Technology Acceptance Model (TAM) was used only as a sensitizing framework to ensure that item generation considered issues related to perceived usefulness, ease of use, attitudes, and intention to use. However, because the GAIE-PM Scale was designed to assess perceived limitations of generative AI in medical education, the item pool was not restricted to TAM constructs. Instead, similar or overlapping limitations were compared, synthesized, and translated into questionnaire items. This process resulted in an initial pool of 42 items for the GAIE-PM Scale.

Expert validation

Face validity

Face validity was assessed using both qualitative and quantitative approaches. Ten medical educators evaluated the preliminary items in terms of clarity, relevance, comprehensibility, ambiguity, and difficulty. Quantitative face validity was assessed using the item impact method. Experts rated the importance of each item on a five-point Likert scale, and the item impact score was calculated by multiplying the frequency of experts rating the item as important by the mean importance score. Items with acceptable item impact scores were retained. In addition, qualitative comments from the experts were reviewed and used to revise item wording, reduce ambiguity, improve clarity, and enhance the contextual appropriateness of the questionnaire items.

Content validity

Content validity was assessed by a panel of eight experts in medical education, educational psychology, and technology-enhanced learning. Experts were selected based on their academic qualifications, experience in medical education, involvement in curriculum development or educational research, and familiarity with technology integration in medical education. The content validity index (CVI) was calculated to assess the relevance, clarity, and simplicity of each item using a four-point rating scale. Items with CVI values above 0.79 were considered acceptable24,25.

The content validity ratio (CVR) was also calculated to assess the essentiality of each item. Experts rated each item using a three-point scale: essential, useful but not essential, or not essential. CVR values were calculated using Lawshe’s formula26, and items below the acceptable threshold based on the number of experts were removed. Based on the combined results of the CVI and CVR assessments, 24 items were removed, and 18 items were retained for pilot testing and preliminary factor analysis.

Pilot testing and data collection

A pilot testing phase was conducted to evaluate the preliminary psychometric properties of the GAIE-PM Scale. Following expert validation, 18 items were retained for testing. Based on the widely accepted psychometric recommendation of five to ten participants per item, a target sample size of 180 respondents (10:1 ratio) was determined to ensure sufficient statistical power for both exploratory and confirmatory analyses. The online questionnaire was distributed via email and social media, and data collection concluded upon receiving 180 complete responses from medical educators across diverse educational settings in Iran. Eligible participants were required to have experience in online education or AI-based tools, and any incomplete or inconsistent responses were excluded. To ensure the scale’s psychometric robustness, we employed a sequential analytical strategy on the collected sample. Rather than splitting the sample, which could reduce statistical power, we utilized the entire dataset to maximize the information density of each response. In the exploratory phase, the data suitability was confirmed via the Kaiser–Meyer–Olkin (KMO) measure and Bartlett’s test of sphericity, followed by Principal Factor Analysis (PCA) with Varimax Rotation to inductively identify the latent factors. Subsequently, the confirmatory phase utilized Partial Least Squares Structural Equation Modeling (PLS-SEM) to deductively verify the measurement model’s stability and structural validity. This sequential approach—moving from inductive factor identification to deductive structural verification—provides a more rigorous validation process and strengthens the overall psychometric evidence of the scale within the medical education context. Internal consistency was evaluated using Cronbach’s alpha, McDonald’s omega (ω), and composite reliability (CR). Convergent and discriminant validity were assessed using Average Variance Extracted (AVE) and the Fornell–Larcker criterion, respectively, while predictive relevance was examined through Q² and R² values. Finally, temporal stability was assessed with a subsample of 30 educators over a two-week interval, using the Intraclass Correlation Coefficient (ICC) and Pearson correlation coefficients to ensure absolute agreement.

Ethical considerations

The study was conducted in accordance with the ethical principles outlined by the Declaration of Helsinki and received approval from the Ethics Committee of the Smart University of Medical Sciences (Code: IR.VUMS.REC.1402.017). Prior to participation, all participants were provided with a detailed explanation of the study’s objectives, procedures, potential risks, and benefits. Participation was entirely voluntary, and participants were informed of their right to withdraw from the study at any time without penalty. Informed consent was obtained from each participant through a written consent form, which included assurances of confidentiality and anonymity. Data was securely stored and accessed only by authorized researchers. All procedures were performed in compliance with relevant guidelines and regulations to ensure the protection of participants’ rights and welfare.

Results

As shown in Tables 1 and 97 participants were male (53.88%), 82 were female (45.56%), and one participant did not report sex (0.56%). Regarding academic rank, assistant professors formed the largest group (33.33%), followed by others such as instructors, lecturers, and research fellows (31.11%), associate professors (20.00%), and full professors (15.56%). The largest age group was 41–50 years (34.44%), and the most common work experience category was 10–20 years (33.33%).

Table 1 Demographic information about participants.

Full size table

Table 2 summarizes medical educators’ perceptions of generative AI limitations in medical education based on the 18 retained pilot items. Participants reported moderate to high levels of concern across pedagogical, technical and access, and ethical and quality domains. The main concerns were related to students’ critical thinking and creativity, integration with existing teaching methods, accessibility, privacy and data security, accuracy and reliability, bias, lack of clear guidelines, and potential conflicts with social norms.

Table 2 Descriptive Statistics of the 18 retained pilot items of the GAIE-PM scale.

Full size table

Literature review

The literature review suggested three broad domains relevant to educators’ perceptions of generative AI limitations in medical education: pedagogical limitations, technical limitations, and ethical/implementation-related concerns. Pedagogical limitations included possible effects on critical thinking, problem-solving, assessment quality, depth of learning, and the development of practical clinical skills. Technical limitations included accuracy, reliability, hallucination, accessibility, integration with existing educational systems, usability, and dependence on digital infrastructure. Ethical and implementation-related concerns included privacy, data security, bias, accountability, lack of clear guidelines, and the need for responsible institutional governance. These domains informed the focus group guide and contributed to the development of the initial item pool for the GAIE-PM Scale.

Qualitative study

Qualitative analysis generated 432 initial codes, which were subsequently organized into 170 subcategories, 40 categories, and eight overarching themes related to the limitations of generative AI in medical education. As shown in Table 3, these themes and categories represented the main conceptual domains identified from expert discussions and informed the development of the initial item pool for the GAIE-PM Scale.

Table 3 An overview of the themes and findings of the qualitative study.

Full size table

Development of items

The item development process was guided by the convergence between the limitation domains identified in the qualitative study and those extracted from the literature review. Similar or overlapping limitations were compared, synthesized, and translated into questionnaire items. Each item was designed to assess medical educators’ perceptions of a specific limitation of generative AI in medical education and was formatted using a Likert-type response scale. This process resulted in an initial pool of 42 items for the GAIE-PM Scale.

Expert validation

Face and content validity

The preliminary item pool was evaluated by a panel of experts in terms of relevance, clarity, and simplicity. The content validity index (CVI) values were higher than 0.79, indicating acceptable content validity. In the content validity ratio (CVR) assessment, 24 items did not meet the predetermined acceptable threshold and were removed. As a result, 18 items were retained for pilot testing and preliminary factor analysis.

Pilot testing

Principal factor analysis (PCA)

Before examining the preliminary factor structure of the scale, the suitability of the data was assessed using the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity. The KMO value was 0.81, indicating that the data were suitable for principal factor analysis. Bartlett’s test of sphericity was statistically significant, χ² (153) = 799.85, p < 0.001, confirming that the correlation matrix was appropriate for factor analysis (Table 4).

Table 4 KMO and Bartlett’s test.

Full size table

Principal Factor Analysis with Varimax rotation and Kaiser Normalization was used as a preliminary procedure to examine the empirical grouping of items during pilot testing. PCA was used to explore the preliminary factor structure and to inform subsequent item evaluation, rather than to model factor latent. Three interpretable factors were retained and labelled as Pedagogical Limitations, Technical and Access Limitations, and Ethical and Quality Limitations. Together, these factors explained 48.37% of the total variance. The retention of the three-factor structure was supported by eigenvalues, scree plot inspection, and conceptual interpretability. The eigenvalues and percentage of variance explained by each retained factor are presented in Table 5. Visual inspection of the scree plot also supported the retention of a three-factor exploratory structure (Fig. 2).

Table 5 Eigenvalues and percent variance explained by each extracted factor.

Full size table

Fig. 2

Scree plot supporting the three-factoral exploratory structure of the scale.

The rotated factor loadings are presented in Table 6. Items were interpreted according to their highest loading and conceptual relevance to the corresponding factor. The first factor, Pedagogical Limitations, included items related to students’ understanding, critical thinking, creativity, teaching integration, complex medical topics, and the role of medical educators. The second factor, Technical and Access Limitations, included items related to privacy and security, technical barriers, complexity of use, accessibility, workload, and lack of clear guidelines. The third factor, Ethical and Quality Limitations, included items related to ethical implications, social norms, accuracy and reliability, quality and depth of learning, practical skills, and potential bias or discrimination.

Table 6 Rotated factor loadings from preliminary principal factor analysis.

Full size table

Partial least squares structural equation modeling (PLS-SEM)

SmartPLS was used to assess the measurement model of the GAIE-PM Scale. The assessment focused on internal consistency reliability, convergent validity, discriminant validity, predictive relevance, and the higher-order structure of the scale.

The GAIE-PM Scale was specified as a higher-order measurement model. Based on the theoretical assumption that pedagogical, technical, and ethical limitations represent distinct yet interrelated manifestations of a broader construct, the overall construct was modeled as a higher-order factor (F). The three first-order factors—Pedagogical Limitations, Technical and Access Limitations, and Ethical and Quality Limitations—were retained as distinct domains contributing to the broader construct of perceived generative AI limitations in medical education. Thus, the higher-order construct was not intended to replace the three factors; rather, it provided an overall representation of perceived generative AI limitations while preserving the multifactoral structure of the GAIE-PM Scale.

Table 7 presents the reliability, convergent validity, and higher-order model indicators of the GAIE-PM Scale. Cronbach’s alpha values ranged from 0.69 to 0.81 across the three factors. Although the alpha value for Pedagogical Limitations was slightly below 0.70, it was considered acceptable in the context of early-stage scale development, particularly because the composite reliability value for this factor exceeded 0.800. Composite reliability values ranged from 0.81 to 0.86, supporting internal consistency reliability. Average variance extracted values ranged from 0.50 to 0.57, indicating acceptable convergent validity.

Table 7 PLS-SEM measurement model assessment of the GAIE-PM scale.

Full size table

The R² values ranged from 0.55 to 0.66, suggesting that the higher-order construct explained a moderate to substantial proportion of variance in the three first-order factors. The Q² values ranged from 0.27 to 0.36, indicating acceptable predictive relevance. These findings supported the adequacy of the higher-order measurement model.

Discriminant validity was assessed using the Fornell–Larcker criterion. As shown in Table 8, the square root of the AVE for each construct was greater than its correlations with the other constructs, supporting the discriminant validity of the three factors. In addition, the retained items demonstrated acceptable standardized indicator loadings, all exceeding 0.40 and statistically significant at p < 0.05, indicating that the items adequately represented their corresponding constructs.

Reliability and stability analyses

The internal consistency reliability of the GAIE-PM Scale was evaluated using Cronbach’s alpha and McDonald’s omega coefficients across the three first-order factors. Cronbach’s alpha values ranged from 0.69 to 0.81, with a mean alpha coefficient of 0.75. Specifically, Cronbach’s alpha was 0.69 for Pedagogical Limitations, 0.81 for Technical and Access Limitations, and 0.75 for Ethical and Quality Limitations. Although the alpha coefficient for the Pedagogical Limitations factor was slightly below the conventional threshold of 0.70, the composite reliability and McDonald’s omega values indicated acceptable internal consistency. McDonald’s omega further supported the reliability of the three factors, with a mean omega coefficient of 0.81 (Fig. 3).

Table 8 Discriminant validity assessment using the Fornell–Larcker criterion.

Full size table

Fig. 3

Higher-order measurement model of the GAIE-PM scale with standardized indicator loadings. F represents the higher-order construct of Overall Perceived Limitations of Generative AI in Medical Education. F1, F2, and F3 represent the three first-order factors of the GAIE-PM Scale: Pedagogical Limitations, Technical and Access Limitations, and Ethical and Quality Limitations, respectively. The three subscales were retained as distinct factors and contributed to the total GAIE-PM score.

Item removal and model refinement

During PLS-SEM measurement model refinement, four items were removed from the final model: V8 and V13 from Pedagogical Limitations, V18 from Technical and Access Limitations, and V10 from Ethical and Quality Limitations. Items were removed after examining indicator loadings, cross-loadings, and their contribution to construct reliability and content representation. The retained items continued to represent the three intended factors of the GAIE-PM Scale. In the final higher-order PLS-SEM model, the paths from the overall construct to the three first-order factors were positive, supporting the contribution of each factor to the broader construct of perceived generative AI limitations in medical education. The overall Goodness-of-Fit value was 0.56, exceeding the recommended threshold of 0.36 for strong model fit. Additionally, the Standardized Root Mean Square Residual (SRMR) for the measurement model was calculated at 0.065. This value is well below the conservative threshold of 0.08, further confirming an acceptable and strong model fit. Taken together, the findings concerning reliability, convergent validity, discriminant validity, indicator loadings, predictive relevance, and overall model assessment suggest satisfactory psychometric properties of the GAIE-PM Scale.

Infographic 1 presents the final three-factoral structure of the GAIE-PM Scale and its 14 retained items.

Infographic 1

Final structure of the GAIE-PM scale.

Scoring and interpretation of the GAIE-PM scale

The GAIE-PM total score was further justified based on the supported higher-order measurement model, in which the three factors (Pedagogical Limitations, Technical and Access Limitations, and Ethical and Quality Limitations) represent distinct but interrelated domains of a broader construct of perceived generative AI limitations in medical education. Therefore, the overall score was calculated by summing the 14 retained items, with a possible score range of 14–70; higher scores indicate greater perceived limitations. The reliability of the overall score was confirmed by acceptable internal consistency (Cronbach’s α = 0.86; McDonald’s ω = 0.88). In addition to the total score, factor-specific scores were retained to preserve the multifactoral structure of the scale: Pedagogical Limitations (4 items; range: 4–20), Technical and Access Limitations (5 items; range: 5–25), and Ethical and Quality Limitations (5 items; range: 5–25). Respondents with more than 20% missing responses were excluded, while sporadic missing values (< 10%) were handled using mean imputation within the corresponding factor. The GAIE-PM total score is intended for faculty development needs assessment and research purposes rather than diagnostic classification.

Discussion

This study developed and psychometrically evaluated the GAIE-PM Scale, a 14-item instrument for assessing medical educators’ perceived limitations of generative AI in medical education. The final structure consisted of three factors: Pedagogical Limitations, Technical and Access Limitations, and Ethical and Quality Limitations. This structure indicates that educators’ concerns are multifactoral and include educational, implementation-related, and ethical-quality domains rather than representing a single general attitude toward generative AI.

The multifactoral structure of the GAIE-PM is consistent with recent medical education literature emphasizing that generative AI integration requires attention to curriculum design, institutional policy, governance, faculty preparation, equity, and implementation readiness. Recent publications have argued that medical schools need structured approaches to AI integration rather than isolated or informal use of generative AI tools27,28,29,30. The present study extends this literature by translating these broad concerns into measurable domains from the perspective of medical educators.

The Pedagogical Limitations factor reflects concerns about critical thinking, creativity, complex medical topics, practical skills, and the continuing role of human educators. Recent literature in health professions education similarly indicates that generative AI may support learning access, efficiency, feedback, and personalization, but may also create risks related to superficial learning, overreliance, academic integrity, and insufficient development of independent reasoning31,32. The present findings therefore suggest that educators’ pedagogical concerns are not simply resistance to innovation, but are related to the quality of learning, the development of clinical reasoning, and the preservation of educator-guided judgment.

The findings also have important epistemic implications. Generative AI systems can produce fluent, coherent, and persuasive responses, but such outputs should not be equated with understanding, truth, or educational quality. In medical education, this distinction is particularly important because learners are not expected merely to retrieve information; they must learn how to evaluate evidence, justify clinical judgments, recognize uncertainty, and take responsibility for decisions. Recent educational scholarship has emphasized that generative AI changes how knowledge is produced, accessed, validated, and trusted in learning environments33,34. These changes raise concerns about epistemic authority because learners may treat AI-generated responses as authoritative even when they are incomplete, inaccurate, or insufficiently contextualized. Other scholars have argued that responsible use of generative AI requires preserving human epistemic agency, meaning that learners and educators must remain active in questioning, verifying, interpreting, and contextualizing AI-generated outputs rather than passively accepting them35,36. From this perspective, the Pedagogical Limitations and Ethical and Quality Limitations factors of the GAIE-PM are closely connected: concerns about critical thinking, complex medical topics, depth of learning, reliability, and the role of human educators all reflect the need to preserve judgment, practical wisdom, and professional responsibility in AI-supported medical education.

The Technical and Access Limitations factor reflects concerns about usability, technical barriers, accessibility, workload, and equitable access. These findings are consistent with recent studies showing that medical schools need formal policies, faculty development, structured training, and institutional support to use AI and generative AI responsibly in medical education37,38,39. This suggests that technical and access-related concerns should not be considered secondary issues; they may directly affect whether educators can integrate generative AI safely, equitably, and effectively into teaching. In this sense, technical limitations are not merely operational problems; they are part of the institutional conditions that shape responsible educational implementation.

The Ethical and Quality Limitations factor captures concerns about accuracy, reliability, privacy, bias, accountability, and learning quality. Recent medical education and healthcare literature has emphasized the need for human supervision, critical appraisal of AI-generated outputs, ethical guidance, and responsible implementation of LLM-based tools in educational and clinical learning contexts40,41. This is particularly important in medical education because inaccurate or poorly evaluated AI-generated information may influence learners’ reasoning and, indirectly, future patient care. Therefore, ethical and quality-related concerns should be interpreted as core educational concerns rather than peripheral risks.

AI literacy is central to interpreting GAIE-PM scores. AI literacy includes understanding AI capabilities and limitations, evaluating AI-generated outputs, recognizing ethical and social implications, and using AI tools responsibly42,43,44. Because this study did not directly measure AI literacy, digital literacy, prior generative AI use, or confidence in evaluating AI-generated outputs, educators’ perceptions may have been influenced by differences in exposure, competence, and familiarity. High GAIE-PM scores may reflect informed professional caution, but they may also reflect uncertainty or limited experience with generative AI. Conversely, low scores should not automatically be interpreted as readiness for responsible implementation, because they may reflect insufficient awareness of AI-related risks. Future research should therefore examine whether GAIE-PM scores differ according to educators’ AI literacy, prior experience with generative AI tools, and participation in AI-related faculty development.

The findings may also be interpreted within the broader context of technology-mediated education in medical sciences. Previous studies have shown that online learning systems are shaped by professional functions and contextual conditions, that social network-based education can be used as a structured health education intervention, and that learner outcomes may be influenced by psychological such as anxiety and academic burnout45,46,47. Although these studies did not examine generative AI directly, they support the broader interpretation that educational technologies should be understood within pedagogical, contextual, professional, and learner-related conditions rather than as isolated technical tools.

From a practical perspective, the GAIE-PM may help institutions identify which domains of concern are most prominent among medical educators. High Pedagogical Limitations scores may indicate a need for faculty development focused on instructional design, critical thinking, clinical reasoning, and responsible use of AI-supported learning activities. High Technical and Access Limitations scores may indicate a need for infrastructure, technical support, access policies, and workload planning. High Ethical and Quality Limitations scores may indicate a need for guidance on accuracy checking, privacy, bias, accountability, academic integrity, and quality assurance. Therefore, interpretation of the scale should include both the total score and the three subscale scores.

Several limitations should be considered. First, the study assessed perceived limitations, not the actual performance, safety, or educational impact of generative AI systems. Therefore, high GAIE-PM scores should not be interpreted as evidence that generative AI is ineffective, and low scores should not be interpreted as evidence of readiness for safe implementation. Second, the study was conducted among medical educators in Iran, and the generalizability of the findings to other cultural, institutional, and regulatory contexts requires further testing. Third, although the study provides initial evidence of internal structure, reliability, convergent validity, discriminant validity, and temporal stability, additional validation is needed. Future research should examine measurement invariance across institutions and countries, relationships with AI literacy and AI-use behavior, responsiveness to faculty development interventions, and predictive utility for institutional planning. Future studies should validate the GAIE-PM Scale in larger, more diverse, and international samples and examine its relationship with AI literacy, prior AI use, faculty development, and institutional readiness.

Conclusion

The GAIE-PM Scale showed acceptable preliminary psychometric properties for assessing medical educators’ perceived limitations of generative AI in medical education. The final 14-item scale retained a three-factoral structure consisting of Pedagogical Limitations, Technical and Access Limitations, and Ethical and Quality Limitations, supporting the multifactoral nature of educators’ concerns.

The reliability and validity findings, including acceptable internal consistency, composite reliability, convergent validity, discriminant validity, and test–retest stability, support the use of the GAIE-PM as an initial instrument for needs assessment and institutional planning. The scale may help medical education leaders identify specific areas requiring faculty development, technical support, ethical guidance, and quality assurance before or during generative AI integration.

References

  1. Sun, L., Yin, C., Xu, Q. & Zhao, W. Artificial intelligence for healthcare and medical education: a systematic review. Am. J. Transl. Res. 15 (7), 4820–4828 (2023).

    PubMed  PubMed Central  Google Scholar 

  2. Azer, S. A. & Guerrero, A. P. S. The challenges imposed by artificial intelligence: are we ready in medical education? BMC Med. Educ. 23, 680. https://doi.org/10.1186/s12909-023-04660-z (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  3. Pucchio, A. et al. Exploration of exposure to artificial intelligence in undergraduate medical education: a Canadian cross-sectional mixed-methods study. BMC Med. Educ. 22, 815. https://doi.org/10.1186/s12909-022-03896-5 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Tolsgaard, M. G. et al. The fundamentals of artificial intelligence in medical education research: AMEE guide 156. Med. Teach. 45 (6), 565–573. https://doi.org/10.1080/0142159X.2023.2180340 (2023).

    Article  PubMed  Google Scholar 

  5. Masters, K. Ethical use of artificial intelligence in health professions education: AMEE guide 158. Med. Teach. 45 (6), 574–584. https://doi.org/10.1080/0142159X.2023.2186203 (2023).

    Article  PubMed  Google Scholar 

  6. Masters, K. et al. Artificial intelligence in health professions education assessment: AMEE guide 178. Med. Teach. 47 (9), 1410–1424. https://doi.org/10.1080/0142159X.2024.2445037 (2025).

    Article  PubMed  Google Scholar 

  7. Boscardin, C. K., Gin, B., Golde, P. B. & Hauer, K. E. ChatGPT and generative artificial intelligence for medical education: potential impact and opportunity. Acad. Med. 99 (1), 22–27. https://doi.org/10.1097/ACM.0000000000005439 (2024).

    Article  PubMed  Google Scholar 

  8. Parente, D. J. Generative artificial intelligence and large language models in primary care medical education. Fam. Med. 56 (9), 534–540. https://doi.org/10.22454/FamMed.2024.775525 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Cervantes, J. et al. Decoding medical educators’ perceptions on generative artificial intelligence in medical education. J. Investig. Med. 72 (7), 633–639. https://doi.org/10.1177/10815589241257215 (2024).

    Article  PubMed  Google Scholar 

  10. Pham, T. D. et al. The impact of generative AI on health professional education: a systematic review in the context of student learning. Med. Educ. 59 (12), 1280–1289. https://doi.org/10.1111/medu.15746 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Szabó, A. & Laein, G. D. Comparative evaluation of large language models performance in medical education using urinary system histology assessment. Sci. Rep. 15, 31933. https://doi.org/10.1038/s41598-025-17571-4 (2025).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Singh, S. et al. The pitfalls of multiple-choice questions in generative AI and medical education. Sci. Rep. 15, 42096. https://doi.org/10.1038/s41598-025-26036-7 (2025).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Arpaci, I., Kuşci, I. & Gibreel, O. The role of personality traits in predicting educational use of generative AI in higher education. Sci. Rep. 15, 30440. https://doi.org/10.1038/s41598-025-16339-0 (2025).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Fan, L., Deng, K. & Liu, F. Educational impacts of generative artificial intelligence on learning and performance of engineering students in China. Sci. Rep. 15, 26521. https://doi.org/10.1038/s41598-025-06930-w (2025).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Jowsey, T., Stokes-Parish, J., Singleton, R. & Todorovic, M. Medical education empowered by generative artificial intelligence large language models. Trends Mol. Med. 29 (12), 971–973. https://doi.org/10.1016/j.molmed.2023.08.012 (2023).

    Article  PubMed  Google Scholar 

  16. Yan, L. et al. Practical and ethical challenges of large language models in education: a systematic scoping review. Br. J. Educ. Technol. 55 (1), 90–112. https://doi.org/10.1111/bjet.13370 (2024).

    Article  Google Scholar 

  17. D’Souza, R. F., Mathew, M., Mishra, V. & Surapaneni, K. M. Twelve tips for addressing ethical concerns in the implementation of artificial intelligence in medical education. Med. Educ. Online. 29 (1), 2330250. https://doi.org/10.1080/10872981.2024.2330250 (2024).

    Article  Google Scholar 

  18. Cross, J. L., Choma, M. A. & Onofrey, J. A. Bias in medical AI: implications for clinical decision-making. PLoS Digit. Health. 3 (11), e0000651. https://doi.org/10.1371/journal.pdig.0000651 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Li, Q. & Qin, Y. AI in medical education: medical student perception, curriculum recommendations and design suggestions. BMC Med. Educ. 23, 852. https://doi.org/10.1186/s12909-023-04700-8 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Cooper, A. & Rodman, A. AI and medical education: a 21st-century Pandora’s box. N. Engl. J. Med. 389 (5), 385–387. https://doi.org/10.1056/NEJMp2304993 (2023).

    Article  PubMed  Google Scholar 

  21. Artino, A. R., La Rochelle, J. S., Dezee, K. J. & Gehlbach, H. Developing questionnaires for educational research: AMEE Guide 87. Med. Teach 36 (6), 463–474. https://doi.org/10.3109/0142159X.2014.889814 (2014).

    Article  Google Scholar 

  22. Davis, F. D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13 (3), 319–340. https://doi.org/10.2307/249008 (1989).

    Article  Google Scholar 

  23. Graneheim, U. H. & Lundman, B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ. Today 24 (2), 105–112. https://doi.org/10.1016/j.nedt.2003.10.001 (2004).

    Article  CAS  PubMed  Google Scholar 

  24. Connell, J. et al. The importance of content and face validity in instrument development: lessons learnt from service users when developing the Recovering Quality of Life measure (ReQoL). Qual. Life Res. 27 (7), 1893–1902. https://doi.org/10.1007/s11136-018-1847-y (2018).

    Article  Google Scholar 

  25. Polit, D. F., Beck, C. T. & Owen, S. V. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res. Nurs. Health 30 (4), 459–467. https://doi.org/10.1002/nur.20199 (2007).

    Article  PubMed  Google Scholar 

  26. Lawshe, C. H. A quantitative approach to content validity. Pers. Psychol 28 (4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x (1975).

    Article  Google Scholar 

  27. Triola, M. M. & Rodman, A. Integrating generative artificial intelligence into medical education: curriculum, policy, and governance strategies. Acad. Med. 100 (4), 413–418. https://doi.org/10.1097/ACM.0000000000005963 (2025).

    Article  PubMed  Google Scholar 

  28. Hersh, W. & Fultz Hollis, K. Results and implications for generative AI in a large introductory biomedical and health informatics course. NPJ Digit. Med. 7, 247. https://doi.org/10.1038/s41746-024-01251-0 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Wachter, R. M. Artificial intelligence in medical education. Acad. Med. 100 (9 Suppl), S1–S2. https://doi.org/10.1097/ACM.0000000000006101 (2025).

    Article  PubMed  Google Scholar 

  30. Feigerlova, E., Hani, H. & Hothersall-Davies, E. A systematic review of the impact of artificial intelligence on educational outcomes in health professions education. BMC Med. Educ. 25, 129. https://doi.org/10.1186/s12909-025-06719-5 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Boman, M. Scaffolding for success: blending learning with and about generative AI in medical education. Med. Teach. 47 (12), 1911–1917. https://doi.org/10.1080/0142159X.2025.2571041 (2025).

    Article  PubMed  Google Scholar 

  32. Komasawa, N. & Yokohira, M. Generative artificial intelligence (AI) in medical education: a narrative review of the challenges and possibilities for future professionalism. Cureus 17 (6), e86316. https://doi.org/10.7759/cureus.86316 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  33. Malfatti, F. I. ChatGPT, education, and understanding. Soc. Epistemol. 39 (6), 652–666. https://doi.org/10.1080/02691728.2025.2449599 (2025).

    Article  Google Scholar 

  34. Jose, B. et al. Epistemic authority and generative AI in learning spaces: rethinking knowledge in the algorithmic age. Front. Educ. 10, 1647687. https://doi.org/10.3389/feduc.2025.1647687 (2025).

    Article  Google Scholar 

  35. Wu, J. Y., Lee, Y. H., Chai, C. S. & Tsai, C. C. Strengthening human epistemic agency in the symbiotic learning partnership with generative artificial intelligence. Educ. Res. 54 (6), 358–368. https://doi.org/10.3102/0013189X251333628 (2025).

    Article  Google Scholar 

  36. Henriksen, D., Mishra, P., Woo, L. & Oster, N. The education doctorate in the context of generative artificial intelligence: epistemic shifts and challenges to practical wisdom. Impact. Educ. J. Transform. Prof. Pract. 10 (1), 73–79. https://doi.org/10.5195/ie.2025.485 (2025).

    Article  Google Scholar 

  37. Ichikawa, T. et al. Generative artificial intelligence in medical education—policies and training at US osteopathic medical schools: descriptive cross-sectional survey. JMIR Med. Educ. 11, e58766. https://doi.org/10.2196/58766 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  38. Chadha, N., Popil, E., Gregory, J., Armstrong-Davies, L. & Justin, G. How do we teach generative artificial intelligence to medical educators? Pilot of a faculty development workshop using ChatGPT. Med. Teach. 47 (1), 160–162. https://doi.org/10.1080/0142159X.2024.2341806 (2025).

    Article  PubMed  Google Scholar 

  39. McCoy, L. et al. A training needs analysis for AI and generative AI in medical education: perspectives of faculty and students. J. Med. Educ. Curric. Dev. 12, 23821205251339226. https://doi.org/10.1177/23821205251339226 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  40. Abdulnour, R. E. E., Gin, B. & Boscardin, C. K. Educational strategies for clinical supervision of artificial intelligence use. N. Engl. J. Med. 393 (8), 786–797. https://doi.org/10.1056/NEJMra2503232 (2025).

    Article  PubMed  Google Scholar 

  41. Haltaufderheide, J. & Ranisch, R. The ethics of ChatGPT in medicine and healthcare: a systematic review on large language models. NPJ Digit. Med. 7, 183. https://doi.org/10.1038/s41746-024-01157-x (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  42. Gazquez-Garcia, J., Sánchez-Bocanegra, C. L. & Sevillano, J. L. AI in the health sector: systematic review of key skills for future health professionals. JMIR Med. Educ. 11, e58161. https://doi.org/10.2196/58161 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  43. Naamati-Schneider, L. & Alt, D. Developing AI literacy in medical and health professions education: exploring ChatGPT, ChatBots development, and simulation-based learning. Nurse Educ. Pract. 88, 104572. https://doi.org/10.1016/j.nepr.2025.104572 (2025).

    Article  PubMed  Google Scholar 

  44. Mehta, N., Mehta, S., Rubenstein, A. & Wood, S. K. Not replaced, but reinvented: AI education pathways to prepare future physicians to lead healthcare transformation. Perspect. Med. Educ. 14 (1), 849–859. https://doi.org/10.5334/pme.2233 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  45. Mirmoghtadaie, Z., Ahmady, S., Kohan, N. & Rakhshani, T. Explaining the concept and factors of professional functions in online learning system of medical sciences: a qualitative content analysis. Turk. Online J. Dist. Educ. 20 (4), 61–72. https://doi.org/10.17718/tojde.640510 (2019).

    Article  Google Scholar 

  46. Kashfi, S. M. et al. The effect of education of physical activity via social networks on the quality of life in menopausal women: a randomized controlled trial. J. Health Sci. Surveill. Syst. 9 (3), 149–154. https://doi.org/10.30476/jhsss.2020.87531.1114 (2021).

    Article  Google Scholar 

  47. Keshavarzi, M. H. et al. Investigating the mediating role of anxiety in academic burnout and academic performance of rehabilitation sciences students. J. Educ. Health Promot. 13, 293. https://doi.org/10.4103/jehp.jehp_862_23 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

来源:Nature 健康护理主题 · nature.com