跳到正文
原文
Nature 健康护理主题·· 15 天前AI 评分35

评估网络、大型语言模型和搜索引擎中健康信息质量的综述

Evaluating the quality of consumer health information across the web, large language models, and search engines: a review of criteria, instruments, and methods

AI 导读

该综述指出,尽管网络和大型语言模型生成的健康信息质量存在显著差异,但缺乏统一的质量定义。研究发现,商业和非专业来源提供的信息质量通常低于健康相关或专业来源。此外,大型语言模型在不同临床领域的表现不一,有时准确但有时不准确或不完整。这些发现强调了评估健康信息质量的多维度性和局限性。

正文

Introduction

The literature review is broadly recognized as a crucial step for assessing the knowledge produced within a specific field of study, pinpointing its limitations, and outlining future developments1,2,3. Searching for ‘health information quality’ in academic search engines such as Google Scholar yields numerous results, indicating growing research interest4. Notably, extensive studies assess the quality of health information using both automated and manual approaches5,6,7,8. These findings underscore the importance of the topic. However, no universally agreed-upon definition of health information quality (HIQ) exists. Instead, various studies use different definitions and evaluate them using a range of indicators and criteria9,10. The concern has grown considerably, as numerous studies have shown that the distribution of health information on the internet is often dominated by low-quality content11,12,13,14,15,16,17. A growing body of research has examined the quality of online health information, focusing on both traditional web-based sources and, more recently, health information generated by large language models.

Research on web-based health information consistently reveals significant variability in quality across different health topics, languages, and sources of information. Studies of online information on Alzheimer’s disease prevention, COVID-19, and periodontal disease have identified shortcomings in key quality indicators such as HONcode certification, DISCERN scores, and JAMA benchmark criteria6,8,12. These results also indicate that quality depends on the source, with commercial and non-professional providers generally offering lower-quality information than health-related or professional sources8. Overall, this evidence shows that just because health information is available online does not mean it meets established standards of accuracy, reliability, transparency, or completeness.

Recent reviews of health information generated by LLMs reveal more inconsistent patterns. In various clinical fields, LLMs have sometimes provided accurate and thorough responses, but at other times they have been inaccurate, incomplete, too complex, lacking transparency, or citing unreliable sources18,19,20,21,22. Observed performance varies depending on the clinical area, model version, and question difficulty. Studies utilizing expert evaluations, DISCERN, EQIP, guideline adherence, Global Quality Scores, and triage criteria show that a high score in one aspect does not guarantee overall satisfactory performance across all dimensions18,19,20,21,22. These findings highlight the multidimensional nature of LLM health information quality and the limitations of depending on a single assessment method.

In addition to variability in performance across domains, a growing body of research has documented persistent limitations in AI-generated health information. Several studies have identified factual inaccuracies, hallucinated or fabricated content, and unreliable or missing citations as recurring issues in model responses22,23,24,25. Other research has highlighted poor adherence to clinical guidelines and the presence of outdated or incomplete medical information26,27. Furthermore, multiple studies have reported that AI-generated responses often exhibit high reading complexity, excessive verbosity, and limited personalization, which may reduce accessibility for patients with lower health literacy28,29,30.

In more complex clinical scenarios, studies have also documented problems such as misclassification of clinical urgency, omission of critical safety information, and lack of contextual judgment, all of which may affect the reliability of the provided information20,31,32. Additionally, inconsistencies in reasoning and variation in response quality across languages and domains have been observed33,34,35.

Beyond these limitations, several studies have highlighted persistent transparency issues in AI-generated health information. In particular, the absence of verifiable citations, fabricated references, and unclear sourcing of medical claims have been widely reported, raising concerns about the trustworthiness of generated content19,22,36,37. In some cases, AI systems provided confident responses without indicating uncertainty, which may lead users to overestimate the information’s reliability. Additionally, studies have shown that models may struggle to interpret or apply clinical knowledge appropriately, leading to oversimplified explanations, incomplete reasoning, or unsupported medical advice38,39.

Moreover, evidence from recent studies suggests that these limitations are further strengthened in specialized and high-stakes medical contexts. For example, research has reported that AI systems may produce unsafe or inconsistent recommendations in areas such as oncology and surgical planning, including inappropriate treatment suggestions or a lack of specificity in clinical decision-making32,40. Other studies have identified deficiencies in diagnostic reasoning, including failure to recognize severe conditions or provide appropriate therapeutic guidance41,42. In addition, domain-specific evaluations have revealed moderate-to-low performance in certain specialties, with some studies reporting a substantial proportion of low-quality responses in emergency scenarios43. Taken together, these findings reinforce the need for more robust and standardized evaluation approaches that account not only for accuracy but also for safety, reliability, and contextual appropriateness in AI-generated health information.

In light of this, the literature reveals a lack of consistency in how health information quality is evaluated. Different studies apply varying, sometimes inconsistent, quality criteria, and no universally accepted set of dimensions exists for assessing health information quality9,10. Furthermore, evidence from studies examining health-related search results indicates that high search engine rankings do not necessarily correspond to high information quality. For instance, studies of information on HPV vaccination, erectile dysfunction, thyroid cancer, and neonatal jaundice have reported discrepancies between search ranking and quality assessments, with highly ranked content sometimes receiving lower quality scores or containing lower-quality information44,45,46,47. These findings suggest that information visibility and quality should be treated as distinct yet interconnected dimensions, and that health-related search engine ranking mechanisms should incorporate information quality criteria alongside conventional ranking factors17.

This review examines the quality of consumer health information from online sources and LLM-generated outputs. It also explores how search-engine rankings influence the visibility and accessibility of web-based health content. The main question addressed in this review is: How is the quality of consumer health information be benchmarked in both web-based and LLM-generated contexts? Furthermore, what is the relationship between search rankings and information quality? This review integrates existing criteria, assessment tools, conceptual frameworks, and evaluation methods to identify common dimensions of quality, assess their applicability across different sources, and highlight gaps in current evaluation approaches.

Definitions of concepts

This section clearly defines the main concepts used in the paper to ensure readers share a common understanding of the terminology throughout the study. Wilkinson48 states that scientists keep well-defined terms to think transparently about their research and communicate their results effectively. Additionally,49 highlights that defining terms or concepts improves the precision of scientific studies.

  • Health Information Quality: refers to how well information meets the needs and expectations of its users or consumers; high-quality information is characterized by accuracy, completeness, relevance, and reliability.

  • Quality criteria: Quality criteria are abstract, multidimensional rules that assess health information quality, with each dimension representing a different attribute. For instance, the author’s information and qualifications fall under the Authoritative dimension, which emphasizes the author’s expertise, credentials, and reputation.

  • Measurement tools: Measurement tools for health information consist of various quality dimensions used to evaluate specific aspects of health information quality. For example, the Discern tool offers guidelines to help consumers and health information providers assess the quality of treatment options.

  • Accuracy : The degree to which health information is accurate, medically valid, and aligned with evidence, guidelines, or expert opinion.

  • Reliability: The degree to which the information is reliably consistent, including response uniformity, lack of misleading or conflicting info, and sound clinical judgment.

  • Trustworthiness: A broader assessment of whether the information or source is reliable and trustworthy, considering factors such as authorship, references, transparency, disclosures, supporting evidence, and currency.

  • Credibility Refers to the perceived quality of online health information and the user’s willingness to rely on it.

Methodology

Study design

This study used a traditional narrative literature review approach, identifying relevant literature through searches of major academic databases and selecting it purposively based on its relevance to the review objectives. Unlike systematic reviews, this approach allows a more flexible, interpretive synthesis of prior work, enabling a comprehensive view of the diverse evaluation criteria, instruments, and methodological strategies reported in the literature.

Literature search and selection strategy

The literature search was conducted in late 2025 using major academic databases, including PubMed, ScienceDirect, Scopus, Google Scholar, SpringerLink, and Web of Science, to identify relevant studies on health information quality and evaluation methods. Some of the strings used in the search strategy included: "consumer health information", “health information quality", “online health information evaluation", “search engine optimization", “search-ranking algorithms",“quality assessment”, “health information ranking", “quality assessment tools", “large language models", “AI-generated health information", “ChatGPT evaluation", and “health information reliability".

Since this study was carried out as a traditional narrative review, relevant publications were found through iterative searches of the identified academic databases using terms aligned with the review’s goals. We evaluated publications for relevance to the review’s scope and included them if they provided insights into definitions, criteria, instruments, or evaluation methods used to assess consumer health information. Consequently, we did not systematically record the total number of records retrieved from each database or the number excluded at each stage. The selection process followed the review’s objectives and thematic focus rather than exhaustively screening every record. Special focus was given to peer-reviewed publications that met at least one of the following inclusion criteria:

  1. 1.

    Defined or conceptualized health information quality.

  2. 2.

    Proposed or applied criteria or dimensions for evaluating health information quality.

  3. 3.

    Utilized established assessment tools (e.g., JAMA, DISCERN, HON, QUEST).

  4. 4.

    Evaluated health information in web-based or AI-generated contexts.

Publications were excluded if they met any of the following criteria:

  1. 1.

    Were not peer-reviewed scholarly publications, such as websites, blogs, news articles, opinion pieces.

  2. 2.

    Did not address health information quality or its evaluation.

  3. 3.

    Focused exclusively on clinical or patient outcomes without assessing the quality of health information.

  4. 4.

    Did not provide sufficient methodological or evaluative information relevant to the objectives of this review.

  5. 5.

    Were duplicates or inaccessible in full text.

We also incorporated recent studies to capture emerging developments, particularly LLM-based approaches to health information quality evaluation. Following a comprehensive selection process depicted in Fig. 1, approximately  88 articles were incorporated into the narrative review regarding the Definition of Health Information Quality, Tools for Quality Evaluation, Frameworks for Assessing Quality in LLM-Generated Content, Conceptual Frameworks for Health Information Quality, Search Engine Optimization, and the Determinants of Quality in Online Health Information.

Fig. 1

Studies selection process

Data analysis and synthesis

The selected studies were analyzed qualitatively. The findings were organized into thematic categories, including:

  1. 1.

    Definitions of health information quality

  2. 2.

    Evaluation tools and criteria

  3. 3.

    Methods used to assess web-based and AI-generated health information

  4. 4.

    Conceptual Frameworks for HIQ

  5. 5.

    Search Engine Optimization and Online Health Information Quality

Within each category as relevant, studies were further examined to identify commonly used dimensions such as accuracy, reliability, readability, transparency, and completeness, as well as the types of evaluation approaches applied (e.g., expert-based assessments, Likert-scale evaluations, and automated metrics) as appropriate.

Findings, synthesis, and analysis

This section reviews the literature on defining health information quality, the key criteria used to evaluate it, and the instruments and methodological approaches developed to assess the quality of online consumer health information. It also examines the similarities and differences among these approaches in order to provide a clearer understanding of how researchers currently evaluate health information quality on the web and LLM-based systems.

The results reveal a layered framework for evaluating the quality of health information from both web-based sources and LLM-generated content as shown in Fig. 2. At the top level, it defines Health Information Quality (HIQ), setting the groundwork for further assessment. Existing tools and conceptual models for HIQ offer criteria and theoretical support for systematically examining health information across various sources. Building on these, the framework guides the evaluation and benchmarking of content based on key quality dimensions for both web-based and LLM-generated health information. It also considers end-user interaction, focusing on how patients and the general public access and use health information, including via digital platforms like mobile apps. Overall, the framework progresses from defining HIQ, establishing measurement bases, to assessing, benchmarking, and ultimately supporting end-user access to quality health information.

Fig. 2

Layered framework for assessing health information quality across web-based and LLM-generated health information.

Health information quality definition

Information quality is a multidimensional concept widely studied in business and manufacturing50. It is commonly defined as the fitness of data for use by its users51, indicating that data quality is inherently relative. In other words, information quality varies across domains, and the quality attributes used in one context may not fully apply in another51,52. Consequently, accurately measuring health information quality requires not only identifying relevant quality criteria but also evaluating them in the context in which the information is used53. In the literature, the most widely accepted perspective on information quality emphasizes assessing it from the user’s viewpoint52.

The authors of54 argue that data quality cannot be evaluated independently of the individuals who use the data, reflecting a perspective similar to that proposed by52. When applying the perspective of54 to health information quality, two main challenges arise. First, users face cognitive constraints, and second, limited health literacy can hinder their ability to accurately assess the quality of health information55,56. A recent study17 proposed an updated definition of information quality that builds upon the framework introduced by51. In this research, the quality of health information is character as information that corresponds with the user’s viewpoint and is assessed by healthcare professionals to guarantee it genuinely represents human values and fulfils healthcare requirements.

It is worth noting that currently available Information Quality (IQ) frameworks define health information quality as a multidimensional concept, as drawn from prior studies. One such dimension is Authorship, which includes the author’s credentials, affiliation, and other relevant details and is categorized under source-related dimensions. On the other hand, content-related dimensions, such as Attribution, ensure that all references and sources used to create the content are appropriately cited.

These different conceptualizations focus on various aspects of information quality. The fitness-for-use approach assesses quality by whether information fulfils the needs of its target users and the context of use. Conversely, the definition in17 stresses the importance of health information satisfying healthcare requirements, embodying human values, and being verified for accuracy by health experts. While both perspectives are complementary, they emphasize distinct aspects of quality, underscoring the context-dependent and multidimensional nature of health information quality

Overall, the literature does not offer a clear, universally accepted definition of health information quality. Although some studies have adapted definitions from other domains, they may not fully apply to the specific context of health information. Consequently, consensus is lacking to guide the development of precise dimensions and models for assessing health information quality. In the following sections, we review the tools used to evaluate the quality of health information and the frameworks proposed in the literature.

Quality evaluation tools

Numerous tools have been developed to evaluate the quality of online health information, with the most widely used being the Journal of the American Medical Association (JAMA) score57, the Health On the Net (HON) code58, and the Discern Criteria59. These tools set standards that a website or LLM response must meet to be considered high-quality. Their academic impact is notable: the JAMA score has been cited 2768 times, HON 572 times, and the Discern Criteria 3350 times. However, there is no available data on how many health information seekers have actually used these tools to assess online health information60. Additionally, these tools have been applied in both automatic evaluations61,62 and manual assessments7,63,64,65. The following sections discuss each tool in detail.

American medical association (JAMA) score

The Journal of the American Medical Association (JAMA) benchmarks assess the trustworthiness of health-related websites and include four key quality criteria. The first is Authorship, which requires clear identification of the author responsible for the health information, including their credentials, institutional affiliation, and other relevant background details. The second criterion, Attribution, requires references or sources to support the health information presented, such as medical advice, general health knowledge, or descriptions of treatments. The third criterion, Disclosure, requires transparency about funding sources, sponsorships, advertising, and any potential conflicts of interest that may influence the content. Finally, the Currency criterion indicates the date the health information was published or last updated, allowing users to determine whether it is current. Table 1 summarizes the components of the JAMA score.

Table 1 JAMA criteria

Full size table

Health on the net foundation (HON) code

The HONcode has long been a leading framework for evaluating the credibility of online health information. However, the Health On the Net Foundation ceased its website certification service in 202266. This change underscores the difficulty of sustaining certification-based methods for assessing online health content and emphasizes the need for robust, sustainable, and standardized evaluation frameworks. Nevertheless, the HONcode criteria remain in use in the literature to manually evaluate the trustworthiness of websites, chatbots, and teleconsultation services67,68,69. The HON Code aims to improve the quality of health information by introducing additional evaluation criteria beyond the trustworthiness principles of Authorship, Attribution, Disclosure, and Currency included in the JAMA score. In this sense, the HON Code provides a broader framework for assessing the quality of health information, extending beyond the credibility of the information source alone. The HON Code consists of eight principles. Historically, the Health On the Net Foundation manually reviewed compliance with these principles before granting certification to websites that met the required standards.

In addition to the JAMA-related criteria, health information providers must adhere to five further principles under the HON Code. The first principle, Complementarity, requires that online health information should support, rather than replace, the doctor-patient relationship. The second principle, Privacy, requires providers to protect users’ personal information. The third principle, Justifiability, obliges providers to support claims regarding the effectiveness or benefits of treatments with appropriate scientific evidence. The fourth principle, Transparency, requires clear communication channels between information providers and users, such as email addresses or telephone numbers. Finally, the Advertising Policy principle requires a clear separation between editorial content and advertising. Table 2 summarizes the components of the HON Code.

Table 2 HONcode criteria

Full size table

Discern Criteria

Discern criteria, such as the HON code, offer more options for assessing the quality of health information than the source’s trustworthiness; the DISCERN instrument (Criteria) provides guidelines for judging treatment choices by consumers and health information providers. Nevertheless, authors and producers can also use DISCERN to guide the design of health information. Most importantly, DISCERN was designed to help end users and health information producers evaluate written material on treatment options for different health problems. For the DISCERN tool, the review included all the quality criteria recently used62 and suggested by70.

In this study, regarding the DISCERN instrument, health information providers must meet six quality criteria for the information to be considered high quality. First, the treatment works criterion entails a detailed description of how each treatment works. Include details on how the treatment functions within the body to address the underlying condition or symptoms. Second, the treatment benefits criterion requires a description of the benefits or advantages of the chosen treatment. Third, the treatment risks criterion outlines the risks associated with each treatment in relation to the treatment outcome.

Fourth, the impact of treatment on the quality of life criterion offers a broader perspective on treatment choice in the patient’s life, extending beyond the treatment’s risks and benefits. It is crucial to assess how different treatment options affect daily life, including potential lifestyle changes and their impact on close relationships. Fifth, the no-treatment criterion involves describing the potential side effects of not using any treatment. This is particularly crucial in emergency cases. Finally, the Areas of Uncertainty criterion clarifies the ambiguous aspects of treatment effectiveness, such as uncertainty about which individuals will benefit from the treatment or experience adverse effects. For instance, uncertainty about who will benefit or suffer from the treatment option. The Table 3 summarises the DISCERN Instrument.

Table 3 DISCERN Instrument

Full size table

Quest tool

The authors, Julie and others71, aim to create a tool that undergoes validation and covers various health-related information. However, the tool includes only six dimensions of quality in health information; most are identical to those in older tools. For instance, Authorship is similar to that of the JAMA and HON guidelines, with the only variation being the measurement approach. The tool assigns three scores: 0 if no author information is provided, 1 if some author identification is provided, and 2 if complete author information is available. Conflict of interest and Tone were the only two dimensions that differed in name from the old tools. First, the conflict-of-interest quality criterion ensures that the health information provided is not affected by the funding source or advertising policy; it combines two dimensions from the old tools: financial disclosure and advertising policy. Second, the tone quality criterion ensures that authors of health articles balance their claims with cautious language and discuss limitations, as in the justifiability dimension. Nevertheless, this tool has been cited in many recent articles to evaluate health information from web sources and LLMs72,73,74,75,76,77. For more details on each quality criterion, refer to Table 4.

Table 4 QUEST Tool

Full size table

Comparison of evaluation criteria across health information quality assessment tools

The comparison in Table 5 demonstrates substantial variation in how the four assessment tools operationalize health information quality. The HON Code provides relatively broad coverage of source-related and transparency dimensions, including authorship, attribution, disclosure, currency, and complementarity, but gives limited attention to core content-quality dimensions such as accuracy and completeness. JAMA primarily focuses on structural indicators of trustworthiness, particularly authorship, attribution, disclosure, and currency. In contrast, DISCERN places greater emphasis on treatment-related information, including evidence support, risks, uncertainty, and the effect of treatment on quality of life. QUEST provides a more concise framework that includes authorship, attribution, conflict of interest, currency, complementarity, and tone. As discussed in the next section, Quality Assessment Frameworks for Evaluating LLM-Generated Content include accuracy, completeness, and safety as important dimensions for evaluating health information generated by LLMs, while traditional assessment tools do not consistently address them. These differences indicate that the instruments address overlapping but distinct aspects of health information quality; therefore, relying on a single tool may provide only a partial assessment, and complementary criteria from multiple instruments may be necessary for a more comprehensive evaluation.

Table 5 Comparison of the Operationalization of Quality Dimensions Across Health Information Quality Assessment Tools

Full size table

Quality assessment for evaluating LLM-generated content

Table 6 provides a structured overview of the quality dimensions and evaluation methods used in studies examining LLM-generated outputs. It captures both automated metrics (e.g., ROUGE, BLEU, F1-score) and human-centred evaluations (e.g., expert review and Likert-scale assessments). This variation reflects the ongoing challenge of establishing standardised evaluation frameworks for LLM systems and health information quality in general.

Table 6 Quality Criteria, Tools, and Type of Evaluation Methods in the Review

Full size table

Notably, the analysis indicates a predominance of non-standardised approaches, with many studies relying on expert-based, subjective assessments to evaluate dimensions such as accuracy, coherence, and clinical relevance, which may limit reproducibility and comparability across studies. In contrast, a smaller subset of studies uses standardized tools and validated instruments, including DISCERN, EQIP, guideline-based assessments, and readability indices, which provide more consistent, objective benchmarks. However, even these standardized methods are often supplemented by subjective evaluations, reflecting the inherent complexity of assessing LLM-generated content. Overall, this variation in approaches highlights the lack of a unified evaluation framework and underscores the need for more robust, standardized, and domain-specific approaches to improve the reliability and comparability of future LLM evaluations.

Among the reviewed studies, accuracy was the most frequently evaluated dimension, appearing in 22 out of 45 studies (48.9%). Readability was examined in 8 studies (17.8%), while coherence and reliability appeared in 5 studies each (11.1%). Medical adequacy and completeness were assessed in 4 studies (8.9% each). Less commonly, 3 studies (6.7%) evaluated dimensions such as comprehensibility, comprehensiveness, conciseness, and safety. Studies operationalization these dimensions differently, using methods such as expert judgment, Likert-scale ratings, guideline comparisons, validated instruments, readability indices, and automated metrics. This methodological diversity complicates direct comparisons and restricts the generalisability of findings across different clinical domains and LLM systems.

A further methodological challenge is the dominance of expert-based and subjective evaluations. While expert assessment is crucial for health information due to the need for clinical accuracy and safety, variability can arise from differences in evaluator expertise, sample size, rating scales, and evaluation criteria across studies. Although standardized tools and guideline-based assessments improve comparability, researchers often use them alongside subjective expert judgments rather than as standalone measures. Automated metrics such as ROUGE, BLEU, and F1-score can provide consistent measurements, but on their own, they do not confirm the clinical accuracy, safety, or appropriateness of health information.

The findings also show that traditional health information quality tools do not fully capture the unique features of health information generated by LLMs. DISCERN evaluates treatment-related aspects such as benefits, risks, and uncertainty and has been adapted for use in several LLM studies. JAMA provides criteria for authorship, attribution, disclosure, and currency. However, these tools were originally designed for traditional health content and do not sufficiently address LLM-specific issues such as hallucinations, citation accuracy, explainability, response consistency, uncertainty disclosure, and AI safety risks. Therefore, while these tools are useful as supplements, their use for LLM-generated content should not be seen as a complete assessment of the quality of AI-produced health information.

Overall, the evidence indicates that assessing health information produced by LLMs demands a multidimensional approach. This approach should integrate established health information quality standards with new dimensions specific to generative AI. While accuracy, completeness, readability, and clinical relevance remain essential, they must be supplemented with evaluations of citation validity, hallucinations, transparency about uncertainty, explainability, consistency, and safety. Therefore, rather than replacing traditional quality assessment tools, LLM evaluation should expand them to address the unique features and risks of generative AI.

Conceptual frameworks for HIQ

The rise of new technologies and the evolution of the web have revolutionized how people seek advice about their health conditions. As a result, evaluation tools such as JAMA and others mentioned earlier have emerged. However, these quality instruments fail to examine web page content or verify the scientific or medical validity of the information97.

The accuracy of medical information available on a webpage cannot be guaranteed, even when the authors possess medical credentials and the website complies with established quality standards. Previous studies have shown that adherence to quality standards for medical web pages does not necessarily correspond to the accuracy of the information presented97,98,99. Furthermore, these evaluation tools have limitations in applicability and assessment criteria. For instance, the DISCERN instrument omits several critical aspects when assessing online health information, including accuracy, completeness, disclosure, and readability. These elements are essential to ensure that health information is not only reliable but also transparent and accessible to consumers.

Consequently, this section discusses several attempts from various studies to develop a conceptual framework that addresses the shortcomings of previous tools. Research on information quality criteria and indicators falls into three primary categories: empirical approaches, theoretical approaches, and common sense and the experience of specialist doctors100,101. A brief description of the included studies is presented in Table 7.

Table 7 Conceptual Framework for HIQ

Full size table

Based on an empirical approach

This approach adopts an empirical methodology, using questionnaires, interviews, or other practical methods to define Information Quality (IQ) based on end-user input102. In their study, Wand and Wang102 distributed a questionnaire to 355 participants from business and administration backgrounds and classified information quality into four categories: Intrinsic IQ, Accessibility IQ, Contextual IQ, and Representational IQ. Recent studies have applied this approach to evaluate health information quality60,103. Al-Jefri et al.60 aimed to establish quality criteria for online health information. The researchers adopted a theoretical approach to identify potential criteria from the literature and complemented it with an empirical approach using a questionnaire to assess users’ perceptions. The study included 329 participants, all consumers of online health information with university-level education. Although only 26.5% of the participants had a medical background (e.g., medicine or pharmacology), the study identified five quality criteria, with scientific completeness ranked as the most important.

Donghua Tao et al. conducted another study103 that employed both qualitative and quantitative methods to examine quality criteria and their associated indicators. The researchers used Likert-scale questionnaires to allow end users to rate the importance of various information quality criteria. In addition, the researchers conducted semi-structured group interviews and applied an open coding procedure for qualitative data analysis. The study identified five criteria for evaluating information quality: completeness, understandability, relevance, depth, and accuracy.

Quality criteria based on theoretical approaches

This method, originally introduced by Wand and Wang102 and referred to as an ontological approach, focuses on theoretical understanding and empirically ranking the importance of quality criteria101. In their work, the authors applied this approach to identify existing quality criteria through a literature review and subsequently employed an empirical method to rank their relative importance. To develop a concise quality assessment tool for evaluating online health information, Robillard and colleagues71 conducted an extensive literature review and analyzed several studies57,106,107 using this approach. As a result, they developed the QUEST tool, which includes the following criteria: Authorship, Attribution, Conflict of Interest, Currency, Complementarity, and Tone. Further details about each quality criterion are presented in Table 4.

Anastasija Nikiforova104 proposed an object-driven approach for evaluating data quality. The proposed model consists of three main components: first, the data object whose quality is to be assessed; second, the data quality specification, which defines the conditions that must be satisfied for the data to be considered high quality; and third, the data quality verification process, which describes the procedures required to assess the quality of the data object.

This study does not adopt the concept of predefined quality dimensions. Instead, it enables end users to define their own quality requirements for the data objects they use, depending on the specific use case. Therefore, this approach can be considered user-driven.

In another study105, the authors applied hierarchical cluster analysis to classify distinct patterns based on website categories and the type of information provided. They also examined therapies and prevention methods. The findings indicated that analyzing treatment types improved trustworthiness measures and helped determine whether the information was supported by evidence-based medicine (EBM).

Based on common sense and the experience of specialist doctors

Many instruments have been developed to evaluate the quality of online health information based on common sense and the expertise of specialist doctors or expert panels in consumer health information. Among these, the most widely used tools include the Health on the Net Code (HON)58, the Journal of the American Medical Association (JAMA) benchmark criteria57, and the DISCERN instrument59.

A key limitation of many previous studies is their reliance primarily on the perceptions of general health information users who often lack medical backgrounds when assessing the importance of quality criteria for health information. This approach introduces three major limitations:

  1. 1.

    Public perceptions of the quality of online health information are often subjective and shaped by their information needs, personal characteristics, and demographic factors108,109.

  2. 2.

    General users of health information encounter two significant challenges when evaluating the quality of health claims. First, cognitive limitations affect their ability to properly assess health information55. Second, limited health literacy further restricts their capacity to critically evaluate such information56.

  3. 3.

    Most of the studies do not precisely determine whether the identified quality dimensions reflect genuine health specialists’ perspectives, since they are primarily derived from samples of general users.

Finally, the recent study17 identifies 18 quality criteria closely related to the overall quality of online health information, based on responses from 253 participating physicians and caregivers. The authors grouped these criteria into three categories: source quality, treatment quality, and content trustworthiness. Privacy and currency were the most important source-related criteria, while treatment risks and treatment benefits were the most important treatment-related factors. The findings also indicate that doctors and caregivers share similar perceptions of health information quality, with only minor differences by geographical region. Overall, the study provides a practical framework that can support the evaluation of online health information and the development of AI-based quality assessment systems. Furthermore, the study combines three approaches to identify and validate these criteria: an empirical approach, a theoretical approach, and common sense based on specialist doctors’ experience.

Comparative synthesis of conceptual frameworks

The reviewed conceptual frameworks vary significantly in how they define health information quality, despite differences in terminology, methodology, and context. Accuracy and completeness are the only dimensions that appear explicitly in more than one framework, each appearing in two out of seven (28.6%). However, the frameworks differ in both the scope and focus of the dimensions they consider. Some emphasize users’ perceptions of information quality, while others also include aspects related to the source, treatment, and content of the information. The framework introduced by17 offers extensive coverage, incorporating 18 criteria across three categories: source quality, treatment quality, and content trustworthiness. It combines empirical, theoretical, and professional perspectives. However, it focuses primarily on traditional health information and does not specifically address features associated with LLM-generated content.

LLM-generated health information requires extra evaluation criteria, such as identifying hallucinated or unsupported information, verifying citations, revealing uncertainty, ensuring explainability, maintaining response consistency, and ensuring safety. These features are not addressed across the reviewed conceptual frameworks. Consequently, while current frameworks provide a useful basis for evaluating LLMs, they need to be expanded to include AI-specific dimensions to properly assess the quality and safety of the health information they produce.

Search engine optimization and the quality of online health information: key factors

With the rapid expansion of the Internet and the increase of websites, finding relevant information has become increasingly complex. Search engines emerged to address this challenge by indexing websites and ranking them by relevance to user queries110. This section of the literature review examines the primary factors influencing webpage ranking and their relationship to health information quality.

In consumer health information, search engine ranking matters not because it directly indicates quality, but because it affects what health information users can see and access. Consequently, visibility and quality are related but separate aspects. A top-ranked webpage might be more frequently encountered but not necessarily offer better information. Understanding this difference matters because ranking systems can shape public exposure to health information, even when they don’t explicitly evaluate quality.

Search engines evaluate various characteristics of webpages, including title tags, content quality, and keyword density, among other technical and off-page factors. These factors contribute to the evaluation and ranking of webpages in search engine results pages (SERPs)110.

A significant amount of research has explored how technical, structural, and on-page optimization factors impact search engine ranking performance. This research highlights the importance of website performance, content quality, keyword optimization, and structural characteristics in determining search engine rankings110.

Technical performance indicators, such as website size and website loading time, have been highlighted in110,111,112. At the same time, on-page optimization has been associated with the use and density of keywords in title tags110,111,113,114 and heading tags, such H1110,111,112,113, keyword density110,113, and the inclusion of keywords in the URL and meta description110,113,114,115.

Some studies highlighted the importance of alt text for images116,117, and appropriately optimized title and description length111,113,118,119. Structural characteristics such as URL length113,118,119 are also noted as contributing factors.

From a site architecture perspective, internal linking was identified in111,114, while website structure appeared in113,114,118,120,121. Off-site measures received substantial attention, particularly the quality and quantity of backlinks, which a broad range of studies supported115,118,121,122. The role of social media support was likewise reinforced in118,121,122,123.

Usability-related elements, such as a custom 404 page124,125,126 and a Sitemap111, are reported as essential factors. In terms of trust and reliability, studies have showed the significance of domain age and name114,127 and of SSL certification128,129. Finally, user engagement metrics such as bounce rate130,131 and time on site132 have been discussed as website performance and evaluation metrics.

Despite the breadth of factors reported, a critical observation is that none of these studies explicitly examined the quality dimension of website content. Content was typically assessed using quantitative measures such as keyword frequency, density, or length, whereas dimensions such as credibility, authority, and trustworthiness were largely absent from operational definitions. Consequently, high-quality content that lacks optimization may rank lower than well-optimized but lower-quality content, underscoring the need to prioritize trusted sources in rankings133.

Notably, several empirical studies examining health-related search results have found that higher search engine rankings do not necessarily indicate higher-quality information. For instance,44 found that webpages critical of the HPV vaccine often ranked highly in Google results despite providing lower-quality content. This illustrates the risk that individuals may base crucial decisions, such as whether to vaccinate, on misleading information simply because it appears highly in search results134,135. Similarly,45 reported a negative correlation between Google ranking and DISCERN scores in searches on erectile dysfunction treatments. Correspondingly,46 observed the same findings in Arabic-language resources on thyroid cancer.

Further evidence comes from an evaluation of online resources for anterior cruciate ligament surgery, where none of the measures of readability, understandability, or actionability was significantly associated with Google rank136. Likewise, an assessment of information on neonatal jaundice showed that DISCERN and JAMA benchmark scores were not correlated with Google rankings on either English- or German-language websites47. Collectively, these findings indicate that search algorithms may privilege lower-quality health information, highlighting the importance of integrating content quality into ranking mechanisms. Table 8 shows that most SEO factors are technical or optimisation-focused, with only limited direct attention to health information quality aspects such as credibility, accuracy, and authority.

Table 8 SEO Factors and Their Relationship to Health Information Quality

Full size table

Discussion

This study offers an in-depth overview of how the quality of consumer health information is conceptualized and assessed in both conventional web-based resources and content produced by large language models (LLMs). It synthesizes findings from existing research to identify the most commonly used quality dimensions, evaluation instruments (e.g., JAMA, HON, DISCERN, QUEST), and assessment strategies. The study underscores the heterogeneity of current practices and reveals that no universally accepted framework for evaluating the quality of health information currently exists.

The findings align with prior research showing that health information quality is inherently multidimensional and context-dependent, with no single set of criteria capable of capturing all relevant aspects5,9,137,138. This reliance on varied tools and evaluation methods reflects the domain’s complexity. In particular, the increasing use of LLMs introduces new challenges, such as variability in response accuracy, lack of transparency, and the potential for hallucinated content, which traditional evaluation instruments do not fully address.

The results of this study align with earlier research indicating that the definition of Health Information Quality is not universal. Various studies use different definitions, which in turn influence how they are assessed5,9.

The results of this review indicate that quality evaluation tools have not evolved significantly since their introduction in the nineties, with only minor modifications over time. In particular, current frameworks for evaluating LLM-generated content continue to rely heavily on these traditional tools. However, these approaches often fail to account for the inherently complex internal representations of large language models139. Given the sophisticated and opaque nature of these models, relying solely on conventional evaluation methods may be insufficient for capturing their true quality and misleading behavior140,141,142.

Existing conceptual frameworks17,60,71,103,104,105 move beyond traditional evaluation tools by integrating multiple dimensions of health information quality. Nonetheless, many of these frameworks still face challenges, including limited generalizability, dependence on non-expert user judgments, and inadequate validation of content accuracy. These shortcomings underscore the need for more robust and adaptive methods, especially those tested on LLMs, to more accurately reflect the complexity of these models. Still, the study in17 represents a recent advancement and may be regarded as one of the most effective solutions currently available.

Drawing on a synthesis of conceptual frameworks and empirical studies, this review identifies recurring dimensions that may provide a foundation for evaluating consumer health information across different information environments. Accuracy and completeness are the most consistently represented dimensions across the reviewed conceptual frameworks, and accuracy is also the most frequently assessed dimension in empirical studies of LLM-generated health information. Other commonly examined dimensions include readability, coherence, reliability, and medical adequacy. However, these core dimensions alone may not be sufficient for evaluating LLM-generated health information. Traditional web-based evaluation approaches commonly consider factors such as authorship, attribution, currency, transparency, and treatment-related information, whereas LLM-generated content introduces additional concerns, including hallucinations, citation validity, uncertainty disclosure, explainability, response consistency, and safety. Therefore, the findings support distinguishing between core health information quality dimensions that are applicable across information environments and context-specific dimensions that arise from the characteristics of the information source or generation process.

These review results also indicate an ongoing problem: information is primarily determined by technical and structural SEO factors rather than by content quality. The review emphasizes that information visibility and quality are related but distinct. While search engine rankings influence what users are likely to find, the evidence shows that higher rankings don’t always mean better information quality45,47,136. Therefore, improving access to health information requires addressing both quality and visibility, rather than assuming the most visible information is always trustworthy. This finding has implications for both health-information providers and the design of health-information retrieval and ranking systems. These systems might improve by adding quality signals alongside traditional relevance and technical factors.

Previous research has consistently highlighted that search engine rankings are influenced by elements such as keyword frequency, meta-tag optimization, heading structure, URL length, backlinks, and site architecture110,113,115,118,122. For example, studies on health-related topics such as anterior cruciate ligament surgery136, neonatal jaundice47, erectile dysfunction treatments45, and Arabic thyroid cancer resources46 found that high-quality content did not necessarily correspond to higher search rankings. Similarly, webpages critical of the HPV vaccine often ranked highly despite low content quality44,134,135, further illustrating the limitations of SEO-driven visibility in ensuring access to reliable health information.

In contrast, a recent study explicitly integrates content type, provider category, and quality score into the evaluation process17, moving beyond traditional SEO metrics that focus primarily on technical and structural factors such as keyword frequency, backlinks, URL structure, and website architecture110,115,118,122. Whereas prior research measured performance using quantitative optimization indicators alone, this study’s approach provides a multidimensional framework that evaluates both the credibility and reliability of health information alongside its visibility. This enables a more nuanced understanding of which sources provide high-quality, trustworthy content and highlights the mismatch between algorithmic ranking and content integrity, a gap largely unaddressed in previous SEO-focused studies. By moving beyond oversimplified binary models (credible vs. non-credible)61,62,97,143, this study better reflects the complexity of real-world health information assessment and offers a systematic framework for identifying authoritative sources.

Based on the combined synthesis of conceptual frameworks and empirical evaluation studies, this narrative review integrates these sources to propose a structured approach for assessing consumer health information across different informational settings. It begins by identifying health information sources, including large language models like ChatGPT, Claude AI, and Google Gemini, alongside traditional Internet-based health data from search engines. The quality evaluation component focuses on six key dimensions repeatedly highlighted in the literature: accuracy, completeness, coherence, reliability, medical adequacy, and readability. These dimensions provide a common basis for assessing the quality of health information, regardless of its source or delivery environment.

Since LLM-generated data carries risks not fully covered by the main dimensions, the framework adds specific LLM-related factors such as hallucination, citation accuracy, explainability, response consistency, and safety. These factors enhance the core quality criteria by focusing on how LLMs produce and share information. After assessing quality, health information can be accessed by various users, such as patients, healthcare providers, and the general public, for purposes like diagnosis and treatment information, among others. The framework complements clinical judgment rather than replacing it; it shows how quality-validated consumer health information from web-based and LLM-based sources can be identified and incorporated into health information searches. For a visual illustration of the proposed conceptual model and its elements, please see Fig. 3.

Fig. 3

Integrating Quality Criteria into Health Information Search for Improved Decision-Making

Implications and future research

The study has several important implications:

  1. 1.

    Standardization of core dimensions: Future research should establish standardised definitions and practical measures for essential health information quality dimensions, such as accuracy, completeness, readability, and reliability.

  2. 2.

    Development of LLM-specific criteria: Existing instruments need to be assessed and modified to target features unique to generative AI, such as hallucinations, citation accuracy, uncertainty transparency, explainability, response consistency, and safety.

  3. 3.

    Integration of human and automated evaluation: Future research should investigate how expert assessment, validated instruments, guideline-based evaluation, and automated metrics can be combined to minimize subjectivity and improve reproducibility.

  4. 4.

    Longitudinal evaluation of LLMs: Since LLMs and their foundational models evolve quickly, it is important to continually test evaluation frameworks to check if their quality and safety features stay consistent across different model versions.

Limitations

This study has several limitations. As a narrative review, it depends on the availability, selection, and interpretation of existing studies, which can introduce biases and may limit the generalizability of the findings. While it provides a comparative overview of current definitions, evaluation tools, and conceptual frameworks and presents a conceptual model based on the reviewed evidence, the model has not been empirically tested. Its dimensions and structure require further assessment across diverse health information settings, user groups, clinical fields, and LLM systems. Additionally, because LLM technologies are rapidly advancing, evaluation methods and the identified limitations may evolve, potentially affecting the relevance of some findings over time. Future research could build on this narrative review by conducting a systematic literature review using established guidelines such as PRISMA.

Conclusion

This review shows that consumer health information quality is multidimensional, with substantial variation in definitions, criteria, and evaluation methods. Existing tools, including JAMA, HON, DISCERN, and QUEST, address different aspects of quality and therefore may provide only partial assessments when used individually. The review also highlights that LLM-generated health information requires additional evaluation of hallucinations, citation accuracy, uncertainty, explainability, consistency, and safety. Furthermore, search-engine visibility does not necessarily indicate information quality, highlighting the need to treat quality and visibility as distinct yet interconnected dimensions. Overall, future evaluation frameworks should integrate established quality dimensions with LLM-specific criteria and combine standardised, automated, and expert-based approaches.

Data Availability

No datasets were generated or analysed during the current study.

Code availability

Not Applicable

References

  1. Grant, M. J. & Booth, A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health information & libraries journal26, 91–108 https://doi.org/10.1111/j.1471-1842.2009.00848.x (2009).

    Article  Google Scholar 

  2. Tranfield, D., Denyer, D. & Smart, P. Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British journal of management14, 207–222 https://doi.org/10.1111/1467-8551.00375 (2003).

    Article  Google Scholar 

  3. Pahlevan Sharif, S., Mura, P. & Wijesinghe, S. N. Systematic reviews in asia: Introducing the “prisma” protocol to tourism and hospitality scholars. In Quantitative Tourism Research in Asia: Current Status and Future Directions, 13–33, https://doi.org/10.1007/978-981-13-2463-5_2 (Springer, 2019).

  4. Baqraf, Y. & Keikhosrokiani, P. The prediction of health information quality perception using machine learning and deep learning techniques. In 2023 11th International Conference on Information and Communication Technology (ICoICT), 104–109, https://doi.org/10.1109/ICoICT58202.2023.10262623 (IEEE, Melaka, Malaysia, 2023).

  5. Baqraf, Y. K. A., Keikhosrokiani, P. & Al-Rawashdeh, M. Evaluating online health information quality using machine learning and deep learning: A systematic literature review. Digital Health9, 1–33 https://doi.org/10.1177/20552076231212296 (2023).

    Article  Google Scholar 

  6. Robillard, J. M. & Feng, T. L. Health advice in a digital world: quality and content of online information about the prevention of alzheimer’s disease. Journal of Alzheimer’s Disease55, 219–229 https://doi.org/10.3233/JAD-160650 (2017).

    Article  PubMed  Google Scholar 

  7. Halboub, E., Al-Ak’hali, M. S., Al-Mekhlafi, H. M. & Alhajj, M. N. Quality and readability of web-based arabic health information on covid-19: an infodemiological study. BMC public health21, 151 https://doi.org/10.1186/s12889-021-10218-9 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Al-Ak’hali, M. S., Fageeh, H. N., Halboub, E., Alhajj, M. N. & Ariffin, Z. Quality and readability of web-based arabic health information on periodontal disease. BMC medical informatics and decision making21, 41 https://doi.org/10.1186/s12911-021-01413-0 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Zhang, Y., Sun, Y. & Xie, B. Quality of health information for consumers on the web: a systematic review of indicators, criteria, tools, and evaluation results. Journal of the Association for Information Science and Technology66, 2071–2084 https://doi.org/10.1002/asi.23311 (2015).

    Article  Google Scholar 

  10. Daraz, L. et al. Can patients trust online health information? a meta-narrative systematic review addressing the quality of health information on the internet. Journal of general internal medicine34, 1884–1891 https://doi.org/10.1007/s11606-019-05109-0 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Jamal, M. S. et al. Evaluation of the quality of information online for arthroscopic bankart repair. The Annals of The Royal College of Surgeons of England105, 394–399 https://doi.org/10.1308/rcsann.2022.0024 (2023).

    Article  Google Scholar 

  12. Batar, N., Kermen, S., Sevdin, S., Yıldız, N. & Güçlü, D. Assessment of the quality and reliability of information on nutrition after bariatric surgery on youtube. Obesity Surgery30, 4905–4910 https://doi.org/10.1007/s11695-020-05015-z (2020).

    Article  PubMed  Google Scholar 

  13. Corcelles, R., Daigle, C. R., Talamas, H. R., Brethauer, S. A. & Schauer, P. R. Assessment of the quality of internet information on sleeve gastrectomy. Surgery for Obesity and Related Diseases11, 539–544 https://doi.org/10.1016/j.soard.2014.08.014 (2015).

    Article  PubMed  Google Scholar 

  14. Koball, A. M. et al. Content and accuracy of nutrition-related posts in bariatric surgery facebook support groups. Surgery for Obesity and Related Diseases14, 1897–1902 https://doi.org/10.1016/j.soard.2018.08.017 (2018).

    Article  PubMed  Google Scholar 

  15. Palma, A. F. et al. Quality of information for women seeking breast augmentation in the internet. Journal of Plastic Surgery and Hand Surgery50, 262–271 https://doi.org/10.3109/2000656X.2016.1154469 (2016).

    Article  PubMed  Google Scholar 

  16. Kloosterboer, A., Yannuzzi, N. A., Patel, N. A., Kuriyan, A. E. & Sridhar, J. Assessment of the quality, content, and readability of freely available online information for patients regarding diabetic retinopathy. JAMA Ophthalmology137, 1240–1245 https://doi.org/10.1001/jamaophthalmol.2019.3116 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  17. Baqraf, Y., Keikhosrokiani, P., Cheah, Y.-N., Alomarji, A. & Baqraf, F. Artificial intelligence for automating the establishment of an arabic benchmark dataset for enhancing health information quality assessment. Discover Artificial Intelligence5, 411 https://doi.org/10.1007/s44163-025-00679-x (2025).

    Article  Google Scholar 

  18. King, R. C. et al. Appropriateness of ChatGPT in answering heart failure related questions. Heart, Lung and Circulation33, 1314–1318 https://doi.org/10.1016/j.hlc.2024.03.005 (2024).

    Article  PubMed  Google Scholar 

  19. Jedrzejczak, W. W. et al. ChatGPT for tinnitus information and support: Response accuracy and retest after three and six months. Brain Sciences14, 465 https://doi.org/10.3390/brainsci14050465 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Psilopatis, I. et al. The role of artificial intelligence in gynecologic and obstetric emergencies. European Journal of Obstetrics & Gynecology and Reproductive Biology306, 94–100 https://doi.org/10.1016/j.ejogrb.2025.01.007 (2025).

    Article  Google Scholar 

  21. Walker, H. L. et al. Reliability of medical information provided by ChatGPT: Assessment against clinical guidelines and patient information quality instrument. Journal of Medical Internet Research25, e47479 https://doi.org/10.2196/47479 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  22. Whiles, B. B., Bird, V. G., Canales, B. K., DiBianco, J. M. & Terry, R. S. Caution! AI bot has entered the patient chat: ChatGPT has limitations in providing accurate urologic healthcare advice. Urology180, 278–284 https://doi.org/10.1016/j.urology.2023.07.010 (2023).

    Article  PubMed  Google Scholar 

  23. Buhr, C. R. et al. Assessing unknown potential—quality and limitations of different large language models in the field of otorhinolaryngology. Acta Oto-Laryngologica144, 237–242 https://doi.org/10.1080/00016489.2024.2352843 (2024).

    Article  PubMed  Google Scholar 

  24. Köroğlu, E. Y. et al. Evaluation of the impact of chatgpt support on acromegaly management and patient education. Endocrine87, 1141–1149 https://doi.org/10.1007/s12020-024-04086-7 (2025).

    Article  CAS  PubMed  Google Scholar 

  25. Tang, L. et al. Evaluating large language models on medical evidence summarization. npj Digital Medicine6, 158 https://doi.org/10.1038/s41746-023-00896-7 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  26. Hanna, D. R. et al. Bing chat for kidney stone management questions based on the aua guidelines: a comparison of chatbot conversation style modes. World Journal of Urology43, 151 https://doi.org/10.1007/s00345-025-05533-4 (2025).

    Article  PubMed  Google Scholar 

  27. Yeo, Y. H. et al. Assessing the performance of chatgpt in answering questions regarding cirrhosis and hepatocellular carcinoma. Clinical and Molecular Hepatology29, 721–732 https://doi.org/10.3350/cmh.2023.0089 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  28. Høj, S. et al. Evaluating the scientific reliability of chatgpt as a source of information on asthma. Journal of Allergy and Clinical Immunology: Global3, 100330 https://doi.org/10.1016/j.jacig.2024.100330 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Cao, Y. et al. Performance of popular large language models in glaucoma patient education: A randomized controlled study. Advances in Ophthalmology Practice and Research5, 88–94 https://doi.org/10.1016/j.aopr.2024.12.002 (2025).

    Article  PubMed  Google Scholar 

  30. Fahy, S., Oehme, S., Milinkovic, D., Jung, T. & Bartek, B. Assessment of quality and readability of information provided by chatgpt in relation to anterior cruciate ligament injury. Journal of Personalized Medicine14, 104 https://doi.org/10.3390/jpm14010104 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Mak, G. et al. Utility of chatgpt and large language models in enhancing patient understanding of urological conditions. Société Internationale d’Urologie Journal5, 843–851 https://doi.org/10.3390/siuj5060064 (2024).

    Article  Google Scholar 

  32. Buhr, C. R. et al. Assessment of decision-making with locally run and web-based large language models versus human board recommendations in otorhinolaryngology, head and neck surgery. European Archives of Oto-Rhino-Laryngology282, 1593–1607 https://doi.org/10.1007/s00405-024-09153-3 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  33. Faisal, S. et al. Evaluating the comprehension and accuracy of chatgpt’s responses to diabetes-related questions in urdu compared to english. Digital Health10, 20552076241289730 https://doi.org/10.1177/20552076241289730 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  34. Samaan, J. S. et al. Chatgpt’s ability to comprehend and answer cirrhosis related questions in arabic. Arab Journal of Gastroenterology24, 145–148 https://doi.org/10.1016/j.ajg.2023.08.001 (2023).

    Article  PubMed  Google Scholar 

  35. Alharbi, A. O. et al. Evaluating large language models on health-related claims across arabic dialects. In Proceedings of the 1st Workshop on NLP for Languages Using Arabic Script (AbjadNLP 2025), 95–103 (Association for Computational Linguistics, Abu Dhabi, UAE, 2025).

  36. Ghanem, Y. K. et al. Dr. google to dr. chatgpt: assessing the content and quality of artificial intelligence-generated medical information on appendicitis. Surgical Endoscopy38, 2887–2893 https://doi.org/10.1007/s00464-024-10739-5 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  37. Zhou, M., Pan, Y., Zhang, Y., Song, X. & Zhou, Y. Evaluating ai-generated patient education materials for spinal surgeries: Comparative analysis of readability and discern quality across chatgpt and deepseek models. International Journal of Medical Informatics198, 105871 https://doi.org/10.1016/j.ijmedinf.2025.105871 (2025).

    Article  PubMed  Google Scholar 

  38. Hu, H., Yan, J., Zhang, X., Jiao, Z. & Tang, B. Overview of chip2023 shared task 4: Chip-yier medical large language model evaluation. In Health Information Processing. Evaluation Track Papers (CHIP 2023), vol. 2080 of Communications in Computer and Information Science, 127–134, https://doi.org/10.1007/978-981-97-1717-0_11 (Springer, 2023).

  39. Szczesniewski, J. J. et al. Chatgpt and most frequent urological diseases: analysing the quality of information and potential risks for patients. World Journal of Urology41, 3149–3153 https://doi.org/10.1007/s00345-023-04563-0 (2023).

    Article  PubMed  Google Scholar 

  40. Griewing, S. et al. Evolution of publicly available large language models for complex decision-making in breast cancer care. Archives of Gynecology and Obstetrics310, 537–550 https://doi.org/10.1007/s00404-024-07565-4 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  41. Duan, P. & Su, X. Medical knowledge q&a evaluation based on chatgpt ensemble learning. In Health Information Processing. Evaluation Track Papers (CHIP 2023), vol. 2080 of Communications in Computer and Information Science, 148–155, https://doi.org/10.1007/978-981-97-1717-0_13 (Springer, 2023).

  42. Lan, L. et al. Establishing a novel score system and using it to assess and compare the quality of chatgpt-4 consultation with physician consultation for obstetrics and gynecology: A pilot study. International Journal of Gynecology & Obstetrics168, 1251–1257 https://doi.org/10.1002/ijgo.15934 (2025).

    Article  Google Scholar 

  43. Cocci, A. et al. Quality of information and appropriateness of chatgpt outputs for urology patients. Prostate Cancer and Prostatic Diseases27, 159–160 https://doi.org/10.1038/s41391-023-00754-3 (2024).

    Article  PubMed  Google Scholar 

  44. Fu, L. Y., Zook, K., Spoehr-Labutta, Z., Hu, P. & Joseph, J. G. Search engine ranking, quality, and content of web pages that are critical versus noncritical of human papillomavirus vaccine. Journal of Adolescent Health58, 33–39 https://doi.org/10.1016/j.jadohealth.2015.09.016 (2016).

    Article  Google Scholar 

  45. Pattenden, T. A., Raleigh, R. A., Pattenden, E. R. & Thangasamy, I. A. Quality and readability of online patient information on treatment for erectile dysfunction. BJUI Compass2, 412–418 https://doi.org/10.1002/bco2.87 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  46. Alseheimi, A. M., Alseheimi, S. M. & Alhysoni, K. A. Quality assessment of online health information on thyroid cancer in the arabic language: A cross-sectional study. Cureus16, e76526 https://doi.org/10.7759/cureus.76526 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  47. Baumgartner, M. K. et al. Diving into the digital landscape: assessing the quality of online information on neonatal jaundice for parents. Children11, 877 https://doi.org/10.3390/children11070877 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  48. Wilkinson, A. M. The scientist’s handbook for writing papers and dissertations. (No Title) (1991).

  49. Firestone, W. A. Meaning in method: The rhetoric of quantitative and qualitative research. Educational researcher16, 16–21 https://doi.org/10.2307/1174685 (1987).

    Article  Google Scholar 

  50. Illari, P. & Floridi, L. Information quality, data and philosophy. In The philosophy of information quality, 5–23, https://doi.org/10.1007/978-3-319-07121-3_1 (Springer, 2014).

  51. Tayi, G. K. & Ballou, D. P. Examining data quality. Communications of the ACM41, 54–57 https://doi.org/10.1145/269012.269021 (1998).

    Article  Google Scholar 

  52. Knight, S.-A. & Burn, J. Developing a framework for assessing information quality on the world wide web. Informing Science: The International Journal of an Emerging Transdiscipline8, 159–172 https://doi.org/10.28945/493 (2005).

    Article  Google Scholar 

  53. Katerattanakul, P. & Siau, K. Measuring information quality of web sites: Development of an instrument. In Proceedings of the 20th International Conference on Information Systems (ICIS 1999), 279–285 (Charlotte, NC, 1999).

  54. Wang, R. Y. & Strong, D. M. Beyond accuracy: What data quality means to data consumers. Journal of Management Information Systems12, 5–33 https://doi.org/10.1080/07421222.1996.11518099 (1996).

    Article  Google Scholar 

  55. Pan, X., Lin, Y. & He, C. A review of cognitive models in human reliability analysis. Quality and Reliability Engineering International33, 1299–1316 https://doi.org/10.1002/qre.2111 (2017).

    Article  Google Scholar 

  56. Sørensen, K. et al. Health literacy in europe: comparative results of the european health literacy survey (hls-eu). European Journal of Public Health25, 1053–1058 https://doi.org/10.1093/eurpub/ckv043 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  57. Silberg, W. M., Lundberg, G. D. & Musacchio, R. A. Assessing, controlling, and assuring the quality of medical information on the internet: Caveant lector et viewor—let the reader and viewer beware. JAMA277, 1244–1245 https://doi.org/10.1001/jama.1997.03540390074039 (1997).

    Article  CAS  PubMed  Google Scholar 

  58. Boyer, C., Selby, M., Scherrer, J.-R. & Appel, R. D. The Health On the Net code of conduct for medical and health websites. Computers in Biology and Medicine28, 603–610 https://doi.org/10.1016/s0010-4825(98)00037-7 (1998).

    Article  CAS  PubMed  Google Scholar 

  59. Charnock, D., Shepperd, S., Needham, G. & Gann, R. Discern: an instrument for judging the quality of written consumer health information on treatment choices. Journal of Epidemiology & Community Health53, 105–111 https://doi.org/10.1136/jech.53.2.105 (1999).

    Article  CAS  Google Scholar 

  60. Al-Jefri, M., Evans, R., Uchyigit, G. & Ghezzi, P. What is health information quality? ethical dimension and perception by users. Frontiers in Medicine5, 260 https://doi.org/10.3389/fmed.2018.00260 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  61. Alasmari, A., Alhothali, A. & Allinjawi, A. Hybrid machine learning approach for arabic medical web page credibility assessment. Health Informatics Journal28, 14604582211070998 https://doi.org/10.1177/14604582211070998 (2022).

    Article  PubMed  Google Scholar 

  62. Kinkead, L., Allam, A. & Krauthammer, M. Autodiscern: rating the quality of online health information with hierarchical encoder attention-based neural networks. BMC Medical Informatics and Decision Making20, 104 https://doi.org/10.1186/s12911-020-01131-z (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  63. Alakhali, M. S. Quality assessment of information on oral cancer provided at arabic speaking websites. Asian Pacific Journal of Cancer Prevention: APJCP21, 961–966 https://doi.org/10.31557/APJCP.2020.21.4.961 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  64. Alhajj, M. N. et al. Quality and readability of web-based arabic health information on denture hygiene: an infodemiology study. Journal of Contemporary Dental Practice21, 956–960 https://doi.org/10.5005/jp-journals-10024-2918 (2020).

    Article  PubMed  Google Scholar 

  65. Gagliardi, A. & Jadad, A. R. Examination of instruments used to rate quality of health information on the internet: chronicle of a voyage with an unclear destination. BMJ324, 569–573 https://doi.org/10.1136/bmj.324.7337.569 (2002).

    Article  PubMed  PubMed Central  Google Scholar 

  66. Bayani, A., Ayotte, A. & Nikiema, J. N. Automated credibility assessment of web-based health information considering health on the net foundation code of conduct (honcode): model development and validation study. JMIR Formative Research7, e52995 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  67. Nikiema, J. N. Assessing the quality of direct-to-consumer teleconsultation services in canada. Studies in Health Technology and Informatics (2022).

  68. Rayess, N., Li, A. S., Do, D. V. & Rahimy, E. Assessment of online sites reliability, accountability, readability, accessibility, and translation for intravitreal injections. Ophthalmology Retina4, 1188–1195 (2020).

    Article  PubMed  Google Scholar 

  69. Wang, F. et al. Evaluating the quality of health-related wechat public accounts: cross-sectional study. JMIR mHealth and uHealth8, e14826 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  70. Khazaal, Y. et al. Brief discern, six questions for the evaluation of evidence-based content of health-related websites. Patient Education and Counseling77, 33–37 https://doi.org/10.1016/j.pec.2009.02.016 (2009).

    Article  PubMed  Google Scholar 

  71. Robillard, J. M., Jun, J. H., Lai, J.-A. & Feng, T. L. The quest for quality online health information: validation of a short quantitative tool. BMC Medical Informatics and Decision Making18, 87 https://doi.org/10.1186/s12911-018-0668-9 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  72. Sparks, C. A. et al. Chatgpt is moderately accurate in providing a general overview of orthopaedic conditions. JBJS Open Access9, e23.00129 https://doi.org/10.2106/JBJS.OA.23.00129 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  73. Yau, J. Y.-S. et al. Accuracy of prospective assessments of 4 large language model chatbot responses to patient questions about emergency care: experimental comparative study. Journal of Medical Internet Research26, e60291 https://doi.org/10.2196/60291 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  74. Malak, A. & Şahin, M. F. How useful are current chatbots regarding urology patient information? comparison of the ten most popular chatbots’ responses about female urinary incontinence. Journal of Medical Systems48, 102 https://doi.org/10.1007/s10916-024-02125-4 (2024).

    Article  PubMed  Google Scholar 

  75. Mitsutake, S. et al. ehealth literacy and web-based health information–seeking behaviors on covid-19 in japan: internet-based mixed methods study. Journal of Medical Internet Research26, e57842 https://doi.org/10.2196/57842 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  76. Hu, R.-h et al. Quality and accuracy of gastric cancer-related videos in social media videos platforms. BMC Public Health22, 2025 https://doi.org/10.1186/s12889-022-14417-w (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  77. Robillard, J. M., Alhothali, A., Varma, S. & Hoey, J. Intelligent and affectively aligned evaluation of online health information for older adults. In Joint Workshop on Health Intelligence (W3PHIAI), AAAI-17 Workshops, 15–18 (AAAI Press, 2017).

  78. Chang, L.-C. et al. Evaluation of the quality and readability of chatgpt responses to frequently asked questions about myopia in traditional chinese language. Digital Health10, 20552076241277021 https://doi.org/10.1177/20552076241277021 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  79. Buhr, C. R. et al. Chatgpt versus consultants: blinded evaluation on answering otorhinolaryngology case–based questions. JMIR Medical Education9, e49183 https://doi.org/10.2196/49183 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  80. Fahy, S., Oehme, S., Milinkovic, D. D. & Bartek, B. Enhancing patient education on the role of tibial osteotomy in the management of knee osteoarthritis using a customized chatgpt: a readability and quality assessment. Frontiers in Digital Health6, 1480381 https://doi.org/10.3389/fdgth.2024.1480381 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  81. McMahon, H. V. & McMahon, B. D. Automating untruths: Chatgpt, self-managed medication abortion, and the threat of misinformation in a post-roe world. Frontiers in Digital Health6, 1287186 https://doi.org/10.3389/fdgth.2024.1287186 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  82. Høj, S., Thomsen, S. F., Meteran, H., Sigsgaard, T. & Meteran, H. Artificial intelligence and allergic rhinitis: does chatgpt increase or impair the knowledge? Journal of Public Health46, 123–126 https://doi.org/10.1093/pubmed/fdad219 (2024).

    Article  PubMed  Google Scholar 

  83. Eckrich, J. et al. Urology consultants versus large language models: Potentials and hazards for medical advice in urology. BJUI Compass5, 552–558 https://doi.org/10.1002/bco2.359 (2024).

    Article  Google Scholar 

  84. Ong, Q. C. et al. Advancing health coaching: A comparative study of large language model and health coaches. Artificial Intelligence in Medicine157, 103004 https://doi.org/10.1016/j.artmed.2024.103004 (2024).

    Article  PubMed  Google Scholar 

  85. Olariu, M.-E., Vuza, D. & Iftene, A. Microsoft copilot, google gemini, openai chatgpt and anthropic claude: Assisting in cardiology diagnosing – a case study. In Advances in ICT Research in the Balkans (BCI 2024), vol.2391 of Communications in Computer and Information Science, 256–271, https://doi.org/10.1007/978-3-031-84093-7_18 (Springer, 2024).

  86. Thapa, S. et al. Did you tell a deadly lie? evaluating large language models for health misinformation identification. In Web Information Systems Engineering – WISE 2024, vol. 15440 of Lecture Notes in Computer Science, 391–405, https://doi.org/10.1007/978-981-96-0576-7_29 (Springer, 2024).

  87. Rotem, R. et al. The future of patient education: a study on ai-driven responses to urinary incontinence inquiries. International Journal of Gynecology & Obstetrics167, 1004–1009 https://doi.org/10.1002/ijgo.15751 (2024).

    Article  Google Scholar 

  88. Dastani, M., Mardaneh, J. & Rostamian, M. Large language models’ capabilities in responding to tuberculosis medical questions: testing chatgpt, gemini, and copilot. Scientific Reports15, 18004 https://doi.org/10.1038/s41598-025-03074-9 (2025).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  89. Yilmaz, B. E., Gokkurt Yilmaz, B. N. & Ozbey, F. Artificial intelligence performance in answering multiple-choice oral pathology questions: a comparative analysis. BMC Oral Health25, 573 https://doi.org/10.1186/s12903-025-05926-2 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  90. Hurley, E. T. et al. Evaluation high-quality of information from chatgpt (artificial intelligence—large language model) artificial intelligence on shoulder stabilization surgery. Arthroscopy: The Journal of Arthroscopic & Related Surgery40, 726–731.e6 https://doi.org/10.1016/j.arthro.2023.07.048 (2024).

    Article  Google Scholar 

  91. Fahy, S., Niemann, M., Böhm, P., Winkler, T. & Oehme, S. Assessment of the quality and readability of information provided by chatgpt in relation to the use of platelet-rich plasma therapy for osteoarthritis. Journal of Personalized Medicine14, 495 https://doi.org/10.3390/jpm14050495 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  92. Sallam, M. et al. Chinese generative ai models (deepseek and qwen) rival chatgpt-4 in ophthalmology queries with excellent performance in arabic and english. Narra J5, e2371 https://doi.org/10.52225/narra.v5i1.2371 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  93. Ayers, J. W. et al. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Internal Medicine183, 589–596 https://doi.org/10.1001/jamainternmed.2023.1838 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  94. Samaan, J. S. et al. Assessing the accuracy of responses by the language model chatgpt to questions regarding bariatric surgery. Obesity Surgery33, 1790–1796 https://doi.org/10.1007/s11695-023-06603-5 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  95. Biswas, S., Logan, N. S., Davies, L. N., Sheppard, A. L. & Wolffsohn, J. S. Assessing the utility of chatgpt as an artificial intelligence-based large language model for information to answer questions on myopia. Ophthalmic and Physiological Optics43, 1562–1570 https://doi.org/10.1111/opo.13207 (2023).

    Article  PubMed  Google Scholar 

  96. Grilo, A., Marques, C., Corte-Real, M., Carolino, E. & Caetano, M. Assessing the quality and reliability of chatgpt’s responses to radiotherapy-related patient queries: Comparative study with gpt-3.5 and gpt-4. JMIR Cancer11, e63677 https://doi.org/10.2196/63677 (2025).

    Article  PubMed  PubMed Central  Google Scholar 

  97. Meppelink, C. S. et al. Reliable or not? an automated classification of webpages about early childhood vaccination using supervised machine learning. Patient Education and Counseling104, 1460–1466 https://doi.org/10.1016/j.pec.2020.11.013 (2021).

    Article  PubMed  Google Scholar 

  98. Frické, M., Fallis, D., Jones, M. & Luszko, G. M. Consumer health information on the internet about carpal tunnel syndrome: indicators of accuracy. The American Journal of Medicine118, 168–174 https://doi.org/10.1016/j.amjmed.2004.04.032 (2005).

    Article  PubMed  Google Scholar 

  99. Bernstam, E. V. et al. Commonly cited website quality criteria are not effective at identifying inaccurate online information about breast cancer. Cancer112, 1206–1213 https://doi.org/10.1002/cncr.23308 (2008).

    Article  PubMed  Google Scholar 

  100. Baqraf, Y. & Keikhosrokiani, P. Health information quality assessment using artificial intelligence: quality dimensions from healthcare professionals’ perspective. In Advances in Intelligent Computing Techniques and Applications (IRICT 2023), vol. 210 of Lecture Notes on Data Engineering and Communications Technologies, 1–14, https://doi.org/10.1007/978-3-031-59711-4_1 (Springer, Cham,2023).

  101. Maguire, H. Book review: Data quality: concepts, methodologies and techniques by c. batini and m. scannapieco. International Journal of Information Quality1, 444–450 https://doi.org/10.1504/IJIQ.2007.016717 (2007).

    Article  Google Scholar 

  102. Wand, Y. & Wang, R. Y. Anchoring data quality dimensions in ontological foundations. Communications of the ACM39, 86–95 https://doi.org/10.1145/240455.240479 (1996).

    Article  Google Scholar 

  103. Tao, D., LeRouge, C., Smith, K. J. & De Leo, G. Defining information quality into health websites: a conceptual framework of health website information quality for educated young adults. JMIR Human Factors4, e25 https://doi.org/10.2196/humanfactors.6455 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  104. Nikiforova, A. Definition and evaluation of data quality: user-oriented data object-driven approach to data quality assessment. Baltic Journal of Modern Computing8, 391–432 https://doi.org/10.22364/bjmc.2020.8.3.02 (2020).

    Article  Google Scholar 

  105. Yaqub, M. & Ghezzi, P. Adding dimensions to the analysis of the quality of health information of websites returned by google: cluster analysis identifies patterns of websites according to their classification and the type of intervention described. Frontiers in Public Health3, 204 https://doi.org/10.3389/fpubh.2015.00204 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  106. Chumber, S., Huber, J. & Ghezzi, P. A methodology to analyze the quality of health information on the internet: the example of diabetic neuropathy. The Diabetes Educator41, 95–105 https://doi.org/10.1177/0145721714560772 (2015).

    Article  PubMed  Google Scholar 

  107. Sandvik, H. Health information and interaction on the internet: a survey of female urinary incontinence. BMJ319, 29–32 https://doi.org/10.1136/bmj.319.7201.29 (1999).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  108. Diviani, N., Van den Putte, B., Meppelink, C. S. & van Weert, J. C. Exploring the role of health literacy in the evaluation of online health information: insights from a mixed-methods study. Patient Education and Counseling99, 1017–1025 https://doi.org/10.1016/j.pec.2016.01.007 (2016).

    Article  PubMed  Google Scholar 

  109. Song, S., Zhao, Y., Song, X. & Zhu, Q. The role of health literacy on credibility judgment of online health misinformation. In 2019 IEEE International Conference on Healthcare Informatics (ICHI), 1–3, https://doi.org/10.1109/ICHI.2019.8904844 (IEEE, 2019).

  110. Ziakis, C., Vlachopoulou, M., Kyrkoudis, T. & Karagkiozidou, M. Important factors for improving google search rank. Future Internet11, 32 https://doi.org/10.3390/fi11020032 (2019).

    Article  Google Scholar 

  111. Zilincan, J. Search engine optimization. In CBU International Conference Proceedings, vol. 3, 506–510, https://doi.org/10.12955/cbup.v3.645 (Central Bohemia University, 2015).

  112. Gregurec, I. & Grd, P. Search engine optimization (seo): Website analysis of selected faculties in croatia. In Proceedings of the Central European Conference on Information and Intelligent Systems (CECIIS), 211–218 (Faculty of Organization and Informatics Varaždin, 2012).

  113. Krrabaj, S., Baxhaku, F. & Sadrijaj, D. Investigating search engine optimization techniques for effective ranking: A case study of an educational site. In 2017 6th Mediterranean Conference on Embedded Computing (MECO), 1–4, https://doi.org/10.1109/MECO.2017.7977137 (IEEE, 2017).

  114. Chandra, A., Suaib, M. & Beg, R. Low cost page quality factors to detect web spam (2014). https://arxiv.org/abs/1410.2085.

  115. Evans, M. P. Analysing google rankings through search engine optimization data. Internet Research17, 21–37 https://doi.org/10.1108/10662240710730470 (2007).

    Article  Google Scholar 

  116. Zhou, H., Qin, S., Liu, J. & Chen, J. Study on website search engine optimization. In 2012 International Conference on Computer Science and Service System (CSSS), 930–933, https://doi.org/10.1109/CSSS.2012.236 (IEEE, 2012).

  117. Wilson, R. F. & Pettijohn, J. B. Search engine optimisation: A primer on keyword strategies, https://doi.org/10.1057/palgrave.dddmp.4340563 (2006).

  118. Yalçın, N. & Köse, U. What is search engine optimization: Seo? Procedia - Social and Behavioral Sciences9, 487–493 https://doi.org/10.1016/j.sbspro.2010.12.185 (2010).

    Article  Google Scholar 

  119. Patil, S. P., Pawar, B. V. & Patil, A. S. Search engine optimization: A study. Research Journal of Computer and Information Technology Sciences1, 10–13 (2013).

    Google Scholar 

  120. Wang, F., Li, Y. & Zhang, Y. An empirical study on the search engine optimization technique and its outcomes. In 2011 2nd International Conference on Artificial Intelligence, Management Science and Electronic Commerce (AIMSEC), 2767–2770, https://doi.org/10.1109/AIMSEC.2011.6011361 (IEEE, 2011).

  121. Gupta, S., Rakesh, N., Thakral, A. & Chaudhary, D. K. Search engine optimization: Success factors. In 2016 Fourth International Conference on Parallel, Distributed and Grid Computing (PDGC), 17–21, https://doi.org/10.1109/PDGC.2016.7913146 (IEEE, 2016).

  122. Killoran, J. B. How to use search engine optimization techniques to increase website visibility. IEEE Transactions on Professional Communication56, 50–66 https://doi.org/10.1109/TPC.2012.2237255 (2013).

    Article  Google Scholar 

  123. Gudivada, V. N., Rao, D. & Paris, J. Understanding search-engine optimization. Computer48, 43–52 https://doi.org/10.1109/MC.2015.297 (2015).

    Article  Google Scholar 

  124. Garg, K., Alam, S., Ayala, D., Weigle, M. C. & Nelson, M. L. Not here, go there: Analyzing redirection patterns on the web. In Proceedings of the 17th ACM Web Science Conference 2025, 249–260, https://doi.org/10.1145/3717867.3717925 (Association for Computing Machinery, 2025).

  125. Bokingkito Jr, P. et al. Are we inclusive? accessibility challenges in philippine e-government websites. Informatics12, 41 https://doi.org/10.3390/informatics12020041 (2025).

    Article  Google Scholar 

  126. Kim, E., Yoo, J. & Kim, S. Understanding the effects of message context and level of interaction on 404 error page design. Archives of Design Research35, 133–147 https://doi.org/10.15187/adr.2022.11.35.4.133 (2022).

    Article  Google Scholar 

  127. Portier, W. K., Li, Y. & Kouassi, B. A. Feature selection using machine learning techniques based on search engine parameters. In Proceedings of the 2020 3rd International Conference on Signal Processing and Machine Learning, 28–34, https://doi.org/10.1145/3432291.3432308 (Association for Computing Machinery, 2020).

  128. Alkinoon, M. et al. Understanding the security and performance of the web presence of hospitals: A measurement study. In 2023 32nd International Conference on Computer Communications and Networks (ICCCN), 1–10, https://doi.org/10.1109/ICCCN58024.2023.10230186 (IEEE, 2023).

  129. Azevedo, A. C., Scheid, E. J., Franco, M. F. & Granville, L. Z. Assessing ssl/tls certificate centralization: Implications for digital sovereignty, https://doi.org/10.48550/arXiv.2504.16897 (2025).

  130. Jansen, B. J., Jung, S.-g & Salminen, J. Measuring user interactions with websites: A comparison of two industry standard analytics approaches using data of 86 websites. PLOS ONE17, e0268212 https://doi.org/10.1371/journal.pone.0268212 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  131. Mittal, M. & Dhari, S. V. A regression model for analysis of bounce rate using web analytics. International Journal of Innovative Technology and Exploring Engineering8, 646–649 https://doi.org/10.35940/ijitee.I1103.0789S19 (2019).

    Article  Google Scholar 

  132. Thakur, A., Sangal, A. L. & Bindra, H. Quantitative measurement and comparison of effects of various search engine optimization parameters on alexa traffic rank. International Journal of Computer Applications26, 15–23 https://doi.org/10.5120/3100-4257 (2011).

    Article  Google Scholar 

  133. Schultheiß, S., Häußler, H. & Lewandowski, D. Does search engine optimization come along with high-quality content? a comparison between optimized and non-optimized health-related web pages. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval, 123–134, https://doi.org/10.1145/3498366.3505811 (Association for Computing Machinery, 2022).

  134. Ghezzi, P. et al. Online information of vaccines: information quality, not only privacy, is an ethical responsibility of search engines. Frontiers in Medicine7, 400 https://doi.org/10.3389/fmed.2020.00400 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  135. Johnson, N. F. et al. The online competition between pro-and anti-vaccination views. Nature582, 230–233 https://doi.org/10.1038/s41586-020-2281-1 (2020).

    Article  CAS  PubMed  Google Scholar 

  136. Gao, B. et al. Can patients read, understand, and act on online resources for anterior cruciate ligament surgery? Orthopaedic Journal of Sports Medicine10, 23259671221089977 https://doi.org/10.1177/23259671221089977 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  137. Upadhyay, R., Pasi, G. & Viviani, M. Health misinformation detection in web content: a structural-, content-based, and context-aware approach based on web2vec. In Proceedings of the Conference on Information Technology for Social Good, 19–24, https://doi.org/10.1145/3462203.3475898 (Association for Computing Machinery, 2021).

  138. Upadhyay, R., Pasi, G. & Viviani, M. Vec4cred: a model for health misinformation detection in web pages. Multimedia Tools and Applications82, 5271–5290 https://doi.org/10.1007/s11042-022-13368-z (2023).

    Article  PubMed  Google Scholar 

  139. Bereska, L. & Gavves, E. Mechanistic interpretability for AI safety—a review. Transactions on Machine Learning Research (2024). URL: https://openreview.net/forum?id=ePUVetPKu6.

  140. Meinke, A. et al. Frontier models are capable of in-context scheming. arXiv preprint arXiv:2412.04984https://doi.org/10.48550/arXiv.2412.04984 (2024).

  141. Hagendorff, T. Deception abilities emerged in large language models. Proceedings of the National Academy of Sciences121, e2317967121 https://doi.org/10.1073/pnas.2317967121 (2024).

    Article  MathSciNet  CAS  Google Scholar 

  142. Hubinger, E. et al. Sleeper agents: Training deceptive llms that persist through safety training. arXiv preprint arXiv:2401.05566https://doi.org/10.48550/arXiv.2401.05566 (2024).

  143. Feng, J., Zou, L., Ye, O. & Han, J. Web2vec: Phishing webpage detection method based on multidimensional features driven by deep learning. IEEE Access8, 221214–221224 https://doi.org/10.1109/ACCESS.2020.3043188 (2020).

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank the School of Computer Sciences at Universiti Sains Malaysia (USM) for its invaluable support throughout this work.

Funding

The author(s) received no financial support for this article’s research, writing, or publication.

Author information

Authors and Affiliations

  1. School of Computer Sciences, Universiti Sains Malaysia, 11800, USM, Penang, Malaysia

    Yousef Baqraf & Yu-N Cheah

  2. Faculty of Information Technology and Electrical Engineering and Faculty of Medicine, University of Oulu, PL 8000, 90014, Oulun, yliopisto, Finland

    Pantea Keikhosrokiani

  3. Faculty of Computer Science & Engineering, Al-Ahgaff University, 50512, Mukalla, Hadhramout, Yemen

    Yousef Baqraf

  4. Department of Management and Marketing, Faculty of Business and Economics, Universiti Malaya, Kuala Lumpur, Malaysia, 50603

    Ahmed H. Alsharif

  5. Faculty of Economic, Business, and Management Sciences, Department of Business Sciences, University of Blida 2 Lounici Ali, El, Affroun, Algeria

    Sofiane Laradi

Authors

  1. Yousef Baqraf
  2. Pantea Keikhosrokiani
  3. Yu-N Cheah
  4. Ahmed H. Alsharif
  5. Sofiane Laradi

Contributions

Y.B.: Developed the review topic, designed the conceptual framework and study objectives, conducted the literature search, analysed and synthesised the evidence, interpreted the findings, and drafted the initial manuscript. P.K. ( Lead Author ): Supervised the study and guided the development of the review framework, methodology, literature review, and interpretation of findings. She critically revised the manuscript, provided important intellectual input throughout the study, and reviewed and approved the final version. Yu-N C. (Supervisor): Supervised the study and provided methodological and conceptual guidance throughout the research process. He helped interpret the findings, critically revised the manuscript for key intellectual content, and reviewed and approved the final version. A.H.A.: Contributed to the literature synthesis, provided critical feedback on the discussion and conclusions, and assisted with manuscript revision. S.L.: Contributed to data presentation and assisted with manuscript formatting and proofreading.

Author biographies

Yousef Baqraf is a PhD researcher at the School of Computer Sciences, UniversitiSains Malaysia, Malaysia, and a faculty member in the Faculty of Computer Scienceand Engineering at Al-Ahgaff University, Yemen. His research interests includehealth information quality, artificial intelligence, machine learning, health informatics, and online health information evaluation. His academic profile is available at: https://orcid.org/0000-0002-8741-4622.

Pantea Keikhosrokiani is a Docent (Adjunct Professor) and researcher at the Faculty of Information Technology and Electrical Engineering and the Faculty of Medicine, University of Oulu, Finland. Her research focuses on AI-driven softwaresystems, digital health, quality assurance, human–AI collaboration, medical informatics, and data-driven decision-making systems. Her academic profile is available at: https://orcid.org/0000-0003-4705-2732.

Yu-N Cheah is an Associate Professor at the School of Computer Sciences, UniversitiSains Malaysia, Malaysia. His research interests include artificial intelligence, knowledge management, semantic technologies, text analytics, health informatics, and intelligent information systems. His academic profile is available at: https://orcid.org/0000-0002-5644-9427.

Ahmed H. Alsharif is a researcher and academic affiliated with the Graduate School of Business, Universiti Sains Malaysia, Malaysia. His research interestsinclude consumer behaviour, digital technologies, marketing analytics, businessintelligence, and technology adoption. His academic profile is available at: https://orcid.org/0000-0002-1364-3545.

Sofiane Laradi is an academic and researcher in the Department of Business Sciences, Faculty of Economic, Business and Management Sciences, University of Blida2 Lounici Ali, Algeria. His research interests include management sciences, businessanalytics, digital transformation, and organizational performance. His academicprofile is available at: https://orcid.org/0000-0002-1345-9078.

Corresponding authors

Correspondence to Yousef Baqraf, Pantea Keikhosrokiani or Yu-N Cheah.

Ethics declarations

Ethics Approval

Not Applicable

Consent to participate

Not Applicable

Competing interests

The authors declare no competing interests.

Additional information

Publisher Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Baqraf, Y., Keikhosrokiani, P., Cheah, YN. et al. Evaluating the quality of consumer health information across the web, large language models, and search engines: a review of criteria, instruments, and methods. Sci. Rev. 1, 25 (2026). https://doi.org/10.1038/s44573-026-00025-x

Download citation

  • Received: 08 June 2026

  • Accepted: 28 September 2026

  • Published: 06 November 2026

  • Version of record: 06 November 2026

  • DOI: https://doi.org/10.1038/s44573-026-00025-x

Keywords

来源:Nature 健康护理主题 · nature.com