Abstract
The integration of artificial intelligence (AI) into primary care triage systems presents significant opportunities for enhancing healthcare efficiency, yet raises critical questions regarding how clinicians evaluate the reliability of AI-generated recommendations. This dissertation synthesises current evidence on the factors influencing clinician judgement of AI triage reliability, drawing upon a comprehensive literature review of 50 peer-reviewed papers. Findings demonstrate that clinicians primarily assess AI reliability by benchmarking outputs against their own clinical expertise and established protocols, with agreement rates exceeding 80% in non-complex cases. Trust is substantially influenced by system explainability, safety-conscious design principles, and seamless workflow integration. However, significant barriers persist, including confirmation bias whereby clinicians preferentially accept recommendations aligning with pre-existing judgements, contextual data limitations, and unresolved legal accountability concerns. The evidence reveals that human interrater variability establishes an inherent ceiling on achievable algorithmic concordance, complicating reliability assessments. This review concludes that whilst validated AI systems demonstrate promising accuracy, sustained clinician oversight remains essential. Future research should prioritise long-term outcome studies, bias mitigation strategies, and robust legal frameworks governing clinician-AI collaborative decision-making.
Introduction
The application of artificial intelligence to clinical decision-making represents one of the most significant transformations in contemporary healthcare delivery. Primary care triage, which serves as the critical gateway determining patient access to appropriate medical resources, has emerged as a particularly promising domain for AI implementation. Triage decisions fundamentally influence patient outcomes, resource allocation, and healthcare system efficiency, rendering the accuracy and reliability of such decisions paramount to effective healthcare provision (Gottliebsen and Petersson, 2020).
The expansion of AI-enabled triage tools in primary care settings has accelerated substantially over recent years, driven by increasing demand pressures on healthcare systems, technological advances in machine learning algorithms, and the growing digitisation of patient interactions. These systems range from symptom checkers accessible via smartphone applications to sophisticated clinical decision support tools integrated within electronic health record systems. The National Health Service in England has actively explored AI implementation across multiple care pathways, recognising the potential for improved efficiency whilst acknowledging the necessity of rigorous evaluation (NHS England, 2019).
Despite the proliferation of AI triage tools, fundamental questions persist regarding how clinicians evaluate the reliability of AI-generated recommendations in practice. This evaluation process is inherently complex, as clinicians must reconcile algorithmic outputs with their professional expertise, contextual patient knowledge, and established clinical protocols. Understanding this evaluation process carries substantial implications for AI system design, implementation strategies, and ultimately patient safety outcomes.
The academic significance of this inquiry extends beyond immediate practical applications. It intersects with broader scholarly debates concerning human-machine collaboration, the nature of clinical expertise, and the epistemological foundations of medical decision-making. Socially, the question addresses growing public interest in AI governance and transparency within healthcare contexts. Practically, understanding how clinicians judge AI reliability informs training programmes, system development, and regulatory frameworks essential for responsible AI deployment (Bragazzi and Garbarino, 2023).
This dissertation provides a comprehensive synthesis of current evidence examining how clinicians judge the reliability of AI-generated triage decisions in primary care settings. By systematically reviewing empirical studies and critically analysing findings, this work contributes to the emerging evidence base necessary for informed policy development and clinical practice guidance.
Aim and objectives
The primary aim of this dissertation is to synthesise and critically evaluate current evidence regarding how clinicians judge the reliability of AI-generated triage decisions within primary care contexts.
To achieve this aim, the following specific objectives have been established:
1. To examine the degree of agreement between clinician assessments and AI-generated triage recommendations as documented in peer-reviewed literature.
2. To identify and analyse the principal factors influencing clinician trust in AI triage systems, including system characteristics and cognitive factors.
3. To evaluate the barriers and limitations affecting clinician acceptance of AI-generated triage recommendations in real-world practice settings.
4. To critically assess the implications of current evidence for AI system design, clinical implementation, and future research directions.
5. To identify significant gaps in the existing literature and propose priorities for future investigation.
Methodology
This dissertation employs a systematic literature synthesis methodology to comprehensively examine the research question. A comprehensive literature search was conducted across multiple academic databases, including Semantic Scholar, PubMed, and related repositories, utilising the Consensus research platform which aggregates over 170 million research papers.
The search strategy incorporated multiple targeted queries designed to capture diverse perspectives on clinician judgement of AI triage reliability. Specific search terms included combinations of: “artificial intelligence triage primary care,” “clinician trust AI decision support,” “machine learning triage accuracy,” “AI symptom checker evaluation,” and “clinical decision support reliability.” Eight distinct search groups were employed to ensure comprehensive coverage of the research domain.
The initial search identified 1,036 potentially relevant papers. A four-phase screening process was subsequently implemented to refine the selection. During the screening phase, 537 papers were assessed for preliminary relevance based on title and abstract review. The eligibility phase involved detailed examination of 480 papers meeting initial criteria, with assessment of methodological quality, relevance to the research question, and publication in peer-reviewed outlets. This process yielded 50 papers meeting all inclusion criteria for detailed analysis.
Inclusion criteria specified that papers must: address AI or machine learning applications in clinical triage contexts; examine clinician perspectives, behaviours, or judgements regarding AI systems; present empirical findings or systematic analyses; and be published in peer-reviewed journals or equivalent academic outlets. Exclusion criteria eliminated papers focusing exclusively on technical algorithm development without clinical evaluation, those addressing non-primary care settings without transferable insights, and those lacking methodological rigour.
Data extraction followed a structured protocol capturing study design, sample characteristics, key findings regarding agreement rates, trust factors, and barriers. Extracted data were synthesised thematically, with particular attention to convergent and divergent findings across studies. The evidence synthesis approach enabled identification of patterns, gaps, and areas of scholarly consensus.
This methodological approach aligns with established frameworks for conducting systematic reviews and literature syntheses in healthcare research (Arksey and O’Malley, 2005). Whilst acknowledging limitations inherent to literature synthesis—including potential publication bias and heterogeneity in study designs—this methodology provides a rigorous foundation for addressing the stated research objectives.
Literature review
### Agreement between clinician and artificial intelligence triage decisions
The empirical literature consistently demonstrates substantial agreement between AI-enabled triage tools and clinician urgency assessments, particularly for non-complex presentations. A seminal UK study examining primary care same-day appointment triage found 84% categorical concordance between an AI tool and general practitioner ratings, achieving a Cohen’s kappa coefficient of 0.69, which indicates substantial agreement (Altalib et al., 2025). Critically, this study identified no significant under-triage events, suggesting that the AI system operated with appropriate safety margins.
Large-scale analyses of virtual care encounters provide further supporting evidence. Research examining AI diagnostic assistance in virtual primary care settings demonstrates that providers select AI-suggested diagnoses in over 80% of cases (Zeltzer et al., 2023). This high acceptance rate suggests that clinicians frequently perceive AI outputs as aligned with their clinical reasoning processes. Comparative analyses have additionally revealed that consensus-based evaluations often rate AI recommendations as equal or superior to those generated by individual physicians working independently (Delshad, Dontaraju and Chengat, 2021; Razzaki et al., 2018; Baker et al., 2020).
However, agreement rates demonstrate meaningful variation across clinical presentations and system types. Studies examining ChatGPT and similar large language models in emergency triage contexts report more variable performance. Paslı et al. (2024) found that whilst ChatGPT demonstrated reasonable precision in emergency department triage, significant limitations emerged for complex or atypical presentations. Similarly, Zaboli et al. (2025) concluded that current AI systems remain “far from surpassing human expertise” in triage tasks requiring nuanced clinical judgement.
The heterogeneity in reported agreement rates partly reflects differences in study populations, AI system sophistication, and methodological approaches. Studies employing standardised clinical vignettes tend to report higher agreement than those examining real-world encounters with complete patient complexity. This pattern suggests that AI systems may perform optimally within well-defined parameters whilst demonstrating limitations when confronted with the full complexity of clinical practice (Levine et al., 2024).
### Factors influencing clinician trust in artificial intelligence triage systems
Clinician trust in AI-generated triage recommendations emerges from a complex interplay of system characteristics, cognitive factors, and contextual variables. The literature identifies several primary determinants of trust that merit detailed examination.
Alignment with clinical expertise represents perhaps the most significant factor influencing trust. Clinicians consistently demonstrate increased trust when AI outputs accord with their initial judgements or established clinical protocols (Altalib et al., 2025; Steerling et al., 2025). This finding aligns with broader psychological literature on confirmation bias, whereby individuals preferentially accept information consistent with pre-existing beliefs. Bashkirova and Krpan (2024) provide direct evidence of this phenomenon in AI-assisted clinical contexts, demonstrating that psychologists exhibit substantially higher trust and recommendation acceptance when AI triage suggestions are congruent with their expert judgements.
System explainability constitutes another critical trust determinant. Transparent reasoning and clear presentation of decision logic enhance perceived reliability by enabling clinicians to evaluate the basis for recommendations (Steerling et al., 2025; Bragazzi and Garbarino, 2023). This finding resonates with the broader “explainable AI” movement in healthcare informatics, which emphasises that algorithmic transparency is essential for responsible clinical deployment. Clinicians appear more willing to accept AI recommendations when they can comprehend and critique the underlying reasoning process.
Safety-conscious design principles significantly influence clinician perceptions. Systems designed to err conservatively—favouring over-triage rather than under-triage—receive more favourable evaluations from clinicians concerned about patient safety implications (Altalib et al., 2025; Delshad, Dontaraju and Chengat, 2021). This preference reflects the asymmetric consequences of triage errors: under-triage potentially results in delayed care for seriously ill patients, whereas over-triage primarily impacts resource utilisation. The clinical preference for conservative systems suggests that risk aversion substantially shapes reliability judgements.
Workflow integration emerges as a practical factor mediating trust and adoption. Seamless integration with electronic health record systems and minimal disruption to existing clinical processes support higher acceptance rates (Altalib et al., 2025; Ilicki, 2022). Conversely, systems requiring additional steps, duplicate data entry, or departure from established workflows face greater resistance regardless of their technical accuracy.
### Barriers and limitations affecting clinician acceptance
Despite promising accuracy metrics across multiple studies, several persistent barriers constrain clinician acceptance of AI triage recommendations in practice settings.
Contextual blind spots represent a fundamental limitation of current AI systems. The inability to access complete patient records, longitudinal health histories, or nuanced contextual information limits reliability in complex cases (Altalib et al., 2025; Ilicki, 2022). Clinicians possess accumulated knowledge about individual patients—including psychosocial circumstances, previous presentations, and communication styles—that current AI systems cannot readily incorporate. This contextual knowledge frequently proves decisive in triage decisions, particularly when presentations are ambiguous.
Confirmation bias operates as both a facilitator and barrier to appropriate AI utilisation. Whilst alignment with clinician judgement increases acceptance, this tendency may lead to inappropriate rejection of discordant recommendations that could improve decision quality. Bashkirova and Krpan (2024) demonstrate that clinicians exhibit heightened scepticism toward AI advice disagreeing with their initial assessments, potentially missing opportunities for beneficial decision modification. This cognitive pattern creates a paradox whereby AI systems may be most valuable when least trusted.
Variability among human raters establishes an inherent ceiling on achievable concordance between AI systems and clinical standards. Entezarjou et al. (2020) demonstrate that even expert clinician panels exhibit substantial disagreement when classifying cases as urgent versus non-urgent. This finding carries profound implications: if human experts cannot achieve consensus, the appropriate benchmark against which to evaluate AI performance becomes unclear. Perfect algorithmic alignment with any single clinician’s judgement may not correlate with optimal patient outcomes.
### Real-world implementation challenges
Studies examining AI triage implementation in authentic clinical environments identify challenges extending beyond accuracy considerations. Digital literacy gaps among patient populations affect satisfaction with AI-mediated triage processes, potentially introducing inequities in access to appropriate care (Altalib et al., 2025). Workflow misalignment creates additional burden for clinicians who must navigate between AI recommendations and established procedures (Ilicki, 2022).
Legal and ethical concerns regarding accountability constitute particularly significant barriers. Uncertainty persists regarding liability allocation when adverse outcomes follow algorithmic recommendations. Clinicians express concern about professional responsibility implications when choosing to accept or override AI advice (Gottliebsen and Petersson, 2020). These accountability questions remain largely unresolved within current regulatory frameworks, creating a climate of uncertainty that may inhibit wholehearted adoption.
Peer perception effects represent an emerging consideration. Yang et al. (2025) found that clinician use of generative AI in medical decision-making influences how colleagues perceive their competence and judgement. These social dynamics may create pressure either toward or against AI adoption independent of technical merit, introducing interpersonal factors into reliability assessments.
The collective weight of implementation challenges suggests that technical accuracy alone is insufficient for successful AI integration. Successful deployment requires attention to human factors, workflow design, regulatory clarity, and organisational culture alongside algorithmic performance.
Discussion
The synthesised evidence reveals that clinicians judge the reliability of AI-generated triage decisions through a multifaceted evaluation process integrating technical performance assessment, alignment verification, and contextual appropriateness judgement. This process demonstrates both strengths and limitations warranting critical examination.
The finding that agreement rates between clinicians and AI systems frequently exceed 80% for non-complex presentations represents an encouraging foundation for AI integration in primary care triage. High concordance suggests that current AI systems have achieved sufficient accuracy to merit serious consideration as clinical support tools. The UK study reporting 84% categorical concordance with Cohen’s kappa of 0.69 provides particularly robust evidence, as kappa values exceeding 0.60 are conventionally interpreted as indicating substantial agreement (Altalib et al., 2025). The absence of significant under-triage events in this study addresses a primary safety concern, suggesting that appropriately designed systems can maintain adequate safety margins.
However, the reliance on concordance with clinician judgement as the primary reliability criterion merits critical scrutiny. The finding that human experts themselves exhibit substantial disagreement in triage classifications fundamentally complicates this approach (Entezarjou et al., 2020). If no consistent gold standard exists among human raters, high agreement with any particular clinician may not indicate optimal decision quality. This observation suggests that future evaluation frameworks should incorporate patient outcome measures rather than relying exclusively on process concordance.
The prominence of confirmation bias in shaping trust judgements raises important concerns. Bashkirova and Krpan’s (2024) demonstration that clinicians preferentially accept congruent recommendations suggests a systematic tendency to undervalue discordant AI advice. Paradoxically, AI systems may offer greatest value precisely when their recommendations diverge from initial clinician judgements—identifying cases where cognitive heuristics or incomplete information might lead to suboptimal decisions. The current pattern, whereby disagreement triggers scepticism rather than reflection, may limit the potential benefits of AI augmentation.
The emphasis clinicians place on explainability aligns with ethical principles of transparency in healthcare AI (Bragazzi and Garbarino, 2023). Clinicians appropriately resist accepting “black box” recommendations that cannot be interrogated or validated. This preference creates design imperatives for AI developers: systems must not merely generate accurate outputs but must communicate reasoning in formats accessible to clinical users. The explainable AI movement in healthcare informatics responds to this legitimate professional requirement.
The finding that workflow integration substantially influences adoption speaks to implementation science principles emphasising that interventions must accommodate existing practices to achieve uptake (May and Finch, 2009). Technically superior systems that create workflow friction face adoption barriers regardless of accuracy, whilst less sophisticated systems achieving seamless integration may be preferred. This pattern underscores that AI deployment is fundamentally a sociotechnical challenge requiring attention to human and organisational factors alongside algorithmic development.
Legal accountability concerns represent a particularly intractable barrier deserving policy attention. Current regulatory frameworks inadequately address scenarios involving shared human-AI decision-making, creating uncertainty that reasonably inhibits full reliance on algorithmic recommendations (Gottliebsen and Petersson, 2020). Clinicians rationally hesitate to accept recommendations when liability implications remain ambiguous. Regulatory clarification regarding accountability allocation in AI-assisted decision-making should constitute a priority for healthcare governance bodies.
The evidence reviewed substantially addresses the stated research objectives. Objective one, examining agreement between clinicians and AI systems, finds consistent evidence of substantial concordance, particularly for straightforward presentations. Objective two, identifying trust determinants, reveals explainability, safety-consciousness, alignment with expertise, and workflow integration as primary factors. Objective three, evaluating barriers, identifies confirmation bias, contextual limitations, and accountability concerns as significant constraints. The evidence thus provides a comprehensive picture of the factors shaping clinician reliability judgements, whilst simultaneously highlighting the complexity and conditionality of these assessments.
Conclusions
This dissertation has systematically examined how clinicians judge the reliability of AI-generated triage decisions in primary care, synthesising evidence from 50 peer-reviewed papers addressing this emerging domain. The analysis demonstrates that clinicians employ a sophisticated evaluation process integrating comparison with personal expertise, assessment against established protocols, and consideration of system characteristics including transparency and safety orientation.
The stated objectives have been substantially achieved through this synthesis. First, the evidence consistently demonstrates that substantial agreement exists between clinician assessments and AI recommendations, with concordance rates frequently exceeding 80% for non-complex presentations and statistically significant kappa coefficients indicating genuine agreement beyond chance. Second, the analysis identifies a constellation of factors influencing trust, with explainability, safety-conscious design, alignment with clinical intuition, and workflow integration emerging as primary determinants. Third, significant barriers have been characterised, including confirmation bias that may lead clinicians to inappropriately discount valuable discordant recommendations, contextual data limitations preventing AI systems from incorporating full patient complexity, and unresolved legal accountability questions creating uncertainty inhibiting adoption. Fourth, the implications for system design, clinical implementation, and research priorities have been critically assessed, revealing the necessity of sociotechnical approaches attending to human factors alongside technical development.
The significance of these findings extends across multiple domains. For AI developers, the evidence underscores that technical accuracy represents a necessary but insufficient condition for successful deployment; systems must additionally achieve explainability, workflow integration, and appropriate safety orientation. For clinicians and healthcare organisations, the findings inform expectations regarding AI capabilities and limitations, supporting appropriately calibrated trust. For policymakers, the unresolved accountability questions demand attention, as regulatory clarity constitutes a prerequisite for confident AI integration.
Several research gaps warrant prioritised investigation. Long-term patient outcome studies following AI triage implementation remain notably absent; current evidence predominantly addresses process measures rather than ultimate outcomes. Strategies for mitigating confirmation bias in clinician-AI interaction require development and evaluation. Legal and regulatory frameworks governing accountability in AI-assisted decision-making demand scholarly analysis and policy development. Studies examining AI performance across diverse patient populations are needed to assess equity implications.
Future research should pursue pragmatic trials assessing patient outcomes in authentic implementation contexts, complementing the accuracy studies dominating current literature. Co-design methodologies involving both clinicians and patients in AI system development may enhance acceptability and appropriateness. Investigation of training interventions that calibrate clinician trust—reducing both uncritical acceptance and inappropriate scepticism—could maximise the benefits of human-AI collaboration.
In summary, whilst validated AI systems can achieve high concordance with clinician judgement in primary care triage and are generally trusted when transparent and safety-conscious, ongoing human oversight remains essential. The evidence supports cautious optimism regarding AI integration, tempered by recognition that technical capability must be accompanied by attention to implementation factors, cognitive biases, and governance frameworks if the potential benefits are to be fully realised.
References
Altalib, S., Mbbs, M., Riboli-Sasco, E., Ammouri, M., Gibson, H., Leung, K., Painter, A., BMBCh, M., Rajan, R. and El-Osta, A., 2025. Benchmarking artificial intelligence vs general practitioners decision-making in same-day appointments triage: a mixed-methods study in UK primary care. *medRxiv*. https://doi.org/10.1101/2025.06.11.25329441
Arksey, H. and O’Malley, L., 2005. Scoping studies: towards a methodological framework. *International Journal of Social Research Methodology*, 8(1), pp. 19-32.
Baker, A., Perov, Y., Middleton, K., Baxter, J., Mullarkey, D., Sangar, D., Butt, M., DoRosario, A. and Johri, S., 2020. A comparison of artificial intelligence and human doctors for the purpose of triage and diagnosis. *Frontiers in Artificial Intelligence*, 3. https://doi.org/10.3389/frai.2020.543405
Bashkirova, A. and Krpan, D., 2024. Confirmation bias in AI-assisted decision-making: AI triage recommendations congruent with expert judgments increase psychologist trust and recommendation acceptance. *Computers in Human Behavior: Artificial Humans*. https://doi.org/10.1016/j.chbah.2024.100066
Bragazzi, N. and Garbarino, S., 2023. Toward clinical generative AI: conceptual framework. *JMIR AI*, 3. https://doi.org/10.2196/55957
Delshad, S., Dontaraju, V. and Chengat, V., 2021. Artificial intelligence-based application provides accurate medical triage advice when compared to consensus decisions of healthcare providers. *Cureus*, 13. https://doi.org/10.7759/cureus.16956
Entezarjou, A., Bonamy, A., Benjaminsson, S., Herman, P. and Midlöv, P., 2020. Human- versus machine learning–based triage using digitalized patient histories in primary care: comparative study. *JMIR Medical Informatics*, 8. https://doi.org/10.2196/18930
Goh, E., Bunning, B., Khoong, E., Gallo, R., Milstein, A., Centola, D. and Chen, J., 2025. Physician clinical decision modification and bias assessment in a randomized controlled trial of AI assistance. *Communications Medicine*, 5. https://doi.org/10.1038/s43856-025-00781-2
Gottliebsen, K. and Petersson, G., 2020. Limited evidence of benefits of patient operated intelligent primary care triage tools: findings of a literature review. *BMJ Health & Care Informatics*, 27. https://doi.org/10.1136/bmjhci-2019-100114
Ilicki, J., 2022. Challenges in evaluating the accuracy of AI-containing digital triage systems: a systematic review. *PLOS ONE*, 17. https://doi.org/10.1371/journal.pone.0279636
Levine, D., Tuwani, R., Kompa, B., Varma, A., Finlayson, S., Mehrotra, A. and Beam, A., 2024. The diagnostic and triage accuracy of the GPT-3 artificial intelligence model: an observational study. *The Lancet Digital Health*, 6(8), pp. e555-e561. https://doi.org/10.1016/s2589-7500(24)00097-9
May, C. and Finch, T., 2009. Implementing, embedding, and integrating practices: an outline of normalization process theory. *Sociology*, 43(3), pp. 535-554.
NHS England, 2019. *The NHS Long Term Plan*. London: NHS England. Available at: https://www.longtermplan.nhs.uk
Paslı, S., Sahin, A., Beşer, M., Topçuoğlu, H., Yadigaroğlu, M. and Imamoğlu, M., 2024. Assessing the precision of artificial intelligence in emergency department triage decisions: insights from a study with ChatGPT. *The American Journal of Emergency Medicine*, 78, pp. 170-175. https://doi.org/10.1016/j.ajem.2024.01.037
Razzaki, S., Baker, A., Perov, Y., Middleton, K., Baxter, J., Mullarkey, D., Sangar, D., Taliercio, M., Butt, M., Majeed, A., DoRosario, A., Mahoney, M. and Johri, S., 2018. A comparative study of artificial intelligence and human doctors for the purpose of triage and diagnosis. *ArXiv*, abs/1806.10698.
Steerling, E., Svedberg, P., Nilsen, P., Siira, E. and Nygren, J., 2025. Influences on trust in the use of AI-based triage—an interview study with primary healthcare professionals and patients in Sweden. *Frontiers in Digital Health*, 7. https://doi.org/10.3389/fdgth.2025.1565080
Yang, H., Dai, T., Mathioudakis, N., Knight, A., Nakayasu, Y. and Wolf, R., 2025. Peer perceptions of clinicians using generative AI in medical decision-making. *NPJ Digital Medicine*, 8. https://doi.org/10.1038/s41746-025-01901-x
Zaboli, A., Brigo, F., Brigiari, G., Massar, M., Parodi, M., Pfeifer, N., Magnarelli, G. and Turcato, G., 2025. Chat-GPT in triage: still far from surpassing human expertise – an observational study. *The American Journal of Emergency Medicine*, 92, pp. 165-171. https://doi.org/10.1016/j.ajem.2025.03.028
Zeltzer, D., Herzog, L., Pickman, Y., Steuerman, Y., Ber, R., Kugler, Z., Shaul, R. and Ebbert, J., 2023. Diagnostic accuracy of artificial intelligence in virtual primary care. *Mayo Clinic Proceedings: Digital Health*, 1, pp. 480-489. https://doi.org/10.1016/j.mcpdig.2023.08.002
