DiagnosisNCT06963957
Automation Bias in Physician-LLM Diagnostic Reasoning
A study testing ChatGPT-4o Recommendations with Hallucinations in people with Diagnosis, run by Lahore University of Management Sciences at 1 site in Pakistan.
Trial summary
This study aims to systematically measure the extent and patterns of automation bias among physicians when utilizing ChatGPT-4o in clinical decision-making. Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Diagnosis
- What is being tested?
- ChatGPT-4o Recommendations with Hallucinations
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 44 participants
- Where
- 1 sitePakistan
- Study dates
- 20 Jun 2025to 15 Aug 2025
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- Completed Bachelor of Medicine, Bachelor of Surgery (MBBS) Exam. The equivalent degree of MBBS in US and Canada is called Doctor of Medicine (MD).
- Full or Provisionally Registered Medical Practitioners with the Pakistan Medical and Dental Council (PMDC).
- Participants must have completed a structured training program on the use of ChatGPT (or a comparable large language model), totaling at least 10 hours of instruction. The program must include hands-on practice related to LLM's aspects, specifically prompt engineering and content evaluation.
Who cannot join
- Any other Registered Medical Practitioners (Full or Provisional) with PMDC (e.g., Professionals with Bachelor of Dental Surgery or BDS).
Trial locations and contacts
1 site in 1 country.
Pakistan1 site
Lahore University of Management Sciences
Lahore, Punjab Province, 54000
About this study
Official title: Trust or Verify? Automation Bias in Physician-LLM Diagnostic Reasoning
This study aims to systematically measure the extent and patterns of automation bias among physicians when utilizing ChatGPT-4o in clinical decision-making.
Read the detailed description
Diagnostic errors represent a significant cause of preventable patient harm in healthcare systems worldwide. Recent advances in Large Language Models (LLMs) have shown promise in enhancing medical decision-making processes.
However, there remains a critical gap in our understanding of how automation bias -- the tendency to over-rely on technological suggestions -- influences medical doctors' diagnostic reasoning when incorporating these AI tools into clinical practice.
Automation bias presents substantial risks in clinical environments, particularly as AI tools become more integrated into healthcare workflows. Although LLMs such as ChatGPT-4o offer potential advantages in reducing errors and improving efficiency, their lack of rigorous medical validation raises concerns about potentially amplifying cognitive biases through the generation of incorrect or misleading information.
Multiple contextual factors can exacerbate automation bias in medical settings: time constraints in high-volume clinical settings, financial incentives that prioritize efficiency over thoroughness, cognitive fatigue during extended shifts, and diminished vigilance when confronting diagnostically challenging cases.
These factors may interact with psychological mechanisms that include the diffusion of responsibility, overconfidence in technological solutions, and cognitive offloading---collectively increasing the risk of uncritical acceptance of AI-generated recommendations.
This randomized controlled trial (RCT) aims to systematically measure the extent and patterns of automation bias among physicians when utilizing ChatGPT-4o in clinical decision-making. The investigators will assess how access to LLM-generated information influences diagnostic reasoning through a novel methodology that precisely quantifies automation bias. In this study, participants will be randomly assigned to one of two groups. The treatment group will receive LLM-generated recommendations containing deliberately introduced errors in a subset of cases, while the control group will receive LLM-generated recommendations without such deliberately introduced errors. Participants will evaluate six clinical vignettes randomly sequenced to prevent detection patterns. The flawed vignettes provided to the treatment group will incorporate subtle yet clinically significant errors that should be identifiable by trained doctors. This will enable investigators to quantify the degree of automation bias by measuring the differential in diagnostic accuracy scores between the treatment and control groups.
Prior to participation, all physicians will complete a comprehensive training program covering LLM capabilities, prompt engineering techniques, and output evaluation strategies. Responses will be evaluated by blinded reviewers using a validated assessment rubric specifically designed to detect uncritical acceptance of erroneous information, with greater score disparities indicating stronger automation bias. This naturalistic approach will yield insights directly applicable to real clinical workflows, where mounting cognitive demands may progressively impact diagnostic decision quality.
- Conditions
- Diagnosis
- Being tested
- ChatGPT-4o Recommendations with Hallucinations
- Keywords
- clinical reasoning, large language models, automation bias, computer-assisted diagnosis
- Medical subject headings
- Disease
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
ChatGPT-4o Recommendations with HallucinationsOther
ChatGPT-4o's differential diagnoses of six clinical vignettes, three of which will contain deliberately introduced inaccurate information.
| Study group | Role | What they receive |
|---|---|---|
| ChatGPT-4o Recommendations with Hallucinations | Active comparator | Participants will evaluate six clinical vignettes. During the trial, they will have access to clinical recommendations from a specific, commercially available LLM (ChatGPT-4o) in addition to conventional diagnostic resources. LLM recommendations for three vignettes will contain deliberately flawed diagnostic information and for three vignettes it will contain accurate recommendations). The cases will be presented in random order. Receives: Other: ChatGPT-4o Recommendations with Hallucinations |
| ChatGPT-4o Recommendations without Hallucinations | No intervention | Participants will evaluate the same six clinical vignettes as in the intervention arm. During the trial, they will have access to clinical recommendations from a specific, commercially available LLM (ChatGPT-4o) in addition to conventional diagnostic resources. However, the LLM-generated recommendations will not contain any deliberately introduced errors. The cases will be presented in random order. |
What is being measured
The primary outcome is the main question the study is designed to answer.
Diagnostic reasoning
The primary outcome will be the percent correct for each case, ranging from 0 to 100%, where higher scores indicate better diagnostic performance. For each case, participants will be asked for their three leading diagnoses, findings that support each diagnosis, and findings that oppose each diagnosis. For each plausible diagnosis, participants will receive 1 point. Findings supporting the diagnosis and findings opposing the diagnosis will also be graded based on correctness, with 1 point for each correct response. Participants will then be asked to name their top diagnosis they believe is most likely, earning 9 points for a reasonable response and 18 points for the most accurate response. Finally participants will be asked to name up to 3 next steps to further evaluate the patient with 0.5 point awarded for a partially correct response and 1 point for a completely correct response. The primary outcome will be compared at the case-level between the randomized groups.
Measured: Assessed at a single time point for each case, during the scheduled diagnostic reasoning evaluation session, which takes place between 0-4 days after participant enrollment.
Secondary outcomes (1)
Top choice diagnosis accuracy score
The secondary outcome will measure participants' performance in identifying the most likely diagnosis for each clinical vignette. After evaluating each case, participants will select their single most likely diagnosis, which will be scored on a pre-specified Three-Tier Diagnostic Accuracy Scale: 18 points for the most accurate diagnosis, 9 points for a clinically reasonable alternative, and 0 points for an incorrect diagnosis. For each participant, a Top Choice Diagnosis Accuracy Score is calculated as (total points earned ÷ maximum possible points) × 100, yielding a 0-100 % range in which higher scores indicate greater diagnostic accuracy. This percentage score will be compared at the case-level between randomized groups to quantify the impact of automation bias on diagnostic decision-making.
Measured: Assessed at a single time point for each case, during the scheduled diagnostic reasoning evaluation session, which takes place between 0-4 days after participant enrollment.
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted9 May 2025
- Study start20 Jun 2025
- Primary completion15 Aug 2025
- Study completion15 Aug 2025
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for DiagnosisStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Model description
- The trial will be designed as a randomized, two-arm, single-blind parallel group study.
- Masking
- Single blind
- Who is masked
- Outcomes assessor
- Enrolment
- 44 participants (actual)
Sponsor and investigators
- Lead sponsor
- Lahore University of Management Sciences
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Ihsan Ayyub Qazi, PhD, Lahore University of Management Sciences
- Investigators
- Ihsan Ayyub Qazi, PhD, Principal investigator, Lahore University of Management Sciences (LUMS)Ayesha Ali, PhD, Principal investigator, Lahore University of Management Sciences (LUMS)Muhammad Asadullah Khawaja, MBBS, Principal investigator, King Edward Medical UniversityAli Zafar Sheikh, MBBS, Principal investigator, Lahore General HospitalMuhammad Junaid Akhtar, MBBS, Principal investigator, Children's Hospital, Lahore
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT06963957
- Sponsor's protocol ID
- IRB-0374
- Status last verified
- Aug 2025
- First submitted
- 23 Apr 2025
- Last updated
- 22 Aug 2025
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Study at a glance
From the registry record, last updated by the sponsor on 22 Aug 2025. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Trust or Verify? Automation Bias in Physician-LLM Diagnostic Reasoning
- ClinicalTrials.gov ID
- NCT06963957
- Acronym / sponsor ID
- IRB-0374
- Conditions
- Diagnosis
- Keywords
- clinical reasoning, large language models, automation bias, computer-assisted diagnosis
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 44 participants (actual)
- Lead sponsor
- Lahore University of Management SciencesUniversity, hospital or other
- Study start
- 20 Jun 2025 (actual)
- Primary completion
- 15 Aug 2025 (actual)
- Study completion
- 15 Aug 2025 (actual)
- First posted
- 9 May 2025 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
This study aims to systematically measure the extent and patterns of automation bias among physicians when utilizing ChatGPT-4o in clinical decision-making.
Detailed description
Diagnostic errors represent a significant cause of preventable patient harm in healthcare systems worldwide. Recent advances in Large Language Models (LLMs) have shown promise in enhancing medical decision-making processes.
However, there remains a critical gap in our understanding of how automation bias -- the tendency to over-rely on technological suggestions -- influences medical doctors' diagnostic reasoning when incorporating these AI tools into clinical practice.
Automation bias presents substantial risks in clinical environments, particularly as AI tools become more integrated into healthcare workflows. Although LLMs such as ChatGPT-4o offer potential advantages in reducing errors and improving efficiency, their lack of rigorous medical validation raises concerns about potentially amplifying cognitive biases through the generation of incorrect or misleading information.
Multiple contextual factors can exacerbate automation bias in medical settings: time constraints in high-volume clinical settings, financial incentives that prioritize efficiency over thoroughness, cognitive fatigue during extended shifts, and diminished vigilance when confronting diagnostically challenging cases.
These factors may interact with psychological mechanisms that include the diffusion of responsibility, overconfidence in technological solutions, and cognitive offloading---collectively increasing the risk of uncritical acceptance of AI-generated recommendations.
This randomized controlled trial (RCT) aims to systematically measure the extent and patterns of automation bias among physicians when utilizing ChatGPT-4o in clinical decision-making. The investigators will assess how access to LLM-generated information influences diagnostic reasoning through a novel methodology that precisely quantifies automation bias. In this study, participants will be randomly assigned to one of two groups. The treatment group will receive LLM-generated recommendations containing deliberately introduced errors in a subset of cases, while the control group will receive LLM-generated recommendations without such deliberately introduced errors. Participants will evaluate six clinical vignettes randomly sequenced to prevent detection patterns. The flawed vignettes provided to the treatment group will incorporate subtle yet clinically significant errors that should be identifiable by trained doctors. This will enable investigators to quantify the degree of automation bias by measuring the differential in diagnostic accuracy scores between the treatment and control groups.
Prior to participation, all physicians will complete a comprehensive training program covering LLM capabilities, prompt engineering techniques, and output evaluation strategies. Responses will be evaluated by blinded reviewers using a validated assessment rubric specifically designed to detect uncritical acceptance of erroneous information, with greater score disparities indicating stronger automation bias. This naturalistic approach will yield insights directly applicable to real clinical workflows, where mounting cognitive demands may progressively impact diagnostic decision quality.
Study design
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind (outcomes assessor)
- Number of arms
- 2
- Model description
- The trial will be designed as a randomized, two-arm, single-blind parallel group study.
| Arm | Type | Receives |
|---|---|---|
| ChatGPT-4o Recommendations with Hallucinations | Active comparator | Other: ChatGPT-4o Recommendations with HallucinationsParticipants will evaluate six clinical vignettes. During the trial, they will have access to clinical recommendations from a specific, commercially available LLM (ChatGPT-4o) in addition to conventional diagnostic resources. LLM recommendations for three vignettes will contain deliberately flawed diagnostic information and for three vignettes it will contain accurate recommendations). The cases will be presented in random order. |
| ChatGPT-4o Recommendations without Hallucinations | No intervention | –Participants will evaluate the same six clinical vignettes as in the intervention arm. During the trial, they will have access to clinical recommendations from a specific, commercially available LLM (ChatGPT-4o) in addition to conventional diagnostic resources. However, the LLM-generated recommendations will not contain any deliberately introduced errors. The cases will be presented in random order. |
Who may qualify
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (3 listed)
- Completed Bachelor of Medicine, Bachelor of Surgery (MBBS) Exam. The equivalent degree of MBBS in US and Canada is called Doctor of Medicine (MD).
- Full or Provisionally Registered Medical Practitioners with the Pakistan Medical and Dental Council (PMDC).
- Participants must have completed a structured training program on the use of ChatGPT (or a comparable large language model), totaling at least 10 hours of instruction. The program must include hands-on practice related to LLM's aspects, specifically prompt engineering and content evaluation.
Key exclusion criteria (1 listed)
- Any other Registered Medical Practitioners (Full or Provisional) with PMDC (e.g., Professionals with Bachelor of Dental Surgery or BDS).
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| ChatGPT-4o Recommendations with HallucinationsChatGPT-4o's differential diagnoses of six clinical vignettes, three of which will contain deliberately introduced inaccurate information. | Other | ChatGPT-4o Recommendations with Hallucinations |
Primary outcome
Diagnostic reasoning
Time frame: Assessed at a single time point for each case, during the scheduled diagnostic reasoning evaluation session, which takes place between 0-4 days after participant enrollment.
The primary outcome will be the percent correct for each case, ranging from 0 to 100%, where higher scores indicate better diagnostic performance. For each case, participants will be asked for their three leading diagnoses, findings that support each diagnosis, and findings that oppose each diagnosis. For each plausible diagnosis, participants will receive 1 point. Findings supporting the diagnosis and findings opposing the diagnosis will also be graded based on correctness, with 1 point for each correct response. Participants will then be asked to name their top diagnosis they believe is most likely, earning 9 points for a reasonable response and 18 points for the most accurate response. Finally participants will be asked to name up to 3 next steps to further evaluate the patient with 0.5 point awarded for a partially correct response and 1 point for a completely correct response. The primary outcome will be compared at the case-level between the randomized groups.
1 secondary outcome
Top choice diagnosis accuracy score
Time frame: Assessed at a single time point for each case, during the scheduled diagnostic reasoning evaluation session, which takes place between 0-4 days after participant enrollment.
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 22 Aug 2025.
- Individual participant dataSharing planned: No
Related trials for Diagnosis
Not yet recruitingPhase 2
Extension Study of Safety and Tolerability of DDY391 in Participants With Moderate to Severe Psoriatic Arthritis (CDDY391A12201-extension)
Moderate to Severe Psoriatic Arthritis
Not yet recruitingPhase 1
Hepatic Artery Infusion of Carfilzomib in Participants With Liver Metastatic Disease Previously Treated With Hepatic Artery Infusion Pump Therapy
Colorectal Neoplasms, Neoplasms
RecruitingPhase 2
Tafasitamab With Acalabrutinib and Venetoclax for the Treatment of Chronic Lymphocytic Leukemia/Small Lymphocytic Lymphoma
Chronic Lymphocytic Leukemia, Chronic Lymphocytic Leukemia/Small Lymphocytic Lymphoma
Active, not recruitingPhase 1/2
Selumetinib in Treating Young Patients With Recurrent or Refractory Low Grade Glioma
Low Grade Glioma, Recurrent Childhood Pilocytic Astrocytoma
Recruiting
Methylation Profiling in Self-Samples for Identification of (Epi)Genetic Biomarkers for Gynecological Cancers
Gynaecological Cancers, Ovarian Cancer
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Ihsan Ayyub Qazi, PhD principal investigator
Lahore University of Management Sciences (LUMS)
Ayesha Ali, PhD principal investigator
Lahore University of Management Sciences (LUMS)
Muhammad Asadullah Khawaja, MBBS principal investigator
King Edward Medical University
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing ChatGPT-4o Recommendations with Hallucinations in people with Diagnosis, run by Lahore University of Management Sciences at 1 site in Pakistan.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 3 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 22 Aug 2025. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in Pakistan. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Diagnostic reasoning (time frame: Assessed at a single time point for each case, during the scheduled diagnostic reasoning evaluation session, which takes place between 0-4 days after participant enrollment.). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 22 Aug 2025.