Artificial Intelligence (AI)TEACH-AINCT07813754
Teaching Doctors in Training to Reason With Artificial Intelligence
A study testing Workshop-First in people with Artificial Intelligence (AI) and Clinical Reasoning, run by Beth Israel Deaconess Medical Center at 4 sites in United States.
Trial summary
The purpose of the TEACH-AI study is to assess whether a brief, structured workshop on artificial intelligence can improve the performance of medicine doctors in training (i.e. residents) in their diagnostic and management reasoning. Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Artificial Intelligence (AI), Clinical Reasoning, Clinical Decision-making
- What is being tested?
- Workshop-First
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 200 participants
- Where
- 4 sitesUnited States
- Study dates
- 24 Sep 2026to Jun 2027
- Phase
- Interventional
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- Participants must be licensed physicians and have started at least post-graduate year 1 (PGY1) of medical training.
- Training in Internal medicine or family medicine or emergency medicine.
- Able to provide informed consent and complete assessments in English.
Who cannot join
- Not currently practicing clinically.
- Resident directly involved in the design of the study, development of the workshop, or creation/piloting of the assessment vignettes.
- Resident who declines or withdraws consent.
Trial locations and contacts
4 sites in 1 country. Contact the study team or a site near you to ask about taking part.
United States4 sites
Beth Israel Deaconess Medical Center
Boston, Massachusetts, 02215
Cambridge Health Alliance
Cambridge, Massachusetts, 02139
AdventHealth
Orlando, Florida, 32804
Stanford University
Palo Alto, California, 94305
About this study
Official title: Teaching Internal Medicine and Family Medicine Residents to Reason With Generative AI: A Multi-Site Randomized Controlled Trial (TEACH-AI)
The purpose of the TEACH-AI study is to assess whether a brief, structured workshop on artificial intelligence can improve the performance of medicine doctors in training (i.e. residents) in their diagnostic and management reasoning.
In this multi-site randomized controlled trial, internal medicine and family medicine residents are assigned either to receive an in-person workshop on safe, effective LLM use before a standardized AI-assisted assessment, or to complete the same assessment before receiving the workshop. Residents will review clinical cases that are fully synthetic, no protected health information is used, using a password protected LLM interface.
Read the detailed description
Large language models (LLMs) have rapidly entered routine use in medical education and clinical practice. Prior randomized trials have shown that, although LLMs can outperform individual clinicians on some reasoning benchmarks, providing physicians with access to an LLM does not necessarily improve their diagnostic or management performance, and LLMs alone may perform better than physician-plus-LLM teams. These findings suggest that human-AI collaboration is fraught, non-trivial and may require explicit training.
TEACH-AI is a pragmatic, multi-site, two-arm randomized controlled trial embedded within protected residency didactic time. The primary objective is to determine whether a single in-person workshop on the basics of LLMs and best practices of prompting/verification strategies improves residents' performance on an AI-assisted assessment, compared with residents who complete the simulation before receiving the workshop. All participants will ultimately receive the same workshop and the same assessment (either workshop-first vs assessment-first).
The simulation consists of multiple fully synthetic vignettes delivered via a password protected LLM interface. Residents interact freely with the LLM using natural-language prompts and then submit structured final responses regarding aspects such as leading diagnosis, differential, management plan, and/or justification. The platform will record prompts, model outputs, final answers, and timing. Vignette scoring combines correctness of the final diagnosis or management plan. Scoring is conducted by blinded faculty using standardized rubrics and then any discrepancies will be resolved through multiple rounds of discussions.
The trial will enroll up to 200 residents across four ACGME-accredited programs (internal medicine at Beth Israel Deaconess Medical Center, Stanford University, and Cambridge Health Alliance; family medicine at AdventHealth Orlando).
- Conditions
- Artificial Intelligence (AI), Clinical Reasoning, Clinical Decision-making, Medical Education
- Being tested
- Workshop-First
- Keywords
- artificial intelligence, medical education, clinical reasoning, clinical decision-making, human-computer interaction
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Workshop-FirstOther
Participants will attend a workshop during their didactic time that will review various aspects of large language models including fundamentals and best practices of interacting and interpreting their outputs.
| Study group | Role | What they receive |
|---|---|---|
| Assessment-First | No intervention | Participants assigned to this group will be tasked with completing the assessment using a password protected LLM interface first in advance of the workshop. The workshop will be delivered during a scheduled mandatory didactic time slot. |
| Workshop-first | Experimental | Participants assigned to this group will be tasked with completing the assessment using a password protected LLM interface after they participate in a scheduled mandatory didactic time slot. Receives: Other: Workshop-First |
What is being measured
The primary outcome is the main question the study is designed to answer.
Total Score on Expert-Development Rubrics
The primary outcome will be the number of correct responses for all cases using select questions from expert-developed scoring rubrics. The rubrics were developed using a Delphi consensus process by expert physicians as established by Goh et al (Nature Medicine, 2025; DOI: 10.1038/s41591-024-03456-y). The primary outcome will be analyzed at the case level, comparing performance between the randomized study groups, with a higher number of correct responses indicating a better outcome.
Measured: Within 24 hours of assessment completion.
Secondary outcomes (4)
Time Spent on Management
Comparison of time spent per case between the two study groups.
Measured: Within 24 hours of assessment completion.
Prompt Count
Comparison of the number of resident-initiated prompts to the LLM per case between the two study groups.
Measured: Within 24 hours of assessment completion.
Management Reasoning Using Expert-Derived Rubrics
Comparison of management reasoning accuracy based on the number of correct responses per case between the two groups on expert-developed scoring rubrics. The rubrics were developed using a Delphi consensus process by expert physicians as established by Goh et al (Nature Medicine, 2025; DOI: 10.1038/s41591-024-03456-y). The primary outcome will be analyzed at the case level, comparing performance between the randomized study groups with more correct responses indicating a better outcome.
Measured: Within 24 hours of assessment completion.
Diagnostic Reasoning
Comparison of diagnostic reasoning accuracy based on the number of correct responses per case between the two groups.
Measured: Within 24 hours of assessment completion.
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted10 Sep 2026
- Study start24 Sep 2026
- Primary completion31 Dec 2026 (estimated)
- Study completionJun 2027 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Artificial Intelligence (AI)Studies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Model description
- The trial will be designed as a randomized, two-arm, single-blind parallel group study.
- Masking
- Single blind
- Who is masked
- Outcomes assessor
- Enrolment
- 200 participants (estimated)
Sponsor and investigators
- Lead sponsor
- Beth Israel Deaconess Medical Center
- Sponsor type
- University, hospital or other
- Collaborators
- Stanford University, Cambridge Health Alliance, AdventHealth
- Responsible party
- Principal investigator, Jacob Koshy, Beth Israel Deaconess Medical Center
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07813754
- Sponsor's protocol ID
- 2026P000369
- Other IDs
- 63901784-350738
- Acronym
- TEACH-AI
- Status last verified
- Sep 2026
- First submitted
- 10 Aug 2026
- Last updated
- 25 Sep 2026
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No