Medical Education AssessmentAHEADNCT07481162
AI vs Human Exam Assessment and Development (AHEAD Trial)
A study testing AI-generated MCQ examination and Human-generated MCQ examination in people with Medical Education Assessment, run by University of British Columbia at 1 site in Canada.
Trial summary
The Artificial Intelligence (AI) vs Human Exam Assessment and Development (AHEAD) Trial is a participant-blinded randomized controlled trial conducted among first-year medical students at the University of British Columbia. Read more about this study
- Who is it for?
- Ages 18 years and olderHealthy volunteers can take part
- Conditions the study is for
- Medical Education Assessment
- What is being tested?
- AI-generated MCQ examination and Human-generated MCQ examination
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 258 participants
- Where
- 1 siteCanada
- Study dates
- 8 Dec 2024to 9 Dec 2024
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 years and older
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Who may be able to join
- Enrolled first-year medical students in the University of British Columbia MD undergraduate program.
- Can voluntarily consent to participate in the formative mock examination study.
Who cannot join
- Students who declined participation.
- Students who did not complete the mock examination or required surveys.
Trial locations and contacts
1 site in 1 country.
Canada1 site
University of British Columbia Faculty of Medicine
Vancouver, British Columbia, V5Z 1M9
About this study
Official title: Psychometric Performance and Student Perceptions of AI- Versus Human-Generated Multiple-Choice Question Development in Medical Education: The AHEAD Randomized Controlled Trial
The Artificial Intelligence (AI) vs Human Exam Assessment and Development (AHEAD) Trial is a participant-blinded randomized controlled trial conducted among first-year medical students at the University of British Columbia. The study evaluates whether multiple-choice examination questions generated using large language models (LLMs) perform comparably to traditionally human-written questions in medical education.
Participants were randomized to complete one of two versions of a formative mock final examination consisting of 112 case-based single-best-answer multiple-choice questions (MCQs) aligned with the same course learning objectives. One exam version contained AI-generated questions produced using a structured LLM workflow with independent AI verification, while the other contained questions authored by senior medical students using conventional methods.
The study evaluates exam feasibility, psychometric reliability, validity, student acceptability, and educational impact. Outcomes include exam performance, item discrimination indices, distractor efficiency, student perceptions of exam quality and difficulty, and changes in perceived preparedness for the upcoming summative examination.
Read the detailed description
The AHEAD Trial (AI vs Human Exam Assessment and Development) is a single-center, participant-blinded randomized controlled trial conducted among first-year Doctor of Medicine (MD) students enrolled in the Foundations of Medical Practice I (MEDD 411) course at the University of British Columbia.
Participants were randomized in a 1:1 ratio to complete either an AI-generated or a human-generated mock final examination. Both exams consisted of 112 case-based single-best-answer multiple-choice questions (MCQs) aligned with the same MEDD 411 curricular objectives.
AI-generated questions were produced using a structured workflow involving ChatGPT for question generation and Google Gemini for independent verification. Human-generated questions were authored by senior medical students without AI assistance and underwent independent peer review. Both exams followed identical formatting guidelines and assessed the same learning objectives.
All participants completed identical pre-exam and post-exam surveys assessing demographic characteristics, familiarity with artificial intelligence in education, and perceptions of the examination experience. The study evaluates the utility of AI-generated assessments using van der Vleuten's Assessment Utility Framework, including feasibility, reliability, validity, acceptability, and educational impact.
The trial aims to determine whether large language models can accelerate the development of formative medical examinations while maintaining comparable psychometric quality and educational value relative to traditional human-authored questions.
- Conditions
- Medical Education Assessment
- Being tested
- AI-generated MCQ examination, Human-generated MCQ examination
- Keywords
- Artificial Intelligence, Medical Education, Large Language Models, Multiple Choice Questions, Educational Assessment, Randomized Controlled Trial, ChatGPT
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
AI-generated MCQ examinationOther
A formative mock examination composed of 112 case-based multiple-choice questions generated using large language models aligned with course learning objectives.
Human-generated MCQ examinationOther
A formative mock examination composed of 112 case-based multiple-choice questions written by senior medical students using conventional item-writing methods aligned with the same course learning objectives.
| Study group | Role | What they receive |
|---|---|---|
| AI-Generated MCQ Examination | Experimental | Participants completed a 112-item case-based single-best-answer mock examination composed of AI-generated multiple-choice questions. Questions were generated using a structured large language model workflow with ChatGPT-4 for generation and Google Gemini for independent validation. Receives: Other: AI-generated MCQ examination |
| Human-Generated MCQ Examination | Active comparator | Participants completed a 112-item case-based single-best-answer mock examination composed of human-authored multiple-choice questions developed by senior medical students using traditional item-writing methods and peer review. Receives: Other: Human-generated MCQ examination |
What is being measured
The primary outcome is the main question the study is designed to answer.
Student performance on the mock examination
Comparison of mean examination scores between students randomized to the AI-generated versus human-generated mock examinations.
Measured: Immediately after completion of the mock examination
Secondary outcomes (5)
Item discrimination index
Item-level discrimination index comparing AI-generated and human-generated multiple-choice questions, representing the difference in the proportion of correct responses between high-performing and low-performing students.
Measured: Immediately after the completion of the mock examination
Distractor efficiency
Proportion of distractors selected by at least 5% of participants, comparing AI-generated and human-generated questions.
Measured: Immediately after the completion of the mock examination
Student-rated examination quality and acceptability
Student ratings of exam difficulty, clarity, relevance to course material, adequacy of time, multiple-choice question quality, understanding of clinical concepts, identification of knowledge gaps, retention for future clinical practice, and preparedness for the upcoming summative exam, measured immediately after exam completion using 10-point Likert scales (1 = lowest rating, 10 = highest rating). For most domains, higher scores indicate greater endorsement of the construct being measured; for the difficulty item, higher scores indicate greater perceived difficulty.
Measured: Immediately after completion of the mock examination
Efficiency ratio of MCQ development time per matched learning objective
The outcome measuring the development efficiency of artificial intelligence (AI)-generated versus human-generated multiple-choice questions (MCQs). The efficiency ratio was calculated as human-generated MCQ development time divided by AI-generated MCQ development time for matched learning objectives.
Measured: Baseline (prior to participant testing)
Change in perceived preparedness for the summative examination
Change from pre-exam to post-exam in self-rated preparedness for the upcoming summative examination, measured on a 10-point Likert scale (1 = not at all prepared; 10 = extremely prepared), with higher scores indicating greater perceived preparedness.
Measured: Before and immediately after completion of the mock examination
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted18 Mar 2026
- Study start8 Dec 2024
- Primary completion9 Dec 2024
- Study completion9 Dec 2024
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (1)Papers the sponsor has linked to this study.
- Other trials for Medical Education AssessmentStudies of the same condition, open and under way.
- More trials run by University of British ColumbiaThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Model description
- Participants were randomized 1:1 to complete either an AI-generated or human-generated mock examination composed of 112 case-based multiple-choice questions aligned with the same course learning objectives.
- Masking
- Single blind
- Who is masked
- Participant
- Enrolment
- 258 participants (actual)
Sponsor and investigators
- Lead sponsor
- University of British Columbia
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Anita Palepu, University of British Columbia
- Investigators
- Anita Palepu, MD, MPH, FRCPC, Principal investigator, University of British Columbia
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07481162
- Sponsor's protocol ID
- Med_AHEAD_Trial
- Other IDs
- H16-00044
- Acronym
- AHEAD
- Status last verified
- Mar 2026
- First submitted
- 7 Mar 2026
- Last updated
- 18 Mar 2026
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- Yes: Deidentified participant-level examination scores and survey responses will be available upon reasonable request.
Publications
Al-Najafi D, Krause KD, Wang Y, Zuo QK, Koblanski ME, Leong CJ, Schmidt E, Faran M, Verma V, Vyas R, Campbell M, Hwang J, Deng J, Palepu A. Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Med Educ. 2026 Jun 17. doi: 10.1186/s12909-026-09671-0. Online ahead of print.
PubMed 42304363
Study at a glance
From the registry record, last updated by the sponsor on 18 Mar 2026. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Psychometric Performance and Student Perceptions of AI- Versus Human-Generated Multiple-Choice Question Development in Medical Education: The AHEAD Randomized Controlled Trial
- ClinicalTrials.gov ID
- NCT07481162
- Acronym / sponsor ID
- AHEAD / Med_AHEAD_Trial
- Conditions
- Medical Education Assessment
- Keywords
- Artificial Intelligence, Medical Education, Large Language Models, Multiple Choice Questions, Educational Assessment, Randomized Controlled Trial, ChatGPT
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 258 participants (actual)
- Lead sponsor
- University of British ColumbiaUniversity, hospital or other
- Study start
- 8 Dec 2024 (actual)
- Primary completion
- 9 Dec 2024 (actual)
- Study completion
- 9 Dec 2024 (actual)
- First posted
- 18 Mar 2026 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
The Artificial Intelligence (AI) vs Human Exam Assessment and Development (AHEAD) Trial is a participant-blinded randomized controlled trial conducted among first-year medical students at the University of British Columbia. The study evaluates whether multiple-choice examination questions generated using large language models (LLMs) perform comparably to traditionally human-written questions in medical education.
Participants were randomized to complete one of two versions of a formative mock final examination consisting of 112 case-based single-best-answer multiple-choice questions (MCQs) aligned with the same course learning objectives. One exam version contained AI-generated questions produced using a structured LLM workflow with independent AI verification, while the other contained questions authored by senior medical students using conventional methods.
The study evaluates exam feasibility, psychometric reliability, validity, student acceptability, and educational impact. Outcomes include exam performance, item discrimination indices, distractor efficiency, student perceptions of exam quality and difficulty, and changes in perceived preparedness for the upcoming summative examination.
Detailed description
The AHEAD Trial (AI vs Human Exam Assessment and Development) is a single-center, participant-blinded randomized controlled trial conducted among first-year Doctor of Medicine (MD) students enrolled in the Foundations of Medical Practice I (MEDD 411) course at the University of British Columbia.
Participants were randomized in a 1:1 ratio to complete either an AI-generated or a human-generated mock final examination. Both exams consisted of 112 case-based single-best-answer multiple-choice questions (MCQs) aligned with the same MEDD 411 curricular objectives.
AI-generated questions were produced using a structured workflow involving ChatGPT for question generation and Google Gemini for independent verification. Human-generated questions were authored by senior medical students without AI assistance and underwent independent peer review. Both exams followed identical formatting guidelines and assessed the same learning objectives.
All participants completed identical pre-exam and post-exam surveys assessing demographic characteristics, familiarity with artificial intelligence in education, and perceptions of the examination experience. The study evaluates the utility of AI-generated assessments using van der Vleuten's Assessment Utility Framework, including feasibility, reliability, validity, acceptability, and educational impact.
The trial aims to determine whether large language models can accelerate the development of formative medical examinations while maintaining comparable psychometric quality and educational value relative to traditional human-authored questions.
Study design
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind (participant)
- Number of arms
- 2
- Model description
- Participants were randomized 1:1 to complete either an AI-generated or human-generated mock examination composed of 112 case-based multiple-choice questions aligned with the same course learning objectives.
| Arm | Type | Receives |
|---|---|---|
| AI-Generated MCQ Examination | Experimental | Other: AI-generated MCQ examinationParticipants completed a 112-item case-based single-best-answer mock examination composed of AI-generated multiple-choice questions. Questions were generated using a structured large language model workflow with ChatGPT-4 for generation and Google Gemini for independent validation. |
| Human-Generated MCQ Examination | Active comparator | Other: Human-generated MCQ examinationParticipants completed a 112-item case-based single-best-answer mock examination composed of human-authored multiple-choice questions developed by senior medical students using traditional item-writing methods and peer review. |
Who may qualify
- Ages
- 18 years and older
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
Key inclusion criteria (2 listed)
- Enrolled first-year medical students in the University of British Columbia MD undergraduate program.
- Can voluntarily consent to participate in the formative mock examination study.
Key exclusion criteria (2 listed)
- Students who declined participation.
- Students who did not complete the mock examination or required surveys.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| AI-generated MCQ examinationA formative mock examination composed of 112 case-based multiple-choice questions generated using large language models aligned with course learning objectives. | Other | AI-Generated MCQ Examination |
| Human-generated MCQ examinationA formative mock examination composed of 112 case-based multiple-choice questions written by senior medical students using conventional item-writing methods aligned with the same course learning objectives. | Other | Human-Generated MCQ Examination |
Primary outcome
Student performance on the mock examination
Time frame: Immediately after completion of the mock examination
Comparison of mean examination scores between students randomized to the AI-generated versus human-generated mock examinations.
5 secondary outcomes
Item discrimination index
Time frame: Immediately after the completion of the mock examination
Distractor efficiency
Time frame: Immediately after the completion of the mock examination
Student-rated examination quality and acceptability
Time frame: Immediately after completion of the mock examination
Efficiency ratio of MCQ development time per matched learning objective
Time frame: Baseline (prior to participant testing)
Change in perceived preparedness for the summative examination
Time frame: Before and immediately after completion of the mock examination
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 18 Mar 2026.
- Derived from this study: PubMed 42304363Al-Najafi D, Krause KD, Wang Y, Zuo QK, Koblanski ME, Leong CJ, Schmidt E, Faran M, Verma V, Vyas R, Campbell M, Hwang J, Deng J, Palepu A. Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Med Educ. 2026 Jun 17. doi: 10.1186/s12909-026-09671-0. Online ahead of print.
- Individual participant dataSharing planned: Yes. Deidentified participant-level examination scores and survey responses will be available upon reasonable request.
Related trials for Medical Education Assessment
RecruitingObservational study
FOXP1 Syndrome: The Seaver Autism Center for Research and Treatment is Characterizing FOXP1-related Neurodevelopmental Disorders Using Genetic, Medical, and Neuropsychological Measures.
FOXP1, Mental Retardation With Language Impairment and With or Without Autistic Features
Not yet recruiting
ECHO-COPD: Impact of a Pharmacist-Led Health Belief Model-Based Educational Intervention Among Chronic Obstructive Pulmonary Disease Patients
Medication Adherence, Quality of Life
RecruitingObservational study
ATUSA Ultrasound Diagnostic Imaging for Breast Lesions Evaluation (AUDIBLE) - iSono Health
Breast Cancer, Benign Breast Condition
Active, not recruitingPhase 3
Evaluation of a Disability-Focused Sexual and Reproductive Health Curriculum for Health Students in Tanzania
Sexual and Reproductive Health, Disability
Not yet recruitingObservational study
A Mixed-Methods Analysis of the Six Core Competencies and AI Literacy Reflected in Medical Students' Self-Assessment During Small-Group Discussions
Core Competency, Reflective Thinking
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Anita Palepu, MD, MPH, FRCPC principal investigator
University of British Columbia
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing AI-generated MCQ examination and Human-generated MCQ examination in people with Medical Education Assessment, run by University of British Columbia at 1 site in Canada.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 2 inclusion and 2 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 18 Mar 2026. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in Canada. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Student performance on the mock examination (time frame: Immediately after completion of the mock examination). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT07481162, which also lists 1 linked publication.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 18 Mar 2026.