Educational ProblemsSIMEVALNCT03884114
Comparison of Three Modalities to Assess Clinical Competence of Medical Students
A study testing High-fidelity simulation, Simulation game and Multiple choice questionnaire in people with Educational Problems, run by Ilumens at 1 site in France.
Trial summary
This study, focusing on the management of pediatric asthma exacerbations, assesses the clinical skills of medical students using three different evaluation tools: (i) the simulation game "Effic'Asthme" developed to train individuals on the management of pediatric asthma exacerbations; (ii) a multiple choice… Read more about this study
- Who is it for?
- Ages 18 years and olderHealthy volunteers can take part
- Conditions the study is for
- Educational Problems
- What is being tested?
- High-fidelity simulation, Simulation game and Multiple choice questionnaire
- Type of treatment
- Pill or oral medicine
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 42 participants
- Where
- 1 siteFrance
- Study dates
- 20 Mar 2019to 6 Jul 2019
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 years and older
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Who may be able to join
- medical students from Paris 5 University who meet the following criteria
- Being in their fifth year of medical school
- Having passed their pediatric exam in the previous 15 days
- Willing to participate in the study
Who cannot join
- refusal to participate
Trial locations and contacts
1 site in 1 country.
About this study
Official title: Assessment of Clinical Competence of Medical Students: a Comparison of High-fidelity Simulation, Serious Game and Multiple-choice Questions
This study, focusing on the management of pediatric asthma exacerbations, assesses the clinical skills of medical students using three different evaluation tools: (i) the simulation game "Effic'Asthme" developed to train individuals on the management of pediatric asthma exacerbations; (ii) a multiple choice questionnaire (MCQ) on the same subject developed for the purpose of the study and (iii) high fidelity (HF)-simulation, considered as the gold-standard for its enhanced realism.
Its objective is to determine which of the simulation game or the MCQ reflects the best the clinical competence of medical students evaluated on a HF simulator.
Read the detailed description
The assessment of medical students' clinical competence has long been of concern to educational institutions. Objective structural clinical examination (OSCE) and assessments in high-fidelity (HF) simulation settings represent interesting evaluation modalities but are associated with huge costs, especially when they are used for large-scale standardized assessments. Multiple choice questionnaires (MCQ) remain the most common evaluation tool in medical schools.
Simulation games may represent an interesting compromise between the cheap but limited assessment allowed by MCQs, and the comprehensive but highly expensive assessment allowed by OSCE and HF simulation.
This study, focusing on the management of pediatric asthma exacerbations, assesses the clinical skills of medical students using three different evaluation tools: (i) the simulation game "Effic'Asthme" developed to train individuals on the management of pediatric asthma exacerbations; (ii) a MCQ on the same subject developed for the purpose of the study and (iii) HF-simulation, considered as the gold-standard for its enhanced realism.
Its objective is to determine which of the simulation game or the MCQ reflects the best the clinical competence of medical students evaluated on a HF simulator.
- Conditions
- Educational Problems
- Being tested
- High-fidelity simulation, Simulation game, Multiple choice questionnaire
- Keywords
- technology-enhanced simulation, serious games, multiple choice questionnaires, medical education
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
High-fidelity simulationOther
The first evaluation modality studied is high fidelity (HF) simulation. In our study, this modality is considered to be the gold-standard assessment method of clinical assessment, because HF simulation corresponds to the modality which reflects the best clinical competence in hospital settings. The HF pediatric manikin used (SimBaby), is able to reproduce all the signs of an asthma exacerbation (coughing, wheezing, tachypnea, chest indrawing, seesaw respiration, cyanosis). In the simulation room, participants can use the same items that those present in the simulation game (a glass of water, paracetamol, short acting beta-agonist, controller treatment (Fluticasone), an asthma spacer with a facial mask, saline nose drops to perform nasal irrigation, oral steroids in tablets (prednisone), and a phone).
Simulation gameOther
EfficAsthme is a simulation game used on a tablet computer. This simulation game was developed to train parents on the management of asthma exacerbations of their children. For the purpose of the study, EfficAsthme is diverted from its original use to assess students' clinical skills. The training scenario "A polluted atmosphere" is used in this study. Participants need to observe the signs presented by the child, and to determine the severity of the asthma exacerbation. From a menu on the right of the screen, the participant can choose several actions, especially to provide the short acting beta-agonist.
Multiple choice questionnaireOther
The third evaluation modality corresponds to a multiple choice questionnaire (MCQ) including 15 questions. As for HF-simulation and the simulation game, the MCQ starts with the same briefing and continues with 15 questions regarding the management of a moderate asthma exacerbation.
| Study group | Role | What they receive |
|---|---|---|
| All participants | Other | All participants are evaluated on a scenario of pediatric asthma exacerbation using three different evaluation tools: (i) the simulation game EfficAsthme developed to train individuals on the management of pediatric asthma exacerbations; (ii) a MCQ on the same subject developed for the purpose of the study and (iii) HF-simulation, considered as the gold-standard for its enhanced realism. Receives: Other: High-fidelity simulation; Other: Simulation game; Other: Multiple choice questionnaire |
What is being measured
The primary outcome is the main question the study is designed to answer.
Items of the "asthma exacerbation performance score"
This score uses a scale designed for the purpose of the study, the "Asthma exacerbation performance score", ranging from 0 to 19, 19 being the highest rate meaning that all the actions were correctly performed (1 point per action correctly performed).
This score will allow to calculate the degree of correlation calculated by Kappa coefficient for each action performed or answer provided, between the simulation game and the high fidelity simulation session on the one hand; and between the multiple choice questionnaire and the high fidelity simulation session on the other hand;
Then, we will compare the median Kappa coefficients for each condition, and determine which of the multiple choice questionnaire or the simulation game reflects the best the clinical competence of medical students observed during a high-fidelity simulation session.
Measured: Baseline
Secondary outcomes (2)
The total score on the "asthma exacerbation performance score"
The total scores of this checklist will allow to calculate:
- the degree of correlation of the scores using this checklist between the simulation game and the high-fidelity simulation session using spearman correlation,
- the degree of correlation of the scores using this checklist between the simulation game and the multiple choice questionnaire using spearman correlation
It will also allow to compare the median scores of the students between the three modalities of assessment (high fidelity simulation, simulation game and multiple choice questionnaire), and the dispersion of the scores between the three modalities.
Measured: Baseline
The satisfaction of students using Likert-scales with each evaluation modality
The satisfaction of students with the simulation game, and with the multiple choice questionnaire, will be assessed using a questionnaire with Likert-scales.
Measured: Baseline
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted21 Mar 2019
- Study start20 Mar 2019
- Primary completion6 Jul 2019
- Study completion6 Jul 2019
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (4)Papers the sponsor has linked to this study.
- Other trials for Educational ProblemsStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Other
- Intervention model
- Sequential
- Model description
- All medical students are evaluated using a high-fidelity simulation environment, then using a serious game, then using multiple choice questionnaire, all on the same topic.
- Masking
- None (open label)
- Enrolment
- 42 participants (actual)
Sponsor and investigators
- Lead sponsor
- Ilumens
- Sponsor type
- University, hospital or other
- Responsible party
- Sponsor
- Investigators
- David Drummond, MD, PhD, Principal investigator, Université Paris Descartes
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT03884114
- Sponsor's protocol ID
- Ilumens0003
- Acronym
- SIMEVAL
- Status last verified
- Jul 2020
- First submitted
- 18 Mar 2019
- Last updated
- 20 Jul 2020
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- Yes: * Participants' characteristics anonymized * Participants' correct and incorrect actions and answers when evaluated on the 3 different modalities: multiple choice questionnaire, high-fidelity simulation and simulation game
Publications
Norcini JJ, Swanson DB, Grosso LJ, Webster GD. Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence. Med Educ. 1985 May;19(3):238-47. doi: 10.1111/j.1365-2923.1985.tb01314.x.
PubMed 4010571Drummond D, Delval P, Abdenouri S, Truchot J, Ceccaldi PF, Plaisance P, Hadchouel A, Tesniere A. Serious game versus online course for pretraining medical students before a simulation-based mastery learning course on cardiopulmonary resuscitation: A randomised controlled study. Eur J Anaesthesiol. 2017 Dec;34(12):836-844. doi: 10.1097/EJA.0000000000000675.
PubMed 28731928Adjedj J, Ducrocq G, Bouleti C, Reinhart L, Fabbro E, Elbez Y, Fischer Q, Tesniere A, Feldman L, Varenne O. Medical Student Evaluation With a Serious Game Compared to Multiple Choice Questions Assessment. JMIR Serious Games. 2017 May 16;5(2):e11. doi: 10.2196/games.7033.
PubMed 28512082Fonteneau T, Billion E, Abdoul C, Le S, Hadchouel A, Drummond D. Simulation Game Versus Multiple Choice Questionnaire to Assess the Clinical Competence of Medical Students: Prospective Sequential Trial. J Med Internet Res. 2020 Dec 16;22(12):e23254. doi: 10.2196/23254.
PubMed 33325833
Study at a glance
From the registry record, last updated by the sponsor on 20 Jul 2020. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Assessment of Clinical Competence of Medical Students: a Comparison of High-fidelity Simulation, Serious Game and Multiple-choice Questions
- ClinicalTrials.gov ID
- NCT03884114
- Acronym / sponsor ID
- SIMEVAL / Ilumens0003
- Conditions
- Educational Problems
- Keywords
- technology-enhanced simulation, serious games, multiple choice questionnaires, medical education
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 42 participants (actual)
- Lead sponsor
- IlumensUniversity, hospital or other
- Study start
- 20 Mar 2019 (actual)
- Primary completion
- 6 Jul 2019 (actual)
- Study completion
- 6 Jul 2019 (actual)
- First posted
- 21 Mar 2019 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
This study, focusing on the management of pediatric asthma exacerbations, assesses the clinical skills of medical students using three different evaluation tools: (i) the simulation game "Effic'Asthme" developed to train individuals on the management of pediatric asthma exacerbations; (ii) a multiple choice questionnaire (MCQ) on the same subject developed for the purpose of the study and (iii) high fidelity (HF)-simulation, considered as the gold-standard for its enhanced realism.
Its objective is to determine which of the simulation game or the MCQ reflects the best the clinical competence of medical students evaluated on a HF simulator.
Detailed description
The assessment of medical students' clinical competence has long been of concern to educational institutions. Objective structural clinical examination (OSCE) and assessments in high-fidelity (HF) simulation settings represent interesting evaluation modalities but are associated with huge costs, especially when they are used for large-scale standardized assessments. Multiple choice questionnaires (MCQ) remain the most common evaluation tool in medical schools.
Simulation games may represent an interesting compromise between the cheap but limited assessment allowed by MCQs, and the comprehensive but highly expensive assessment allowed by OSCE and HF simulation.
This study, focusing on the management of pediatric asthma exacerbations, assesses the clinical skills of medical students using three different evaluation tools: (i) the simulation game "Effic'Asthme" developed to train individuals on the management of pediatric asthma exacerbations; (ii) a MCQ on the same subject developed for the purpose of the study and (iii) HF-simulation, considered as the gold-standard for its enhanced realism.
Its objective is to determine which of the simulation game or the MCQ reflects the best the clinical competence of medical students evaluated on a HF simulator.
Study design
- Primary purpose
- Other
- Intervention model
- Sequential
- Masking
- None (open label)
- Number of arms
- 1
- Model description
- All medical students are evaluated using a high-fidelity simulation environment, then using a serious game, then using multiple choice questionnaire, all on the same topic.
| Arm | Type | Receives |
|---|---|---|
| All participants | Other | Other: High-fidelity simulation, Other: Simulation game, Other: Multiple choice questionnaireAll participants are evaluated on a scenario of pediatric asthma exacerbation using three different evaluation tools: (i) the simulation game EfficAsthme developed to train individuals on the management of pediatric asthma exacerbations; (ii) a MCQ on the same subject developed for the purpose of the study and (iii) HF-simulation, considered as the gold-standard for its enhanced realism. |
Who may qualify
- Ages
- 18 years and older
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
Key inclusion criteria (4 listed)
- medical students from Paris 5 University who meet the following criteria
Key exclusion criteria (1 listed)
- refusal to participate
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| High-fidelity simulationThe first evaluation modality studied is high fidelity (HF) simulation. In our study, this modality is considered to be the gold-standard assessment method of clinical assessment, because HF simulation corresponds to the modality which reflects the best clinical competence in hospital settings. The HF pediatric manikin used (SimBaby), is able to reproduce all the signs of an asthma exacerbation (coughing, wheezing, tachypnea, chest indrawing, seesaw respiration, cyanosis). In the simulation room, participants can use the same items that those present in the simulation game (a glass of water, paracetamol, short acting beta-agonist, controller treatment (Fluticasone), an asthma spacer with a facial mask, saline nose drops to perform nasal irrigation, oral steroids in tablets (prednisone), and a phone). | Other | All participants |
| Simulation gameEfficAsthme is a simulation game used on a tablet computer. This simulation game was developed to train parents on the management of asthma exacerbations of their children. For the purpose of the study, EfficAsthme is diverted from its original use to assess students' clinical skills. The training scenario "A polluted atmosphere" is used in this study. Participants need to observe the signs presented by the child, and to determine the severity of the asthma exacerbation. From a menu on the right of the screen, the participant can choose several actions, especially to provide the short acting beta-agonist. | Other | All participants |
| Multiple choice questionnaireThe third evaluation modality corresponds to a multiple choice questionnaire (MCQ) including 15 questions. As for HF-simulation and the simulation game, the MCQ starts with the same briefing and continues with 15 questions regarding the management of a moderate asthma exacerbation. | Other | All participants |
Primary outcome
Items of the "asthma exacerbation performance score"
Time frame: Baseline
This score uses a scale designed for the purpose of the study, the "Asthma exacerbation performance score", ranging from 0 to 19, 19 being the highest rate meaning that all the actions were correctly performed (1 point per action correctly performed). This score will allow to calculate the degree of correlation calculated by Kappa coefficient for each action performed or answer provided, between the simulation game and the high fidelity simulation session on the one hand; and between the multiple choice questionnaire and the high fidelity simulation session on the other hand; Then, we will compare the median Kappa coefficients for each condition, and determine which of the multiple choice questionnaire or the simulation game reflects the best the clinical competence of medical students observed during a high-fidelity simulation session.
2 secondary outcomes
The total score on the "asthma exacerbation performance score"
Time frame: Baseline
The satisfaction of students using Likert-scales with each evaluation modality
Time frame: Baseline
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 20 Jul 2020.
- Background: PubMed 4010571Norcini JJ, Swanson DB, Grosso LJ, Webster GD. Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence. Med Educ. 1985 May;19(3):238-47. doi: 10.1111/j.1365-2923.1985.tb01314.x.
- Background: PubMed 28731928Drummond D, Delval P, Abdenouri S, Truchot J, Ceccaldi PF, Plaisance P, Hadchouel A, Tesniere A. Serious game versus online course for pretraining medical students before a simulation-based mastery learning course on cardiopulmonary resuscitation: A randomised controlled study. Eur J Anaesthesiol. 2017 Dec;34(12):836-844. doi: 10.1097/EJA.0000000000000675.
- Background: PubMed 28512082Adjedj J, Ducrocq G, Bouleti C, Reinhart L, Fabbro E, Elbez Y, Fischer Q, Tesniere A, Feldman L, Varenne O. Medical Student Evaluation With a Serious Game Compared to Multiple Choice Questions Assessment. JMIR Serious Games. 2017 May 16;5(2):e11. doi: 10.2196/games.7033.
- Derived from this study: PubMed 33325833Fonteneau T, Billion E, Abdoul C, Le S, Hadchouel A, Drummond D. Simulation Game Versus Multiple Choice Questionnaire to Assess the Clinical Competence of Medical Students: Prospective Sequential Trial. J Med Internet Res. 2020 Dec 16;22(12):e23254. doi: 10.2196/23254.
- Site showing the simulation game "EfficAsthme" developedLink provided by the sponsor.
- Individual participant dataSharing planned: Yes. * Participants' characteristics anonymized * Participants' correct and incorrect actions and answers when evaluated on the 3 different modalities: multiple choice questionnaire, high-fidelity simulation and simulation game
Related trials for Educational Problems
Not yet recruiting
Standardized Educational Videos of Laparoscopic Gynecological Surgeries
Educational Problems, Surgery
Active, not recruiting
Impact of Surgical Coaching on Resident ObGyn Laparoscopic Skills
Educational Problems
Not yet recruiting
Evaluation of a Couple-based Psycho-socio-educational Intervention Using WeChat/WhatsApp on Dyadic Psychological and Sleep Outcomes
Sleep Problems, Psychological
Active, not recruiting
Fostering Healthy Futures for Teens: An RCT
Child Abuse, Child Neglect
Recruiting
Health Equity and Rural Education (HERE!) Clinical Trial
Behavioral Symptoms, Community Health Workers
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
David Drummond, MD, PhD principal investigator
Université Paris Descartes
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing High-fidelity simulation, Simulation game and Multiple choice questionnaire in people with Educational Problems, run by Ilumens at 1 site in France.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 4 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 20 Jul 2020. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in France. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Items of the "asthma exacerbation performance score" (time frame: Baseline). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT03884114, which also lists 4 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 20 Jul 2020.