Long-terms Effects of Simulation-based Ultrasound Training: A Randomized Trial

Ultrasonography has become increasingly used in many medical specialties over the last decades as smaller and less expensive ultrasound equipment has become available. Although ultrasound imaging traditionally is considered safe, its use is highly operator dependent (EFSUMB 2010). The lack of sufficient operator skills can lead to diagnostic errors that eventually compromise patient safety due to unnecessary tests or interventions (Moore \& Copel 2011). Clinical ultrasound training is challenged with long learning curves and is therefore both time consuming and requires large teacher resources (Jang et al. 2010). Some trainees may therefore never acquire the basic skills and knowledge needed for independent practice (Tolsgaard et al. 2012). In the search for effective training methods, simulation-based training has proven to be an effective adjunct to clinical training, which has been documented in several studies on interventional procedures (Larsen et al. 2009, Zendejas et al 2011, Barsuk et al. 2009). Most comparative studies on simulation-based training have, however, only examined the initial effects of simulation-based training on the first couple of procedures, it is not known if VR simulation training provides sustained improvements trainee performance throughout their learning curves until they reach clinical proficiency. Large resources are currently being allocated to simulation training in many different specialties but the effects of simulation may easily be over-estimated if only evaluating the immediate effects of simulation training. A small difference in the number of supervised ultrasound examinations needed for proficiency may not justify the financial costs associated with investing in simulation equipment, space for simulation training as well as teacher resources during simulation training. Therefore, the research questions of this study was:

In a group of ultrasound novices, what are the implications of simulation-based training compared to clinical training alone on the technical quality of scans performed after two months of training?

Methods

Study design This study is reported according to the CONSORT statement (www.consort-statement.org). A randomized observer-blind study to compare VR ultrasound training with traditional supervised practice on subsequent patient scans (see Figure 1 for flowchart of study design). Ethical approval was obtained from the Regional Ethical Committee of the Capital Region, Denmark. The Danish Data Protection Agency approved the storing of patient relevant information. This randomized trial is reported to clinicaltrials.gov prior to inclusion of participants.

Setting This study is carried out at the Departments of Gynecology at the Juliane Marie Centre, Copenhagen University Hospital, Denmark, Næstved Hospital, and Hillerød Hospital.

Participants Participants are all new OB/GYN residents at three different hospitals in Eastern Denmark. Inclusion criteria are 1) proficiency in written and oral Danish, 2) informed written consent. The exclusion criteria are 1) prior employment at an OB/GYN department in a clinical position, 2) any formal ultrasound training with or without hands-on practice and (3) prior experience with virtual reality simulation.

Randomization A research fellow at The Center for Clinical Education, Rigshospitalet, independently performs randomisation of participants by computer to either intervention group (simulation training) or control group (clinical training only). The allocation of participants is concealed to the primary investigator during the enrolment of participants. Once informed consent has been signed and a participant number has been provided, the allocation sequence is retrieved per telephone.

Intervention Participants in the intervention group receive simulation training using two types of ultrasound simulators (see Figure 2 for flowchart of the simulation programme). Participants are introduced to a high-fidelity Virtual-Reality (VR) simulator (Scantrainer, Medaphor). The VR simulator provides images obtained from real patients and haptic feedback from the ultrasound probe. Participants train on this type of simulator until a predefined proficiency-level has been attained. The selection of modules is based on a previous validation study and only modules and items that discriminate between novice and expert performances are included in the training programme (Madsen et al. 2013). All participants are provided feedback after completing each module. When all modules are passed on the VR simulator, the participants receive proficiency-based training training on the low-fidelity simulator (BluePhantom) to allow participants to review the functions, they just trained, using real ultrasound equipment. The simulator performances are assessed using the Objective Structured Examination of Ultrasound Skills (OSAUS) and feedback is provided using this framework after each trial. Proficiency is set according pass/fail-scores established in a previous study (Tolsgaard et al. 2013).

Clinical training Participants in both groups receive traditional clinical training. This includes supervised practice in terms of apprenticeship learning followed by supervision during independent practice. The new residents do not usually go on calls for the first one to two weeks. Local department rules dictates which types of examinations that always requires a second examination by a supervising clinicians (e.g. fetal demise or pregnancy of unknown location).

Main outcome measure The main outcome measure is the technical proficiency after two months of practice. Two scans from the emergency department are recorded and subsequently assessed by a blinded clinician, who also had access to the ultrasound descriptions that corresponded to each scan. The assessments were performed using the Objective Structured Assessment of Ultrasound Skills (Tolsgaard et al. 2013), which has been validated in a construct validity study including 30 clinicians with different levels of experience prior to this trial.

Sample size calculation Sample size calculations are based on previous studies on clinical performances of ultrasound novices with the performance of medical students, who completed a simulation-based training programme (Tolsgaard et al. 2013). The average difference in OSAUS scores between these groups was 0.85 (2.65-1.8 (mean SD 0.4). It was anticipated that these differences would be diluted according to literature from other areas of simulation-based technical skills training (Smith et al. 2010), where simulation effects persisted after 3 months of training but the differences between groups was diminished to 55% of the initial effects. Assuming a 55% dilution, an alpha-level of 0.05 and a power of 0.80, the total number of participants needed was 26.