Assessment of Hysteroscopy Skills With a New Assessment Tool

Abstract:

Objective: To investigate validity of the hysteroscopy assessment tool (HYSAT) for assessment of competence in a clinical environment.

Methods: Novices and experienced gynecologists are recruited from three hospitals and observed and assessed while performing hysteroscopy. Performances are assessed using the HYSAT tool by two independent raters. Validity evidence is gathered in accordance with Messick's framework: validity evidence for content was ensured in a prior Delphi study, response process is ensured by standardization of rater instructions. Internal structure is explored using Cronbach's alpha for internal consistency reliability; inter-rater reliability and test-retest reliability are calculated as Pearson's r independently across all ratings. Relationship to other variables is investigated by comparing performances of the participants in each group. Consequences evidence is explored by calculating a pass/fail standard using the contrasting groups' standard setting method.

Introduction Hysteroscopy is an essential tool in gynecology for diagnosing and treating intrauterine pathology. The increasing use of hysteroscopy under local rather than general anesthesia has heightened both technical and communicative demands, as patient comfort is closely linked to procedural efficiency and operator skill. While complications rates are low among experienced clinicians, inadequate proficiency may compromise safety and lead to prolonged, incomplete, or painful procedures particularly in awake patients.

Training in hysteroscopy is typically based on supervised clinical exposure within suggested curricula, yet the apprenticeship model is limited by variability in case volume and lack of standardized assessment.

Current educational paradigms emphasize mastery-based learning, requiring trainees to meet predefined proficiency standards supported by assessment tools with validity evidence. However, existing literature highlights heterogeneity in training and assessment methods, underscoring the need for standardized, evidence-based approaches.

A structured, consensus-based assessment tool for hysteroscopy (HYSAT) has recently been developed by an international panel of experts through a three round Delphi process (KHA et al.). The instrument encompasses key domains of procedural competence.

Prior to clinical implementation, it is essential to gather validity evidence for HYSAT in a real-life setting. Unreliable interpretation of assessment outcomes may lead to under- or overestimation of trainee competence with potential consequences for both training progression and patient safety. Therefore, the application of HYSAT must be supported by robust validity evidence to ensure that score interpretations are meaningful and that derived decisions are justified.

Aims We aim to investigate the validity of the Hysteroscopy Assessment (HYSAT) tool for assessment of competence in a clinical environment.

• To establish an international multi-site validity investigation to show if HYSAT can be used to assess procedural competence in hysteroscopy.
• To gather validity evidence according to Messick's framework validity from the following sources: response process, internal structure, relationship to other variables, and consequences evidence.
• To investigate if HYSAT can be used for both formative and summative assessment, and to establish a credible pass/fail score to help inform decision-making for certification.