Validity of Experts and ScanNav Anatomy PNB When Identifying Sono-anatomical Structures for Ultrasound-Guided Regional Anaesthesia

Ethical Approval:

The ultrasound scans/still frame images used in this study have been acquired from an earlier study conducted at Oregon Health \& Science University (OHSU) (ClinicalTrials.gov ref: NCT04906018). OHSU Institutional Review Board approval (STUDY00022920) was granted for collection of the ultrasound scans.

Ethical approval for the assessment of these scans by regional anaesthesia experts, has been obtained from Oxford University Research Ethics Committee for Medical Sciences (R75449/RE001).

Block Areas Included:

A Delphi study was undertaken earlier in the research programme: 'International Consensus on the Minimum Anatomical Structures to Identify on Ultrasound for the Performance of Plan A Blocks in Ultrasound-Guided Regional Anaesthesia: A Regional Anaesthesia UK Delphi Study' (work currently unpublished).

As ScanNav Anatomy PNB has undergone development and validation work for all of these blocks apart from the femoral nerve block, the remaining six Plan A block regions will be considered for this project. These regions (and structures assessed) are:

• Interscalene level brachial plexus (Structures: anterior scalene, middle scalene, C5 nerve root, C6 nerve root)
• Axillary level brachial plexus (Structures: axillary artery, axillary vein, conjoint tendon, median nerve, musculocutaneous nerve, radial nerve, ulnar nerve)
• Erector spinae plane (Structures: transverse process, erector spinae muscle group)
• Rectus sheath plane (Structures: rectus abdominis, rectus sheath (anterior layer), rectus sheath (posterior layer), peritoneum)
• Adductor canal (Structures: femoral artery, sartorius, saphenous nerve/nerve complex)
• Popliteal level sciatic nerve (Structures: sciatic nerve)

Standardisation of Ultrasound Scans used for Assessment:

The ultrasound videos and still frames will undergo review during the earlier (OHSU) study, including the following:

1. Does this ultrasound scan obtain the correct view for this block? [Yes/no]
2. Does the video or still frame contain any atypical anatomy? [Yes/no]

The answer for question 1 must be 'Yes' and for question 2 must be 'No' for the scan to be included in the present study.

Selecting Ultrasound Scans for Assessment:

Of the acceptable ultrasound scans, five will be selected (at random) for each region. This will be done for each of the six block regions, providing a total of 30 scans/still frame images overall. A suitable still frame in each scan sequence will be selected by the investigators and saved as an image file in the Portable Network Graphic (PNG) format for later segmentation.

Expert Reviewers:

Approximately 15 UGRA experts will be recruited from centres in the UK. One-three will be recruited from each of seven centres:

• Aneurin Bevan University Health Board
• Guy's \& St Thomas' NHS Foundation Trust
• NHS Greater Glasgow \& Clyde
• NHS Tayside
• Oxford University Hospitals NHS Foundation Trust
• Imperial College Healthcare NHS Trust
• Royal Cornwall Hospitals NHS Trust

All experts must hold a UK-recognised qualification for independent practice in anaesthesia (i.e., be qualified to perform UGRA independently and supervise others) and be a member of a relevant professional society (e.g., RA-UK, ESRA, ASRA). They must also meet at least three elements of the following criteria:

• Completed advanced training in UGRA or have >10 years of independent practice in UGRA
• Hold a qualification related to UGRA (e.g., EDRA, higher degree or equivalent)
• Regularly delivers direct clinical care using UGRA, including for 'awake' surgery where indicated
• Regularly teaches UGRA in the course of their clinical work, including advanced techniques (Plan B/C/D blocks) where indicated

Defining the Ground Truth:

The UGRA experts will all view each video in turn, on a tablet computer. At the end of each one they will be asked to mark the boundaries of the relevant anatomical structures (listed above) for the corresponding still frame image using the tablet and a stylus.

There are 21 anatomical structures across all the regions assess. Each region will represented by five ultrasound scans. Thus, 105 structures (21 x 5) will each be assessed by around 15 experts. The expert segmentation masks (for each annotated structure) will be processed in the following manner to derive an average mask for each structure:

• Each segmentation mask, ms, for a given structure will be segmented such that ms = f(m) = {1 if mask present 0 if mask not present
• For each pixel, ps, the frames will be summed across all n segmentation masks
• This will give a final mask, mf, where mf = { p : 0 ≤ p ≤ n }
• The mf will then be filtered with a threshold such that mt = f2(mt) = { 1 if pf > t n 0 otherwise

In the above description, ms represents a segmentation mask produced by each expert. Areas enclosed by an expert's segmentation outline will be allocated a numerical value of 1 (0 if not enclosed by an area of segmentation). Each pixel in the final mask (mf) will be summed across all (n) the segmentation masks of the experts, to give a final value of 0 - n. At least 75% of experts must enclose the pixel in their individual segmentation mask for the pixel to be included in the overall ground truth. ≥75% has been chosen to be consistent with the strong recommendations made in the earlier Delphi study. Lastly, mf will then be filtered at this threshold to produce the ground truth mask (mt). The algorithm defined above will be run using Python2 in jupyter-lab ('Project Jupyter').

Variability in Expert Analysis:

The correlation in the expert markings will be compared using the Dice/Intersection over Union (IoU) and Hausdorff co-efficients. Dice/IoU will be used for all structures except those identified by a non-enclosed area, where the Hausdorff co-efficient will be used. These structures are:

• Conjoint tendon
• Transverse process
• Rectus sheath (anterior layer)
• Rectus sheath (posterior layer)
• Peritoneum

Comparison to ScanNav Anatomy PNB:

The ground truth will provide an expert 'gold standard' against which ScanNav Anatomy PNB will be compared. The level of correlation of ScanNav Anatomy PNB highlighting will be compared to the expert ground truth in the same statistical manner as above