Design of an Augmented Reality System by Integration of CT Scan or MRI Data With Endoscopic Images for Video-assisted Endonasal Endoscopic Surgery

The available literature offers numerous expert and consensus opinions supporting the usefulness and acceptance of endonasal navigation and encouraging development of optimised navigation systems Augmented reality (AR) allows the surgeon to visualise the preoperative imaging information in the form of a projection that is displayed on the endoscopic image screen. This application, used in other surgical fields, is promising. The EnCoV team of the Institut Pascal laboratory has notably developed an innovative functional AR system, in laparoscopy in gynaecological, digestive and urological surgery The feasibility in endonasal surgery has been confirmed in a cadaveric model and its clinical utility shown on various systems. However, the question of accuracy during calibration and during the surgery remains.

We propose a surface registration system based on 3D reconstructions of patient's face and endoscopic images. It would match 3D reconstructions of the patient's images with 3D reconstructions of the CT scan or MRI images. Thus, this device would be able to provide a large number of reference points for precise mapping. Tissue recognition (texture, contours, etc.) processes would allow to display the right information in the right place.

To date, there is no such AR endonasal navigation system. Thus, the technical evolutions developed in laparoscopy by the EnCoV team would be transposed the ENT field in order to optimise endonasal endoscopy.

Outline of the study :

Feasibility study, observational, transversal. Feasibility criterion: To evaluate the feasibility of a device based on AR to guide the gesture in endonasal endoscopic surgery by displaying invisible high-risk areas and surgical landmarks in a reliable and accurate way.

1. Development of a questionnaire for AFR and SJORL ENT surgeons in order to assess their needs and the interest of implementing such a navigation
2. Study of the feasibility of integrating, in the laboratory, CT scan images with facial and endonasal images filmed in the operating theatre, to form a qualitative clinical study.

• Patients and data acquisition:
• Data acquisition The images will be collected in the operating theatre of the ENT and Cervico-Facial Surgery department of the Gabriel Montpied University Hospital in Clermont-Ferrand. They will be obtained from videos of the face and endonasal anatomy filmed using the endoscope. All these images will be anonymised.
• Duration of inclusion: 12 months (November 2020- October 2021)
• Number of patients: Intermediate stages with analysis every 5 patients for the first 15 patients and then every 10 patients will be realised, adapting the recording methods and algorithms to the results. This will be a progressive process of adjustment and refinement. The methods for recording patient data in the operating theatre will evolve in order to optimise image processing. The annual recruitment capacity is 50 patients; a maximum number of 30 to 40 patients is expected.
• Written consent: prospective collection of the patient's non-opposition
• Duration of each patient's participation: 1 day
• Data processing: This will be a collaborative work with the EnCoV team under the responsibility of Pr A. BARTOLI. The most relevant images will be extracted, corrected and then transformed into a 3D model in order to acquire depth data. The CT scan images will also be transformed into 3D format. These two 3D models (from the patient and from the imaging) will be merged using an algorithm, in order to allow surface registration. The data will first be stored on the EnCoV servers.

1. Validation of the navigation system by experimenting it on phantom models made from 3D printing and ex vivo animal models, to form a quantitative preclinical study. In endonasal endoscopy procedures the validity of the display of easily recognisable structures will be checked.