Computer-based Screening for Diabetic Retinopathy

Sensitivity of retinal screening for risk stratification of DR using digital fundus photographs with and without the assistance of the RiskAnalyzer

Sensitivity and specificity of retinal screening for risk stratification of DR using digital fundus photographs and the RiskAnalyzer

Clinical Trial Methods and Research Design: A total of 10,000 subjects will be recruited to participate in this study. These subjects will be enrolled through two parallel collaborators, Extension for Community Healthcare Outcomes (Project ECHO) and CommuniCare Health Centers. Each clinic will obtain approximately 50% of the cases per year or an average of 50 cases per week. Subjects will be identified from clinics, CommuniCare Health Centers of San Antonio, TX and Project ECHO, New Mexico. CommuniCare has two associated sites, the Barrio Family Health Center in Southeast San Antonio, TX and the Dr. Frank Bryant Health Center also in San Antonio. CommuniCare serves approximately 45,000 subjects a year of which it is estimated that 5000 are requiring diabetic care. Project ECHO in conjunction with the University of New Mexico collaborates with over 20 rural clinics in New Mexico to provide service to underserved areas in New Mexico. A recruitment flyer and consent forms will be provided to each site. Clinic staff will recruit subject for the study during scheduled visits. Subject will be given a flyer containing information about the study and the informed consent document. Subjects who are interested in being enrolled in the study will be referred to a photographer within the site for Retinal imaging. Subjects will not receive a payment or stipend for participation in this study.

There are no screening requirements for subjects to participate in this trial. Subjects must be diagnosed as a person with pre-diabetes or diabetes by a medical professional. There is no requirement or solicitation for information regarding prior or concomitant therapy. There are no criteria set forth for subjects to exit or be discontinued from the study once the retinal screening is performed. Subjects will not be withdrawn from the study once retinal screening has occurred. Once the subject completes retinal screening, the period of subject participation is concluded. Images taken from subjects may not be used in the study due to inadequate image quality. The number of images found to be inadequate will be tracked and reported at the completion of the study. Retinal screening marks both the beginning and the end of subject participation in this study.

For each eye, a minimum of two posterior pole images will be collected. One image will be centered on the fovea and the second on the optic disc. A third image will be collected of the anterior segment (front of the eye). Subjects will not be pharmacologically dilated. Imaging will rely on natural dilation due to adaptation to darkness in the imaging room. More than two images per eye will be collected only when the imager (photographer) identifies any given image of insufficient quality for reading. Poor quality images are typically due to physiological reasons, such as blinking or small pupils and will not be used as part of this study. The retinal photographer will attempt up to 3 times to acquire images of a subject's eyes in order to complete the imaging protocol. The study coordinator will conduct weekly audits of the imaging procedure in order to ensure and document compliance with the study protocol.

The Class II Medical Device that will be used for this study will be a non mydriatic or equivalent retinal camera. The retinal camera is intended to be used for taking digital images of the retina of the human eye without the use of pharmacological dilation. The retinal camera has the following photography modes: RGB color. A Digital Camera is mounted to the retinal camera to perform retinal imaging.

A picture archival and communication system (PACS) is used to receive DICOM images, scheduling information and textual reports, organize and store them in an internal format, and to make that information available across a network via HIPAA and FDA-cleared secure user interfaces. The PACS is comprised of a capture component, a central server component, and a viewer component that is used on general purpose computing hardware. Subject information, procedure information, and image acquisition data is captured, sent to a server or alternatively, it is accepted directly from a DICOM compliant device. After receiving image objects and associated data, the server registers the incoming data and images with the archive including any associations with prior cases. The cases are presented to users with the correct permissions from a work list as controlled by a managing user. The image data is transmitted and rendered on the user's workstation using a standard web browser. Once the server registers a report, the report is available to the referring physician or client service electronically or via fax.

Immediately after the images and Subject demographics are collected, the photographer will upload the data as a case to the PACS via a VPN connection established between the imaging workstation and PACS. If the VPN connection cannot be established at the time of imaging, the upload will be done at a later time, e.g. end of the clinic day. The data will reside in the imaging workstation for up to five days in order to assure upload. The clinical coordinator will conduct weekly audits of upload completion. Once in PACS, cases are read by readers using a built in reading template that generates a report with their findings. The study will not disrupt standard of care to the study subjects. Each site will follow its own standard procedures and care. Findings will be reported to the clinic which referred the subject for screening. Clinics will be responsible for follow-up care including any necessary intervention or therapy.

Performance of the RiskAnalyzer: The performance of the RiskAnalyzer will be calculated by applying the minimally acceptable values for sensitivity (Seo) and specificity (Spo) values specified in advance. For purposes of this study, the specified null hypotheses values are Seo=0.85 and Spo=0.70. These values are based on data obtained from ophthalmic exams. The efficacy of trained readers using retinal photography has been demonstrated to provide the most sensitive screening and monitoring test for sight-threatening retinopathy, with sensitivities > 80%. Abramoff, et al. discuss the limits of sensitivity that can be achieved by any system given the fallibility of ground truth based on multiple human readers. He suggests that 0.85 is the limit of sensitivity that one can achieve. The specificity is based on the efficiency that we seek in order to make this implementation commercially viable. We have designed our study to achieve 0.90 statistical power and 0.99 significance level. If confidence limits are based on asymptotic normal distribution theory, then sample sizes can be based on the asymptotic variance formulae, as given by Pepe.

Objective One: Objective one is to test fully the study data flow logistics and information transfer interfaces. The testing will be done entirely at one recruitment, enrollment and imaging site. The duration of time to complete this objective is three months.

Materials and Research Design for Objective One: Subjects will be recruited from single site. All images will be routed to two readers for grading. If the two readers assign equivalent grades to the image, a generated report will be given to the clinic to distribute the referring physician. For cases in which the readers assign different grades for the same image, the image will be routed to an adjudicator for resolution. The adjudicator will have access to the reports generated by each reader, but not the risk levels from the RiskAnalyzer. A report from the adjudicator will be generated and reported to the clinic that will be responsible for distributing the report to the Subject and for any further follow-up. Images will also be de-identified and routed to the RiskAnalyzer. The RiskAnalyzer will perform its analysis of the image and generate a risk stratification of either "Low risk" or "High risk" for diabetic retinopathy. A report containing the risk levels assigned by the RiskAnalyzer will not be distributed to the referring physician, clinic or adjudicator but used only for statistical analysis in support of the endpoints of the trial.

Outcomes for Objective One: Data outcomes will demonstrate the ability to recruit and enroll subject into the study at a single site, perform retinal imaging, upload the images to a PACS, have the same images graded by two different readers, when necessary routed and graded by an adjudicator and the ability to generate a final report. The reading process followed by the readers is based on the International Clinical Diabetic Retinopathy Disease Severity Scale. Additionally data will demonstrate that images are successfully routed to the RiskAnalyzer for processing.

Objective Two: Objective two is to evaluate the efficacy of the RiskAnalyzer to identify the risk of Diabetic Retinopathy in comparison to the gold standard. The duration of time to complete this objective is nine months.

Methods and Research Design for Objective Two: Subjects are recruited from multiple CommuniCare and Project ECHO sites. All images are graded by two readers and an adjudicator should the grading by the first and second reader not be in agreement. The reading process followed by the readers is based on the International Clinical Diabetic Retinopathy Disease Severity Scale. A consensus report generated by the readers or the adjudicator indicating presence and severity of diabetic retinopathy will be given to the Subject through their referring physician and referred to a specialist if necessary by the referring physician. The same images graded by the readers will also be routed to the RiskAnalyzer for processing which will generate a risk level of either "Low risk" or "High risk" for diabetic retinopathy. As the images are routed to the RiskAnalyzer, the images will be de-identified and sequestered until evaluated by the RiskAnalyzer. Risk levels obtained by use of the RiskAnalyzer will not be given to the Subject under any circumstances. The referring physician is also masked from the risk levels assigned by the RiskAnalyzer. Referring physicians are not co-located at VisionQuest Biomedical and therefore will not have any access to the computers or data storage area for the data.

Outcomes for Objective Two: Statistical analysis will indicate the accuracy level of the RiskAnalyzer as compared to the gold standard. Statistical analysis will use sensitivity, specificity, AUC, and a level of agreement using Cohen's kappa coefficients.

Objective Three: Objective three will address the primary endpoint of the trial which is to demonstrate that the reader's accuracy in grading images is enhanced when risk levels from the algorithm are made available to the reader while performing the grading of the images. The duration of time to complete this objective is 12 months.

Methods and Research Design for Objective Three: The RiskAnalyzer will be used as a first reader. There will continue to be two human readers and an adjudicator. Ground Truth will be established by either the agreement of reader one with reader two or by an adjudicator. All images will be routed to the RiskAnalyzer which will generate a risk level of either "Low risk" or "High risk" for diabetic retinopathy. The cases will be divided into two sets and routed to both readers. One set of images will be routed to both readers with the algorithm assigned risk levels while the other set of images will be routed to both readers without the algorithm assigned risk level. An equivalent number of 'Low risk" and "High risk" cases will be selected for each set of images. Ground Truth is established by the two readers assigning the same grade to the same image. When different grades are assigned to the same image by two different readers, the cases will be routed to an adjudicator for grading and reporting. The adjudication will become ground truth. The cases that will be adjudicated will only be those that are different categories of risk stages. Adjudication will be based on the clinical significance of the disagreement between the readers as follows: cases within severity scales 0 and 1 will be classified as "Low Risk" cases, while cases within the severity scales 2 and above will be classified as "High Risk" cases. Cases needing adjudication will be those differing in risk and those within the High Risk Category that differ between scales 2 and 3 (mild and severe), and 2 and 4 (mild and proliferative Diabetic Retinopathy -PDR).

Outcomes for Objective Three: The outcomes from objective three will establish that the RiskAnalyzer improves the reading accuracy and consistency between readers and decreases bias between human readers. Statistical Analysis of level of agreement will be performed on the risk assigned to the images by the readers when having and not having the risk levels assigned by the algorithm using Cohen's kappa coefficients.

Expected outcomes: Telemedicine retinal screening in conjunction with risk levels from the RiskAnalyzer improve the accuracy of readers when grading images. The use of risk levels assigned by the RiskAnalyzer in assisting the readers will decrease the amount of time readers dedicate to the grading of each case. This may be a model to support the increase of access to retinal screening, support compliance with retinal screening recommendations and facilitate a more efficient use of the retinal specialist to evaluate and treat disease leading to a more cost efficient method of care.