Clinical and Genetic Analysis of ROP

Description

This NIH funded multi-center study began July 2011 with 8 study sites approved by their individual IRBs. Recruitment and data was conducted at the following sites: OHSU, Columbia Universtiy, Cornell College, William Beaumont Hospital, Children's Hospital LA, University of Miami, University of Illinois Chicago, Cedars Sinai Medical Center and Asociacion para Evitar la Ceguera (APEC) in Mexico City. For the competitive renewal of the grant which begins 6-01-20, the recruitment sites have been reduced to 5 which include OHSU, William Beaumont, University of Illinois Chicago, University of Utah and Stanford University.

This study will aim to develop a quantitative framework for ROP care using artificial intelligence and analytics to improve clinical disease management. The investigators will evaluate performance of an artificial intelligence system for ROP diagnosis and screening prospectively. This will include: (a) recruit a target of over 2000 eye exams including wide-angle retinal images from 375 subjects at 5 centers, (b) optimize an image quality detection algorithm the investigators have recently developed, and (c) analyze system accuracy for ROP diagnosis (plus vs. pre-plus vs. normal) and screening (using a novel quantitative vascular severity scale).

The proposed work will study infants who will receive routine ophthalmoscopic exams and have retinal images taken at each exam according to the standard of care at each institution. At least one person at each site is trained to capture wide-angle retinal images using a commercially-available camera (RetCam; Natus, Pleasanton, CA). This device is FDA-cleared for premature infants, and has been used throughout the world for 20 years with no known complications.

All participating infants will undergo retinal photography by trained study personnel for up to 3 eye exams, or more if clinically indicated and feasible. "Outborn infants," who were transferred to the study center for specialized ROP care, will have at least one set of images taken if this is clinically indicated and feasible. These coded retinal images will be read and interpreted by remote expert graders using the secure web-based system developed for this study 9 years ago at OHSU. The de-identified images will be housed indefinitely in an OHSU IRB-approved repository for possible future research studies or for other educational purposes.

Most infants recruited from the first 9 years of this study between July 2011 and May 2020 had DNA collected from blood or saliva samples. The coded genetic samples are housed in an OHSU IRB-approved repository and will be analyzed by outside collaborators for specific aim 3 of this study. Note that this current study does not involve new collection of any blood or saliva samples.

In recent years, our team has successfully developed competitive image assessment methods to infer ROP status using (i) engineered image features based on translating descriptive and visual descriptions related to expert assessment . and (ii) deep-learned features based on end-to-end training of neural networks for image analysis. To improve model explainability, the models must not only provide classification (diagnostic) labels or severity scores, but also supplementary information regarding how a model produces its decisions and what about a particular image drives the decision. To this end, it is helpful for a model to (i) visualize its training data; and (ii) illustrate which features of the input image its decision relied heavily on. Visualization can provide an overarching demonstration of how a model produces its decisions across a dataset with known clinical and demographic characteristics, which contributes to overall interpretability of the model's logic. Illustration is essential to gain trust and to facilitate validation when clinicians rely on the model to assess a particular instance for various purposes, including training or regulatory approval.

While training the AI system in previous work, the investigators excluded 5% of images that were rated by the majority of graders as "not acceptable quality". For real-world use, it will be important to balance imageability with diagnostic performance. The investigators propose to evaluate our existing dataset to determine the optimal operating point in the CNN quality algorithm that balances imageability with diagnostic performance of the i-ROP DL classifier. The investigators will continue those studies to systematically examine their impact on improving image quality and diagnostic performance - and maximize rigor and reproducibility of study design. This operating point will then be "locked", and the closed system will be used as below.

Prospective evaluation of i-ROP DL classifier: The investigators propose to calculate the weighted kappa between the RSD and the i-ROP DL system, along with sensitivity, specificity, and imageability based on the optimal operating points identified above.

Prospective evaluation of vascular severity score: In a cross sectional analysis, the investigators will test the hypothesis that the ROP vascular severity score, derived from the i-ROP DL classifier, may demonstrate high sensitivity both for detection of plus disease and for identifying treatment-requiring disease in a real-world ROP screening population.

Details