Symbiotic Registration and Deep Learning for Retinal Image Analysis

Li Ding

Supervised by Professor Gaurav Sharma

Join Zoom Meeting
https://rochester.zoom.us/j/98676431500

Geometry and semantics are two sub-fields in computer vision that have been re- searched extensively as two separate problems over decades. However, the semantics and the geometry in computer vision are not mutually exclusive and techniques developed for one could complement the other. Unfortunately, the interplay of these two fields has received limited attentions. In this thesis, we design symbiotic geometric and semantic computer vision methods in the specific context of retinal image analysis. In particular, we consider the semantic problem of retinal vessel detection and the geometric problem of retinal image registration.

First, we propose a novel pipeline for vessel detection in fluorescein angiography (FA) using deep neural networks (DNNs) that reduces the effort required for labeling ground truth data by combining cross-modality transfer and human-in-the-loop learning. The cross-modality transfer exploits concurrently captured color fundus (CF) and fundus FA. Binary vessels maps detected from CF images with a pre-trained network are geometrically registered with FA images via robust parametric chamfer alignment. Using the transferred vessels as initial ground truth labels, the human-in-the-loop approach progressively improves the ground truth labeling by iterating between deep-learning and labeling. Experiments show that the proposed pipeline significantly reduces the annotation effort and outperforms prior FA vessel detection methods by a significant margin.

Next, we describe an annotation-efficient deep learning framework for vessel detection in UWF fundus photography (FP) that does not require de novo labeled UWF FP vessel maps. Our approach uses concurrently captured UWF FA and iterates be- tween a multi-modal registration step and a weakly-supervised learning step. In the registration step, UWF FA vessel maps detected with a pre-trained DNN are registered with the UWF FP via parametric chamfer alignment. The warped vessel maps are used as the tentative training data but inevitably contain incorrect labels due to the differences between two modalities and the errors in the registration. In the learning step, a robust learning method is proposed to train DNNs with noisy labels. The registration and the vessel detection benefit from each other and are progressively improved. Results on two datasets shows that the proposed approach provides accurate vessel detection, without requiring manually labeled UWF FP training data. Finally, we present a hybrid framework for registering retinal images in the presence of extreme geometric distortions that are commonly encountered in UWF FA. Our approach consists of a feature-based global registration and a vessel-based local refinement. For the global registration, we introduce a modified RANSAC algorithm that jointly identifies corresponding key points and estimates a polynomial geometric transformation consistent with the identified correspondences between reference and target images. Our RANSAC modification particularly improves feature matching and the registration in peripheral regions that are most severely impacted by the geometric distortions. The local refinement is formulated as a parametric chamfer alignment for vessel maps obtained using DNNs. Because the complete vessel maps contribute to the chamfer alignment, this approach not only improves registration accuracy but also aligns with clinical practice, where vessels are typically a key focus of examinations. Experiments conducted on two datasets show that the proposed framework significantly outperforms the existing retinal image registration methods.