Early Prediction of Diabetic Retinopathy using a Multimodal Deep Learning Framework Integrating Neural Network

Abstract

Diabetic retinopathy (DR) is one of the most common diabetes-related causes of preventable vision loss that is equally important in diabetes management for proper diagnosis and treatment at an early stage to save one's vision. This study introduces a combined multimodal deep learning approach for early detection of DR using the fusion of retinal fundus images and structured clinical data. The framework uses an artificial neural network with the feature extraction ability of convolutional network EfficientNet-B3 to process clinical variables, such as age, duration of diabetes, glycated hemoglobin (HbA1c), fasting blood glucose, blood pressure, body mass index, cholesterol level, and family history. The extracted visual and clinical features are integrated in a multimodal feature integration layer to further enhance classification accuracy. Empirical results indicate high accuracy (97.20 %), precision (96.90 %), recall (96.50 %), specificity (97.80 %), F1 score (96.70 %), as well as area under the ROC curve (AUC = 0.989), outperforming conventional image-based deep learning models. The results suggest that multimodal learning has a significant effect on the prediction of early DR and can act as an efficient clinical decision support system for artificial intelligence-based large-scale screening and prompt intervention.