Mobile Implementation of Retinal Image Analysis for Efficient Vessel, Optic Disc, and Lesion Detection

Mubdiul Hossain - Multimedia University, Selangor, Malaysia
Aziah Ali - Multimedia University, Selangor, Malaysia
Noramiza Hashim - Multimedia University, Selangor, Malaysia
Wan Noorshahida Mohd Isa - Multimedia University, Selangor, Malaysia
Wan Mimi Diyana Wan Zaki - Universiti Kebangsaan Malaysia, Selangor, Malaysia
Aini Hussain - Universiti Kebangsaan Malaysia, Selangor, Malaysia


Citation Format:



DOI: http://dx.doi.org/10.30630/joiv.7.3-2.2363

Abstract


Smartphone-based mobile fundus photography is gaining popularity due to the rise of handheld fundus lenses, allowing a portable solution for a mobile-based computer-assisted diagnostic system (CADS). With such a system, professionals can monitor and diagnose numerous retinal diseases, including diabetic retinopathy (DR), glaucoma, age-related macular degeneration, etc. on their smartphone devices. In this study, we proposed a unified CADS tool for smartphone devices that can detect and identify six crucial retinal features utilizing both a filtering approach and a deep learning (DL) approach. These features are retinal blood vessels (RBV), optic discs (OD), hemorrhages (HM), microaneurysm (MA), hard exudates (HE), and soft exudates (SE). Traditional filtering is applied for RBV segmentation using B-COSFIRE and Frangi filter, whereas vessel inpainting and automatic canny edge-based Hough transform are used to localize OD center and radius. The DR lesions (HM, MA, HE, OD segmentation, and SE) are detected using a transfer learning-based Resnet50 backbone and multiclass DL U-net architecture. RBV segmentation achieved 94.94% and 94.44% accuracy in the DRIVE and STARE datasets. OD localization achieved an accuracy of 99.60% in the MESSIDOR dataset. Lastly, the IDRiD dataset is used to train and validate the DR lesions with an overall accuracy of 99.7%, F1-score of 77.4, and mean IoU of 59.2. The smartphone application can perform all the segmentation tasks at once in an average of 30 seconds. Given the availability, it is possible to improve the accuracy of the DL method further by training with more mobile fundus datasets.


Keywords


mobile application; retinal image analysis; fundus image; diabetic retinopathy; lesion detection; vessel segmentation; optic disc localization

Full Text:

PDF

References


K. H. Nguyen, B. C. Patel, and P. Tadi, “Anatomy, Head and Neck: Eye Retina,†StatPearls, Aug. 2022, Accessed: Apr. 26, 2023. [Online]. Available: https://www.ncbi.nlm.nih.gov/books/NBK542332/

S. D. Candrilli, K. L. Davis, H. J. Kan, M. A. Lucero, and M. D. Rousculp, “Prevalence and the associated burden of illness of symptoms of diabetic peripheral neuropathy and diabetic retinopathy,†J Diabetes Complications, vol. 21, no. 5, pp. 306–314, Sep. 2007, doi: 10.1016/J.JDIACOMP.2006.08.002.

R. F. GARIANO and C.-H. KIM, “Evaluation and Management of Suspected Retinal Detachment,†Am Fam Physician, vol. 69, no. 7, pp. 1691–1699, Apr. 2004, Accessed: Apr. 26, 2023. [Online]. Available: https://www.aafp.org/pubs/afp/issues/2004/0401/p1691.html

L. Quaranta, I. Riva, C. Gerardi, F. Oddone, I. Floriano, and A. G. P. Konstas, “Quality of Life in Glaucoma: A Review of the Literature,†Advances in Therapy 2016 33:6, vol. 33, no. 6, pp. 959–981, Apr. 2016, doi: 10.1007/S12325-016-0333-6.

M. K. Ikram, J. C. M. Witteman, J. R. Vingerling, M. M. B. Breteler, A. Hofman, and P. T. V. M. De Jong, “Retinal Vessel Diameters and Risk of Hypertension,†Hypertension, vol. 47, no. 2, pp. 189–194, Feb. 2006, doi: 10.1161/01.HYP.0000199104.61945.33.

M. B. Sasongko et al., “Retinal arteriolar tortuosity is associated with retinopathy and early kidney dysfunction in type 1 diabetes,†Am J Ophthalmol, vol. 153, no. 1, 2012, doi: 10.1016/J.AJO.2011.06.005.

A. Chopdar, U. Chakravarthy, and D. Verma, “Age related macular degeneration,†BMJ, vol. 326, no. 7387, pp. 485–488, Mar. 2003, doi: 10.1136/BMJ.326.7387.485.

K. Mittal and V. M. A. Rajam, “Computerized retinal image analysis - a survey,†Multimed Tools Appl, vol. 79, no. 31–32, pp. 22389–22421, Aug. 2020, doi: 10.1007/S11042-020-09041-Y/TABLES/7.

A. Almazroa, W. Sun, S. Alodhayb, K. Raahemifar, and V. Lakshminarayanan, “Optic disc segmentation for glaucoma screening system using fundus images,†Clin Ophthalmol, vol. 11, p. 2017, Nov. 2017, doi: 10.2147/OPTH.S140061.

A. Grzybowski et al., “Artificial intelligence for diabetic retinopathy screening: a review,†Eye 2019 34:3, vol. 34, no. 3, pp. 451–460, Sep. 2019, doi: 10.1038/s41433-019-0566-0.

U. Iqbal, “Smartphone fundus photography: a narrative review,†Int J Retina Vitreous, vol. 7, no. 1, pp. 1–12, Dec. 2021, doi: 10.1186/S40942-021-00313-9/FIGURES/9.

M. M. Fraz et al., “Blood vessel segmentation methodologies in retinal images – A survey,†Comput Methods Programs Biomed, vol. 108, no. 1, pp. 407–433, Oct. 2012, doi: 10.1016/J.CMPB.2012.03.009.

G. Azzopardi, N. Strisciuglio, M. Vento, and N. Petkov, “Trainable COSFIRE filters for vessel delineation with application to retinal images,†Med Image Anal, vol. 19, no. 1, pp. 46–57, Jan. 2015, doi: 10.1016/J.MEDIA.2014.08.002.

N. Strisciuglio, G. Azzopardi, M. Vento, and N. Petkov, “Supervised vessel delineation in retinal fundus images with the automatic selection of B-COSFIRE filters,†Mach Vis Appl, vol. 27, no. 8, pp. 1137–1149, Nov. 2016, doi: 10.1007/S00138-016-0781-7/TABLES/3.

K. B. Khan, A. A. Khaliq, A. Jalil, and M. Shahid, “A robust technique based on VLM and Frangi filter for retinal vessel extraction and denoising,†PLoS One, vol. 13, no. 2, p. e0192203, Feb. 2018, doi: 10.1371/JOURNAL.PONE.0192203.

S. A. Badawi and M. M. Fraz, “Optimizing the trainable B-COSFIRE filter for retinal blood vessel segmentation,†PeerJ, vol. 6, no. 11, 2018, doi: 10.7717/PEERJ.5855.

A. Ali, M. Diyana, W. Zaki, A. Hussain, W. M. Diyana, and W. Zaki, “Retinal blood vessel segmentation from retinal image using B-COSFIRE and adaptive thresholding,†Indonesian Journal of Electrical Engineering and Computer Science, vol. 13, no. 3, pp. 1199–1207, 2019, doi: 10.11591/ijeecs.v13.i3.pp1199-1207.

J. V. B. Soares, J. J. G. Leandro, R. M. Cesar, H. F. Jelinek, and M. J. Cree, “Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification,†IEEE Trans Med Imaging, vol. 25, no. 9, pp. 1214–1222, Sep. 2006, doi: 10.1109/TMI.2006.879967.

S. Chaudhuri, S. Chatterjee, N. Katz, M. Nelson, and M. Goldbaum, “Detection of blood vessels in retinal images using two-dimensional matched filters,†IEEE Trans Med Imaging, vol. 8, no. 3, pp. 263–269, 1989, doi: 10.1109/42.34715.

A. Hoover, “Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response,†IEEE Trans Med Imaging, vol. 19, no. 3, pp. 203–210, 2000, doi: 10.1109/42.845178.

N. Memari, A. R. Ramli, M. I. Bin Saripan, S. Mashohor, and M. Moghbel, “Supervised retinal vessel segmentation from color fundus images based on matched filtering and AdaBoost classifier,†PLoS One, vol. 12, no. 12, p. e0188939, Dec. 2017, doi: 10.1371/JOURNAL.PONE.0188939.

J. V. B. Soares, J. J. G. Leandro, R. M. Cesar-, H. F. Jelinek, and M. J. Cree, “Retinal Vessel Segmentation Using the 2-D Morlet Wavelet and Supervised Classification,†2006.

S. Tang, T. Lin, J. Yang, J. Fan, D. Ai, and Y. Wang, “Retinal vessel segmentation using supervised classification based on multiscale vessel filtering and gabor wavelet,†J Med Imaging Health Inform, vol. 5, no. 7, pp. 1571–1574, Nov. 2015, doi: 10.1166/JMIHI.2015.1565.

L. Deng and D. Yu, “Deep Learning: Methods and Applications,†Foundations and Trends® in Signal Processing, vol. 7, no. 3–4, pp. 197–387, 2014, doi: 10.1561/2000000039.

M. Badar, M. Haris, and A. Fatima, “Application of deep learning for retinal image analysis: A review,†Comput Sci Rev, vol. 35, p. 100203, Feb. 2020, doi: 10.1016/J.COSREV.2019.100203.

C. Chen, J. H. Chuah, R. Ali, and Y. Wang, “Retinal vessel segmentation using deep learning: A review,†IEEE Access, vol. 9, pp. 111985–112004, 2021, doi: 10.1109/ACCESS.2021.3102176.

Z. Fan and J. J. Mo, “Automated blood vessel segmentation based on de-noising auto-encoder and neural network,†Proc Int Conf Mach Learn Cybern, vol. 2, pp. 849–856, Jul. 2016, doi: 10.1109/ICMLC.2016.7872998.

P. Liskowski and K. Krawiec, “Segmenting Retinal Blood Vessels with Deep Neural Networks,†IEEE Trans Med Imaging, vol. 35, no. 11, pp. 2369–2380, Nov. 2016, doi: 10.1109/TMI.2016.2546227.

E. Uysal and G. E. Güraksin, “Computer-aided retinal vessel segmentation in retinal images: convolutional neural networks,†Multimed Tools Appl, vol. 80, no. 3, pp. 3505–3528, Sep. 2020, doi: 10.1007/S11042-020-09372-W/FIGURES/11.

O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,†Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9351, pp. 234–241, 2015, doi: 10.1007/978-3-319-24574-4_28.

B. Karasulu, “An automatic optic disk detection and segmentation system using multi-level thresholding,†Advances in Electrical and Computer Engineering, vol. 14, no. 2, pp. 161–172, 2014, doi: 10.4316/AECE.2014.02025.

D. Marin, M. E. Gegundez-Arias, A. Suero, and J. M. Bravo, “Obtaining optic disc center and pixel region by automatic thresholding methods on morphologically processed fundus images,†Comput Methods Programs Biomed, vol. 118, no. 2, pp. 173–185, Feb. 2015, doi: 10.1016/J.CMPB.2014.11.003.

J. Dietter et al., “Optic disc detection in the presence of strong technical artifacts,†Biomed Signal Process Control, vol. 53, p. 101535, Aug. 2019, doi: 10.1016/J.BSPC.2019.04.012.

J. Sigut, O. Nunez, F. Fumero, M. Gonzalez, and R. Arnay, “Contrast based circular approximation for accurate and robust optic disc segmentation in retinal images,†PeerJ, vol. 5, no. 9, 2017, doi: 10.7717/PEERJ.3763.

A. Almazroa, W. Sun, S. Alodhayb, K. Raahemifar, and V. Lakshminarayanan, “Optic disc segmentation: level set methods and blood vessels inpainting,†Medical Imaging 2017: Imaging Informatics for Healthcare, Research, and Applications, vol. 10138, p. 1013806, Mar. 2017, doi: 10.1117/12.2254174.

M. Abdullah, M. M. Fraz, and S. A. Barman, “Localization and segmentation of optic disc in retinal images using circular Hough transform and grow-cut algorithm,†PeerJ, vol. 4, no. 5, 2016, doi: 10.7717/PEERJ.2003.

Z. Fan et al., “Optic Disk Detection in Fundus Image Based on Structured Learning,†IEEE J Biomed Health Inform, vol. 22, no. 1, pp. 224–234, Jan. 2018, doi: 10.1109/JBHI.2017.2723678.

A. Ali, M. Hossain, N. Hashim, W. N. M. Isa, Z. C. Embi, and S. M. Desa, “Automatic Detection of Retinal Optic Disc using Vessel Inpainting,†2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp. 254–258, Oct. 2022, doi: 10.1109/SITIS57111.2022.00028.

S. Roychowdhury, D. D. Koozekanani, S. N. Kuchinka, and K. K. Parhi, “Optic disc boundary and vessel origin segmentation of fundus images,†IEEE J Biomed Health Inform, vol. 20, no. 6, pp. 1562–1574, Nov. 2016, doi: 10.1109/JBHI.2015.2473159.

E. Decencière et al., “Feedback on a publicly distributed image database: The Messidor database,†Image Analysis and Stereology, vol. 33, no. 3, pp. 231–234, 2014, doi: 10.5566/IAS.1155.

J. Sivaswamy, A. Chakravarty, G. D. Joshi, and T. A. Syed, “A Comprehensive Retinal Image Dataset for the Assessment of Glaucoma from the Optic Nerve Head Analysis,†2015.

P. Porwal et al., “IDRiD: Diabetic Retinopathy – Segmentation and Grading Challenge,†Med Image Anal, vol. 59, Jan. 2020, doi: 10.1016/J.MEDIA.2019.101561.

M. Alawad et al., “Machine Learning and Deep Learning Techniques for Optic Disc and Cup Segmentation – A Review,†Clinical Ophthalmology, vol. 16, pp. 747–764, 2022, doi: 10.2147/OPTH.S348479.

W. L. Alyoubi, W. M. Shalash, and M. F. Abulkhair, “Diabetic retinopathy detection through deep learning techniques: A review,†Inform Med Unlocked, vol. 20, p. 100377, Jan. 2020, doi: 10.1016/J.IMU.2020.100377.

C. P. Wilkinson et al., “Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales,†Ophthalmology, vol. 110, no. 9, pp. 1677–1682, Sep. 2003, doi: 10.1016/S0161-6420(03)00475-5.

M. Esfahani, … M. G.-L. Electron. J., and undefined 2018, “Classification of diabetic and normal fundus images using new deep learning method,†lejpt.academicdirect.org, Accessed: May 01, 2023. [Online]. Available: http://lejpt.academicdirect.org/A32/get_htm.php?htm=233_248

G. Quellec, K. Charrière, Y. Boudi, B. Cochener, and M. Lamard, “Deep image mining for diabetic retinopathy screening,†Med Image Anal, vol. 39, pp. 178–193, Jul. 2017, doi: 10.1016/J.MEDIA.2017.04.012.

K. Xu, D. Feng, and H. Mi, “Deep Convolutional Neural Network-Based Early Automated Detection of Diabetic Retinopathy Using Fundus Image,†Molecules 2017, Vol. 22, Page 2054, vol. 22, no. 12, p. 2054, Nov. 2017, doi: 10.3390/MOLECULES22122054.

M. D. Abrà moff et al., “Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning,†Invest Ophthalmol Vis Sci, vol. 57, no. 13, pp. 5200–5206, Oct. 2016, doi: 10.1167/IOVS.16-19964.

S. Dutta et al., “Classification of Diabetic Retinopathy Images by Using Deep Learning Models Mathematical Modeling View project Predictive Analytics View project Classification of Diabetic Retinopathy Images by Using Deep Learning Models,†International Journal of Grid and Distributed Computing, vol. 11, no. 1, pp. 89–106, 2018, doi: 10.14257/ijgdc.2018.11.1.09.

V. Gulshan et al., “Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs,†JAMA, vol. 316, no. 22, pp. 2402–2410, Dec. 2016, doi: 10.1001/JAMA.2016.17216.

H. Pratt, F. Coenen, D. M. Broadbent, S. P. Harding, and Y. Zheng, “Convolutional Neural Networks for Diabetic Retinopathy,†Procedia Comput Sci, vol. 90, pp. 200–205, Jan. 2016, doi: 10.1016/J.PROCS.2016.07.014.

X. Wang, Y. Lu, Y. Wang, and W. B. Chen, “Diabetic retinopathy stage classification using convolutional neural networks,†Proceedings - 2018 IEEE 19th International Conference on Information Reuse and Integration for Data Science, IRI 2018, pp. 465–471, Aug. 2018, doi: 10.1109/IRI.2018.00074.

P. Chudzik, S. Majumdar, F. Calivá, B. Al-Diri, and A. Hunter, “Microaneurysm detection using fully convolutional neural networks,†Comput Methods Programs Biomed, vol. 158, pp. 185–192, May 2018, doi: 10.1016/J.CMPB.2018.02.016.

Y. Yan, J. Gong, and Y. Liu, “A Novel Deep Learning Method for Red Lesions Detection Using Hybrid Feature,†Proceedings of the 31st Chinese Control and Decision Conference, CCDC 2019, pp. 2287–2292, Jun. 2019, doi: 10.1109/CCDC.2019.8833190.

K. Adem, “Exudate detection for diabetic retinopathy with circular Hough transformation and convolutional neural networks,†Expert Syst Appl, vol. 114, pp. 289–295, Dec. 2018, doi: 10.1016/J.ESWA.2018.07.053.

H. Wang et al., “Hard exudate detection based on deep model learned information and multi-feature joint representation for diabetic retinopathy screening,†Comput Methods Programs Biomed, vol. 191, p. 105398, Jul. 2020, doi: 10.1016/J.CMPB.2020.105398.

P. Furtado, “Multi-class segmentation of Diabetic Retinopathy lesions: Effects of metrics, improvements and loss,†Proceedings - 19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020, pp. 1410–1417, Dec. 2020, doi: 10.1109/ICMLA51294.2020.00219.

N. Shaukat, J. Amin, M. I. Sharif, M. I. Sharif, S. Kadry, and L. Sevcik, “Classification and Segmentation of Diabetic Retinopathy: A Systemic Review,†Applied Sciences 2023, Vol. 13, Page 3108, vol. 13, no. 5, p. 3108, Feb. 2023, doi: 10.3390/APP13053108.

T. Kauppi, V. Kalesnykiene, J. Kamarainen, L. L.- BMVC, and undefined 2007, “The diaretdb1 diabetic retinopathy database and evaluation protocol.,†it.lut.fi, Accessed: May 01, 2023. [Online]. Available: https://www.it.lut.fi/project/imageret/diaretdb1/doc/diaretdb1_techreport_v_1_1.pdf

T. Li, Y. Gao, K. Wang, S. Guo, H. Liu, and H. Kang, “Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening,†Inf Sci (N Y), vol. 501, pp. 511–522, Oct. 2019, doi: 10.1016/J.INS.2019.06.011.

M. Karakaya and R. E. Hacisoftaoglu, “Comparison of smartphone-based retinal imaging systems for diabetic retinopathy detection using deep learning,†BMC Bioinformatics, vol. 21, no. 4, pp. 1–18, Jul. 2020, doi: 10.1186/S12859-020-03587-2/TABLES/3.

M. A. P. Vilela, F. M. Valença, P. K. M. Barreto, C. E. V. Amaral, and L. C. Pellanda, “Agreement between retinal images obtained via smartphones and images obtained with retinal cameras or fundoscopic exams – Systematic review and meta-analysis,†Clinical Ophthalmology, vol. 12, pp. 2581–2589, 2018, doi: 10.2147/OPTH.S182022.

R. Hu, R. J. Chalakkal, G. Linde, and J. S. Dhupia, “Multi-image Stitching for Smartphone-based Retinal Fundus Stitching,†IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM, vol. 2022-July, pp. 179–184, 2022, doi: 10.1109/AIM52237.2022.9863260.

R. Besenczi, J. Tóth, and A. Hajdu, “A review on automatic analysis techniques for color fundus photographs,†Comput Struct Biotechnol J, vol. 14, pp. 371–384, 2016, doi: 10.1016/J.CSBJ.2016.10.001.

X. Xu et al., “Smartphone-Based Accurate Analysis of Retinal Vasculature towards Point-of-Care Diagnostics,†Scientific Reports 2016 6:1, vol. 6, no. 1, pp. 1–9, Oct. 2016, doi: 10.1038/srep34603.

M. Hossain, W. N. M. Isa, A. Ali, W. M. D. W. Zaki, N. Hashim, and A. Hussain, “Optimized Smartphone-based Implementation of B-COSFIRE Filter for Retinal Blood Vessel Segmentation,†2022 IEEE International Conference on Computing, ICOCO 2022, pp. 74–79, 2022, doi: 10.1109/ICOCO56118.2022.10031761.

T. T. Khaing, P. Aimmanee, S. Makhanov, and H. Haneishi, “Vessel-based hybrid optic disk segmentation applied to mobile phone camera retinal images,†Med Biol Eng Comput, vol. 60, no. 2, pp. 421–437, Feb. 2022, doi: 10.1007/S11517-021-02484-X.

R. E. Hacisoftaoglu, M. Karakaya, and A. B. Sallam, “Deep learning frameworks for diabetic retinopathy detection with smartphone-based retinal imaging systems,†Pattern Recognit Lett, vol. 135, pp. 409–417, Jul. 2020, doi: 10.1016/J.PATREC.2020.04.009.

S. Sengupta, M. D. Sindal, P. Baskaran, U. Pan, and R. Venkatesh, “Sensitivity and Specificity of Smartphone-Based Retinal Imaging for Diabetic Retinopathy: A Comparative Study,†Ophthalmol Retina, vol. 3, no. 2, pp. 146–153, Feb. 2019, doi: 10.1016/J.ORET.2018.09.016.

R. Rajalakshmi, R. Subashini, R. M. Anjana, and V. Mohan, “Automated diabetic retinopathy detection in smartphone-based fundus photography using artificial intelligence,†Eye 2018 32:6, vol. 32, no. 6, pp. 1138–1144, Mar. 2018, doi: 10.1038/s41433-018-0064-9.

O. Russakovsky et al., “ImageNet Large Scale Visual Recognition Challenge,†Int J Comput Vis, vol. 115, no. 3, pp. 211–252, Dec. 2015, doi: 10.1007/S11263-015-0816-Y.

P. Iakubovskii, “Segmentation Models,†GitHub repository. GitHub, 2019.