Lightweight Generative Adversarial Network Fundus Image Synthesis

Nurhakimah Abd Aziz - School of Electrical Engineering, College of Engineering, Universiti Teknologi MARA, 40450 Shah Alam, Selangor, Malaysia
Mohd Azman Hanif Sulaiman - School of Electrical Engineering, College of Engineering, Universiti Teknologi MARA, 40450 Shah Alam, Selangor, Malaysia
Azlee Zabidi - Faculty of Systems & Software Engineering, College of Computing & Applied Sciences, Universiti Malaysia Pahang, 26600 Pekan, Pahang, Malaysia
Ihsan Mohd Yassin - Microwave Research Institute, Universiti Teknologi MARA, 40450 Shah Alam, Selangor, Malaysia
Megat Syahirul Amin Megat Ali - Microwave Research Institute, Universiti Teknologi MARA, 40450 Shah Alam, Selangor, Malaysia
Zairi Ismael Rizman - School of Electrical Engineering, College of Engineering, Universiti Teknologi MARA, 23000 Dungun, Terengganu, Malaysia


Citation Format:



DOI: http://dx.doi.org/10.30630/joiv.6.1-2.924

Abstract


Blindness is a global health problem that affects billions of lives. Recent advancements in Artificial Intelligence (AI), (Deep Learning (DL)) has the intervention potential to address the blindness issue, particularly as an accurate and non-invasive technique for early detection and treatment of Diabetic Retinopathy (DR). DL-based techniques rely on extensive examples to be robust and accurate in capturing the features responsible for representing the data. However, the number of samples required is tremendous for the DL classifier to learn properly. This presents an issue in collecting and categorizing many samples. Therefore, in this paper, we present a lightweight Generative Neural Network (GAN) to synthesize fundus samples to train AI-based systems. The GAN was trained using samples collected from publicly available datasets. The GAN follows the structure of the recent Lightweight GAN (LGAN) architecture. The implementation and results of the LGAN training and image generation are described. Results indicate that the trained network was able to generate realistic high-resolution samples of normal and diseased fundus images accurately as the generated results managed to realistically represent key structures and their placements inside the generated samples, such as the optic disc, blood vessels, exudates, and others. Successful and unsuccessful generation samples were sorted manually, yielding 56.66% realistic results relative to the total generated samples. Rejected generated samples appear to be due to inconsistencies in shape, key structures, placements, and color.

Keywords


Generative Adversarial Network (GAN); fundus; artificial intelligence; data synthesis.

Full Text:

PDF

References


World Health Organization (WHO), “World Report on Vision,†Geneva: WHO, 2019.

Eisa Aghchehli and Malcolm Clarke, “Deep learning based data augmentation using Generative Adversarial Network in Diabetic Retinopathy: A Review,†2020, https://www.researchgate.net/profile/Eisa-Aghchehli-2/publication/339680691_DEEP_LEARNING_BASED_DATA_AUGMENTATION_USING_GENERATIVE_ADVERSARIAL_NETWORK_IN_DIABETIC_RETINOPATHY_A_REVIEW/links/5e6eaf47458515e5557fba74/DEEP-LEARNING-BASED-DATA-AUGMENTATION-USING-GENERATIVE-ADVERSARIAL-NETWORK-IN-DIABETIC-RETINOPATHY-A-REVIEW.pdf.

N. Mukherjee and S. Sengupta, “In Search for the Optimal Preprocessing Technique for Deep Learning Based Diabetic Retinopathy Stage Classication from Fundus Imagesâ€, doi: 10.21203/rs.3.rs-654484/v1.

D. S. W. Ting et al., “Artificial intelligence and deep learning in ophthalmology,†British Journal of Ophthalmology, vol. 103, no. 2. BMJ Publishing Group, pp. 167–175, Feb. 01, 2019. doi: 10.1136/bjophthalmol-2018-313173.

C.-I. Suciu, V.-I. Suciu, A. Cuţaş, and S. D. Nicoară, “Interleaved Optical Coherence Tomography: Clinical and Laboratory Biomarkers in Patients with Diabetic Macular Edema,†Journal of Personalized Medicine, vol. 12, no. 5, p. 765, May 2022, doi: 10.3390/jpm12050765.

C. Sabanayagam et al., “Association of Diabetic Retinopathy and Diabetic Kidney Disease with All-Cause and Cardiovascular Mortality in a Multiethnic Asian Population,†JAMA Network Open, vol. 2, no. 3, Mar. 2019, doi: 10.1001/jamanetworkopen.2019.1540.

S. Mishra and S. Hanchate, “Detection of Diabetic Retinopathy from Fundus Images using Deep Learning:A Review.†[Online]. Available: www.joics.net

J. Xu, C. Wan, W. Yang, B. Zheng, Z. Yan, and J. Shen, “A novel multi-modal fundus image fusion method for guiding the laser surgery of central serous chorioretinopathy,†Mathematical Biosciences and Engineering, vol. 18, no. 4, pp. 4797–4816, 2021, doi: 10.3934/mbe.2021244.

P. Cheng, L. Lin, Y. Huang, J. Lyu, and X. Tang, “I-SECRET: Importance-Guided Fundus Image Enhancement via Semi-supervised Contrastive Constraining,†in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2021, vol. 12908 LNCS, pp. 87–96. doi: 10.1007/978-3-030-87237-3_9.

A. D. Pérez, O. Perdomo, H. Rios, F. Rodríguez, and F. A. González, “A Conditional Generative Adversarial Network-Based Method for Eye Fundus Image Quality Enhancement,†in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020, vol. 12069 LNCS, pp. 185–194. doi: 10.1007/978-3-030-63419-3_19.

S. Sathiya Priya and J. G. R. Sathiaseelan, “Enhanced Retina Blood Vessel Segmentation by Super Resolution Generative Adversarial Networks based U-Net,†Article in Indian Journal of Science and Technology, vol. 14, no. 43, pp. 3246–3253, 2021, doi: 10.17485/IJST/v14i43.1502.

S. A. Rammy, W. Abbas, N. U. Hassan, A. Raza, and W. Zhang, “CPGAN: Conditional patch-based generative adversarial network for retinal vessel segmentation,†IET Image Processing, vol. 14, no. 6, pp. 1081–1090, May 2020, doi: 10.1049/iet-ipr.2019.1007.

P. Cheng, L. Lin, Y. Huang, J. Lyu, and X. Tang, “Prior guided fundus image quality enhancement via contrastive learning,†in Proceedings - International Symposium on Biomedical Imaging, Apr. 2021, vol. 2021-April, pp. 521–525. doi: 10.1109/ISBI48211.2021.9434005.

S. Mirabedini, M. Kangavari, and J. Mohammadzadeh, “Diabetic retinopathy classification via Generative Adversarial Networksâ€, [Online]. Available: www.isisn.org

L. Wang and A. Schaefer, “Diagnosing diabetic retinopathy from images of the eye fundus.†[Online]. Available: https://machinethink.net/blog/mobilenet-v2/

F. Lateef, M. Kas, and Y. Ruichek, “Saliency Heat-Map as Visual Attention for Autonomous Driving Using Generative Adversarial Network (GAN),†IEEE Transactions on Intelligent Transportation Systems, pp. 1–14, 2021, doi: 10.1109/TITS.2021.3053178.

S. O. Patil, V. v. Sajith Variyar, and K. P. Soman, “Speed Bump Segmentation an Application of Conditional Generative Adversarial Network for Self-driving Vehicles,†Proceedings of the 4th International Conference on Computing Methodologies and Communication, ICCMC 2020, no. Iccmc, pp. 935–939, 2020, doi: 10.1109/ICCMC48092.2020.ICCMC-000173.

J. Spooner, V. Palade, M. Cheah, S. Kanarachos, and A. Daneshkhah, “Generation of pedestrian crossing scenarios using ped-cross generative adversarial network,†Applied Sciences (Switzerland), vol. 11, no. 2, pp. 1–21, 2021, doi: 10.3390/app11020471.

B. Liu, Y. Zhu, K. Song, and A. Elgammal, “Towards Faster and Stabilized GAN Training for High-Fidelity Few Shot Image Synthesis,†in Proc.of 2021 Int. Conf. on Learning Representations (ICLR2021), 2021, pp. 1–13. [Online]. Available: https://github.com/odegeasslbc/FastGAN-pytorch

B. Bhatkalkar, A. Joshi, S. Prabhu, and S. Bhandary, “Automated fundus image quality assessment and segmentation of optic disc using convolutional neural networks,†International Journal of Electrical and Computer Engineering, vol. 10, no. 1, pp. 816–827, 2020, doi: 10.11591/ijece.v10i1.pp816-827.

M. AlGhamdi, “Optic Disc Segmentation in Fundus Images with Deep Learning Object Detector,†Journal of Computer Science, vol. 16, no. 5, pp. 591–600, 2020, doi: 10.3844/JCSSP.2020.591.600.

Y. Liu, D. Fu, Z. Huang, and H. Tong, “Optic disc segmentation in fundus images using adversarial training,†IET Image Processing, vol. 13, no. 2, pp. 375–381, Feb. 2019, doi: 10.1049/iet-ipr.2018.5922.

X. He, R. Luo, T. Shou, and H. Xiao, “Automatic Detection of Hard Exudates in Retinal Fundus Images,†2021. [Online]. Available: www.aaai.org

W. Setiawan, M. I. Utoyo, and R. Rulaningtyas, “Transfer learning with multiple pre-trained network for fundus classification,†Telkomnika (Telecommunication Computing Electronics and Control), vol. 18, no. 3, pp. 1382–1388, 2020, doi: 10.12928/TELKOMNIKA.v18i3.14868.

X. Wang, M. Xu, J. Zhang, L. Jiang, and L. Li, “Deep Multi-Task Learning for Diabetic Retinopathy Grading in Fundus Images.†[Online]. Available: www.aaai.org

Bingchen Liu, Yizhe Zhu, Kumpeng Song, and Ahmed Elgamal, “Github - Lightweight GAN, Available: https://github.com/lucidrains/lightweight-gan,†https://github.com/lucidrains/lightweight-gan, 2021.

D. M. Vo, D. M. Nguyen, T. P. Le, and S. W. Lee, “HI-GAN: A hierarchical generative adversarial network for blind denoising of real photographs,†Information Sciences, vol. 570, pp. 225–240, Sep. 2021, doi: 10.1016/j.ins.2021.04.045.

D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,†Dec. 2014, [Online]. Available: http://arxiv.org/abs/1412.6980

Ã. Arcos-García, J. A. Ãlvarez-García, and L. M. Soria-Morillo, “Deep neural network for traffic sign recognition systems: An analysis of spatial transformers and stochastic optimisation methods,†Neural Networks, vol. 99, pp. 158–165, Mar. 2018, doi: 10.1016/j.neunet.2018.01.005.

Xuyang Guo et al., “Exploration of Automatic Mixed-Precision Search for Deep Neural Networks,†2019.