Deep Convolutional Neural Networks Transfer Learning Comparison on Arabic Handwriting Recognition System

Siti Ummi Masruroh - Syarif Hidayatullah State Islamic University Jakarta, Tangerang, Indonesia
Muhammad Fikri Syahid - Syarif Hidayatullah State Islamic University Jakarta, Tangerang, Indonesia
Firman Munthaha - Syarif Hidayatullah State Islamic University Jakarta, Tangerang, Indonesia
Asep Taufik Muharram - Jakarta State Polytechnic, West Java, Indonesia
Rizka Amalia Putri - Syarif Hidayatullah State Islamic University Jakarta, Tangerang, Indonesia

Citation Format:



Around 27 languages and more than 420 million people worldwide use Arabic letters. That makes the Arabic language one of the most used languages. However, the Arabic language has a challenge, namely the difference in letters based on their position. Arabic handwriting recognition is important for various applications, such as education and communication. One example is during a pandemic when most education has turned digital, making recognizing students' Arabic handwriting difficult. This paper aims to create a model that can recognize Arabic handwriting by comparing several CNN architectures using transfer learning to classify Arabic, Hijja, and AHCD handwriting datasets. Transfer learning is a model that has been trained by previous datasets to other datasets and is suitable for use in models with small datasets because it can improve model accuracy even with small datasets. The datasets were split into 60%, 20%, and 20% for training, validation, and testing. Each model uses data augmentation and 50% dropout on a fully connected layer to reduce overfitting. Some of the CNN architectures used in this study to create Arabic writing recognition models are ResNet, DenseNet, VGG16, VGG19, InceptionV3, and MobileNet. The models were compiled and trained with various parameters. The best model achieved to classify AHCD and Hijja dataset is VGG16 with Adam optimizer and 0.0001 learning rate. Based on this research, it is expected to know the performance of the best model for classifying Arabic handwriting.


Arabic; Recognition; CNN; Transfer Learning; Optimizer; Learning Rate.

Full Text:



T. M. Ghanim, M. I. Khalil, and H. M. Abbas, "Comparative Study on Deep Convolution Neural Networks DCNN-Based Offline Arabic Handwriting Recognition," IEEE Access, vol. 8, pp. 95465–95482, 2020, doi: 10.1109/ACCESS.2020.2994290.

M. Shams, A. A. Elsonbaty, and W. Z. El Sawy, "Arabic handwritten character recognition based on convolution neural networks and support vector machine," Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 8, pp. 144–149, 2020, doi: 10.14569/IJACSA.2020.0110819.

D. K. Muhsen, S. M. Ali, R. M. Zaki, and A. A. Ahmed, "Arguments extraction for e-health services based on text mining tools," Period. Eng. Nat. Sci., vol. 9, no. 3, pp. 309–316, 2021, doi: 10.21533/pen.v9i3.2149.

H. Akouaydi, S. Njah, W. Ouarda, A. Samet, M. Zaied, and A. M. Alimi, "Convolutional neural networks for online Arabic characters recognition with beta-elliptic knowledge domain," 2019 Int. Conf. Doc. Anal. Recognit. Work. ICDARW 2019, vol. 6, pp. 41–46, 2019, doi: 10.1109/ICDARW.2019.50114.

H. Butt, M. R. Raza, M. J. Ramzan, M. J. Ali, and M. Haris, "Attention-Based CNN-RNN Arabic Text Recognition from Natural Scene Images," Forecasting, vol. 3, no. 3, pp. 520–540, 2021, doi: 10.3390/forecast3030033.

M. Eltay, A. Zidouri, and I. Ahmad, "Exploring Deep Learning Approaches to Recognize Handwritten Arabic Texts," IEEE Access, vol. 8, pp. 89882–89898, 2020, doi: 10.1109/ACCESS.2020.2994248.

N. Altwaijry and I. Al-Turaiki, "Arabic handwriting recognition system using convolutional neural network," Neural Comput. Appl., vol. 33, no. 7, pp. 2249–2261, 2021, doi: 10.1007/s00521-020-05070-8.

M. N. Aljarrah, M. M. Zyout, and R. Duwairi, "Arabic Handwritten Characters Recognition Using Convolutional Neural Network," 2021 12th Int. Conf. Inf. Commun. Syst. ICICS 2021, no. February, pp. 182–188, 2021, doi: 10.1109/ICICS52457.2021.9464596.

D. Radovanovic and S. Dukanovic, "Image-Based Plant Disease Detection: A Comparison of Deep Learning and Classical Machine Learning Algorithms," 2020 24th Int. Conf. Inf. Technol. IT 2020, no. February, pp. 1–4, 2020, doi: 10.1109/IT48810.2020.9070664.

J. Rozaqi, A. Sunyoto, and R. Arief, "Implementasi Transfer Learning pada Algoritma Convolutional Neural Network Untuk Identifikasi Penyakit Daun Kentang Implementation of Transfer Learning in the Convolutional Neural Network Algorithm for Identification of Potato Leaf Disease," vol. 1, no. 1, 2021.

R. Rismiyati and A. Luthfiarta, “VGG16 Transfer Learning Architecture for Salak Fruit Quality Classification,†Telematika, vol. 18, no. 1, p. 37, 2021, doi: 10.31315/telematika.v18i1.4025.

E. C. Too, L. Yujian, S. Njuki, and L. Yingchun, "A comparative study of fine-tuning deep learning models for plant disease identification," Comput. Electron. Agric., vol. 161, no. February, pp. 272–279, 2019, doi: 10.1016/j.compag.2018.03.032.

S. K. Roy, G. Krishna, S. R. Dubey, and B. B. Chaudhuri, "HybridSN: Exploring 3-D-2-D CNN Feature Hierarchy for Hyperspectral Image Classification," IEEE Geosci. Remote Sens. Lett., vol. 17, no. 2, pp. 277–281, 2020, doi: 10.1109/LGRS.2019.2918719.

D. R. Sarvamangala and R. V. Kulkarni, "Convolutional neural networks in medical image understanding: a survey," Evol. Intell., vol. 15, no. 1, 2022, doi: 10.1007/s12065-020-00540-3.

I. Goodfellow, Y. Bengio, and C. Aaron, Deep Learning. MIT Press, 2016.

Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, "A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects," IEEE Trans. Neural Networks Learn. Syst., vol. 33, no. 12, pp. 6999–7019, 2022, doi: 10.1109/TNNLS.2021.3084827.

T. Kattenborn, J. Leitloff, F. Schiefer, and S. Hinz, "Review on Convolutional Neural Networks (CNN) in vegetation remote sensing," ISPRS J. Photogramm. Remote Sens., vol. 173, no. November 2020, pp. 24–49, 2021, doi: 10.1016/j.isprsjprs.2020.12.010.

Q. A. Al-Haija and A. Adebanjo, "Breast cancer diagnosis in histopathological images using ResNet-50 convolutional neural network," IEMTRONICS 2020 - Int. IOT, Electron. Mechatronics Conf. Proc., vol. 50, 2020, doi: 10.1109/IEMTRONICS51293.2020.9216455.

R. Sujatha, J. M. Chatterjee, N. Z. Jhanjhi, and S. N. Brohi, "Performance of deep learning vs machine learning in plant leaf disease detection," Microprocess. Microsyst., vol. 80, no. November 2020, p. 103615, 2021, doi: 10.1016/j.micpro.2020.103615.

A. G. Howard et al., "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications," 2017.

D. Sarwinda, R. H. Paradisa, A. Bustamam, and P. Anggia, "Deep Learning in Image Classification using Residual Network (ResNet) Variants for Detection of Colorectal Cancer," Procedia Comput. Sci., vol. 179, no. 2019, pp. 423–431, 2021, doi: 10.1016/j.procs.2021.01.025.

K. Deeba and B. Amutha, "ResNet - deep neural network architecture for leaf disease classification," Microprocess. Microsyst., p. 103364, 2020, doi: 10.1016/j.micpro.2020.103364.

A. M. Rafi et al., "Application of DenseNet in Camera Model Identification and Post-processing Detection," pp. 19–28, 2018.

Z. Zhang, Z. Tang, Y. Wang, H. Zhang, S. Yan, and M. Wang, "Compressed DenseNet for Lightweight Character Recognition," arXiv Prepr. arXiv1912.07016, pp. 1–11, 2019.

K. Zhang, Y. Guo, X. Wang, J. Yuan, and Q. Ding, "Multiple feature reweight DenseNet for image classification," IEEE Access, vol. 7, pp. 9872–9880, 2019, doi: 10.1109/ACCESS.2018.2890127.

P. Hridayami, I. K. G. D. Putra, and K. S. Wibawa, "Fish species recognition using VGG16 deep convolutional neural network," J. Comput. Sci. Eng., vol. 13, no. 3, pp. 124–130, 2019, doi: 10.5626/JCSE.2019.13.3.124.

D. Theckedath and R. R. Sedamkar, "Detecting Affect States Using VGG16, ResNet50 and SE-ResNet50 Networks," SN Comput. Sci., vol. 1, no. 2, pp. 1–7, 2020, doi: 10.1007/s42979-020-0114-9.

M. Mujahid, F. Rustam, R. Ãlvarez, J. Luis Vidal Mazón, I. de la T. Díez, and I. Ashraf, "Pneumonia Classification from X-ray Images with Inception-V3 and Convolutional Neural Network," Diagnostics, vol. 12, no. 5, pp. 1–16, 2022, doi: 10.3390/diagnostics12051280.

A. Michele, V. Colin, and D. D. Santika, "Mobilenet convolutional neural networks and support vector machines for palmprint recognition," Procedia Comput. Sci., vol. 157, pp. 110–117, 2019, doi: 10.1016/j.procs.2019.08.147.

C. Bi, J. Wang, Y. Duan, B. Fu, J. R. Kang, and Y. Shi, "MobileNet Based Apple Leaf Diseases Identification," Mob. Networks Appl., 2020, doi: 10.1007/s11036-020-01640-1.