End-To-End Evaluation of Deep Learning Architectures for Off-Line Handwriting Writer Identification: A Comparative Study

Wirmanto Suteddy - Universitas Pendidikan Indonesia, Bandung, 40625, Indonesia
Devi Agustini - Universitas Pendidikan Indonesia, Bandung, 40625, Indonesia
Anugrah Adiwilaga - Universitas Pendidikan Indonesia, Bandung, 40625, Indonesia
Dastin Atmanto - Universitas Pendidikan Indonesia, Bandung, 40625, Indonesia


Citation Format:



DOI: http://dx.doi.org/10.30630/joiv.7.1.1293

Abstract


Identifying writers using their handwriting is particularly challenging for a machine, given that a person’s writing can serve as their distinguishing characteristic. The process of identification using handcrafted features has shown promising results, but the intra-class variability between authors still needs further development. Almost all computer vision-related tasks use Deep learning (DL) nowadays, and as a result, researchers are developing many DL architectures with their respective methods. In addition, feature extraction, usually accomplished using handcrafted algorithms, can now be automatically conducted using convolutional neural networks. With the various developments of the DL method, it is necessary to evaluate the suitable DL for the problem we are aiming at, namely the classification of writer identification. This comparative study evaluated several DL architectures such as VGG16, ResNet50, MobileNet, Xception, and EfficientNet end-to-end to examine their advantages to offline handwriting for writer identification problems with IAM and CVL databases. Each architecture compared its respective process to the training and validation metrics accuracy, demonstrating that ResNet50 DL had the highest train accuracy of 98.86%. However, Xception DL performed slightly better due to the convergence gap for validation accuracy compared to all the other architectures, which were 21.79% and 15.12% for IAM and CVL. Also, the smallest gap of convergence between training and validation accuracy for the IAM and CVL datasets were 19.13% and 16.49%, respectively. The results of these findings serve as the basis for DL architecture selection and open up overfitting problems for future work.


Keywords


offline writer identification; deep learning; convolutional neural network; comparative study; end-to-end.

Full Text:

PDF

References


A. L. Hagström, R. Stanikzai, J. Bigun, and F. Alonso-Fernandez, “Writer Recognition Using Offline Handwritten Single Block Characters.†arXiv, Mar. 07, 2022. Accessed: Sep. 02, 2022.

M. Sonkusare and N. Sahu, “A Survey on Handwritten Character Recognition (HCR) Techniques for English Alphabets,†AVC, vol. 3, no. 1, pp. 1–12, Mar. 2016.

C. Halder, Sk. Md. Obaidullah, and K. Roy, “Offline Writer Identification and Verification—A State-of-the-Art,†in Information Systems Design and Intelligent Applications, vol. 435, S. C. Satapathy, J. K. Mandal, S. K. Udgata, and V. Bhateja, Eds. New Delhi: Springer India, 2016, pp. 153–163.

C. Adak, B. B. Chaudhuri, and M. Blumenstein, “An Empirical Study on Writer Identification and Verification From Intra-Variable Individual Handwriting,†IEEE Access, vol. 7, pp. 24738–24758, 2019.

P. Kumar and A. Sharma, “Segmentation-free writer identification based on convolutional neural network,†Computers & Electrical Engineering, vol. 85, p. 106707, Jul. 2020.

D. Bertolini, L. S. Oliveira, E. Justino, and R. Sabourin, “Texture-based descriptors for writer identification and verification,†Expert Syst. Appl., vol. 40, no. 6, pp. 2069–2080, May 2013.

P. Singh, P. P. Roy, and B. Raman, “Writer identification using texture features: A comparative study,†Comput. Electr. Eng., vol. 71, pp. 1–12, Oct. 2018.

F. A. Khan, M. A. Tahir, F. Khelifi, A. Bouridane, and R. Almotaeryi, “Robust offline text independent writer identification using bagged discrete cosine transform features,†Expert Syst. Appl., vol. 71, pp. 404–415, Apr. 2017.

S. Fiel and R. Sablatnig, “Writer Identification and Retrieval Using a Convolutional Neural Network,†in Computer Analysis of Images and Patterns, vol. 9257, G. Azzopardi and N. Petkov, Eds. Cham: Springer International Publishing, 2015, pp. 26–37.

L. G. Helal, Y. Maldonado, G. da Costa, D. B. Goncalves, and G. Z. Felipe, “Offline writer identification using handcrafted features versus ConvNets,†in 2017 36th International Conference of the Chilean Computer Science Society (SCCC), Arica, Oct. 2017, pp. 1–8.

K. Ni, P. Callier, B. Hatch, J. Mastarone, and J. Cline, “On noise reduction for handwritten writer identification,†in 2017 51st Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, Oct. 2017, pp. 1984–1988.

V. Christlein, M. Gropp, S. Fiel, and A. Maier, “Unsupervised Feature Learning for Writer Identification and Writer Retrieval,†in 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, Nov. 2017, pp. 991–997.

A. Bria et al., “Deep Transfer Learning for writer identification in medieval books,†in 2018 Metrology for Archaeology and Cultural Heritage (MetroArchaeo), Cassino FR, Italy, Oct. 2018, pp. 455–460.

M. Hosoe, T. Yamada, K. Kato, and K. Yamamoto, “Offline Text-Independent Writer Identification Based on Writer-Independent Model using Conditional AutoEncoder,†in 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), Niagara Falls, NY, USA, Aug. 2018, pp. 441–446.

M. Keglevic, S. Fiel, and R. Sablatnig, “Learning Features for Writer Retrieval and Identification using Triplet CNNs,†in 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), Niagara Falls, NY, USA, Aug. 2018, pp. 211–216.

H. T. Nguyen, C. T. Nguyen, T. Ino, B. Indurkhya, and M. Nakagawa, “Text-independent writer identification using convolutional neural network,†Pattern Recognition Letters, vol. 121, pp. 104–112, Apr. 2019.

A. Durou, S. Al-Maadeed, I. Aref, A. Bouridane, and M. Elbendak, “A Comparative Study of Machine Learning Approaches for Handwriter Identification,†in 2019 IEEE 12th International Conference on Global Security, Safety and Sustainability (ICGS3), London, United Kingdom, Jan. 2019, pp. 206–212.

L. G. Helal, D. Bertolini, Y. M. G. Costa, G. D. C. Cavalcanti, A. S. Britto, and L. E. S. Oliveira, “Representation Learning and Dissimilarity for Writer Identification,†in 2019 International Conference on Systems, Signals and Image Processing (IWSSIP), Osijek, Croatia, Jun. 2019, pp. 63–68.

A. Rehman, S. Naz, M. I. Razzak, and I. A. Hameed, “Automatic Visual Features for Writer Identification: A Deep Learning Approach,†IEEE Access, vol. 7, pp. 17149–17157, 2019.

A. Semma, Y. Hannad, I. Siddiqi, C. Djeddi, and M. El Youssfi El Kettani, “Writer Identification using Deep Learning with FAST Keypoints and Harris corner detector,†Expert Systems with Applications, vol. 184, p. 115473, Dec. 2021.

A. Sulaiman, K. Omar, M. F. Nasrudin, and A. Arram, “Length Independent Writer Identification Based on the Fusion of Deep and Handcrafted Descriptors,†IEEE Access, vol. 7, pp. 91772–91784, 2019.

S. He and L. Schomaker, “FragNet: Writer Identification Using Deep Fragment Networks,†IEEE Trans.Inform.Forensic Secur., vol. 15, pp. 3013–3022, 2020.

U.-V. Marti and H. Bunke, “The IAM-database: an English sentence database for offline handwriting recognition,†International Journal on Document Analysis and Recognition, vol. 5, no. 1, pp. 39–46, Nov. 2002.

F. Kleber, S. Fiel, M. Diem, and R. Sablatnig, “CVL-DataBase: An Offline Database for Writer Retrieval, Writer Identification and Word Spotting,†in 2013 12th International Conference on Document Analysis and Recognition, Washington, DC, USA, Aug. 2013, pp. 560–564.

K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks For Large-Scale Image Recognition,†p. 14, 2015.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,†in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, Jun. 2016, pp. 770–778.

A. G. Howard et al., “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications.†arXiv, 2017.

F. Chollet, “Xception: Deep Learning with Depthwise Separable Convolutions,†in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, Jul. 2017, pp. 1800–1807.

M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,†2019.

F. Chollet and others, “Keras.†GitHub, 2015. [Online]. Available: https://github.com/fchollet/keras