Skew Correction and Image Cleaning Handwriting Recognition Using a Convolutional Neural Network

Shofwatul Uyun - Universitas Islam Negeri Sunan Kalijaga Yogyakarta, Indonesia
Seto Rahardyan - Universitas Islam Negeri Sunan Kalijaga Yogyakarta, Indonesia
Muhammad Anshari - Universiti Brunei Darussalam, Bandar Seri Begawan, Brunei Darussalam


Citation Format:



DOI: http://dx.doi.org/10.30630/joiv.7.3.1712

Abstract


Handwriting recognition is a study of Optical Character Recognition (OCR) which has a high level of complexity. In addition, everyone has a unique and inconsistent handwriting style in writing characters upright, affecting recognition success. However, proper pre-processing and classification algorithms affect the success of pattern recognition systems. This paper proposes a pre-processing method for handwriting image recognition using a convolutional neural network (CNN). This study uses public datasets for training and private datasets for testing. This pre-processing consists of three processes: image cleaning, skew correction, and segmentation. These three processes aim to clean the image from unnecessary ink streaks. In addition, to make angle corrections to characters in italics in their writing. The model testing process uses image test data of handwriting that are not straight. There are three images based on the inclination angle: less than 45 degrees, equal to 45 degrees, and more than 45 degrees. Picture cleaning removes unnecessary strokes (noise) from the image using a layer mask, whereas skew correction changes the handwriting to an upright posture based on the detected angle. The pre-processing model we propose worked optimally on handwriting with a skew angle of fewer than 45 degrees and 45 degrees. Our proposed model generally works well for handwriting with fewer than 45 degrees skew with an accuracy of 88,96%. Research with a similar scope can continue to improve optimization with a focus on algorithms related to analysis layout studies. Besides that, it can focus more on automation in the segmentation process of each character.


Keywords


Handwriting; pre-processing; image cleaning; skew correction; CNN.

Full Text:

PDF

References


A. Qaroush, B. Jaber, K. Mohammad, M. Washaha, E. Maali, and N. Nayef, “An efficient, font independent word and character segmentation algorithm for printed Arabic text,†Journal of King Saud University - Computer and Information Sciences, 2019, doi: 10.1016/j.jksuci.2019.08.013.

A. Qaroush, A. Awad, M. Modallal, and M. Ziq, “Segmentation-based, omnifont printed Arabic character recognition without font identification,†Journal of King Saud University - Computer and Information Sciences, 2020, doi: 10.1016/j.jksuci.2020.10.001.

A. Garg, M. K. Jindal, and A. Singh, “Offline handwritten Gurmukhi character recognition: k-NN vs. SVM classifier,†International Journal of Information Technology (Singapore), vol. 13, no. 6, pp. 2389–2396, Dec. 2021, doi: 10.1007/s41870-019-00398-4.

A. Garg, M. K. Jindal, and A. Singh, “Degraded offline handwritten Gurmukhi character recognition: study of various features and classifiers,†International Journal of Information Technology (Singapore), vol. 14, no. 1, pp. 145–153, Feb. 2022, doi: 10.1007/s41870-019-00399-3.

S. D. Pande et al., “Digitization of handwritten Devanagari text using CNN transfer learning – A better customer service support,†Neuroscience Informatics, vol. 2, no. 3, p. 100016, Sep. 2022, doi: 10.1016/j.neuri.2021.100016.

J. Gan, W. Wang, and K. Lu, “Compressing the CNN architecture for in-air handwritten Chinese character recognition,†Pattern Recognit Lett, vol. 129, pp. 190–197, Jan. 2020, doi: 10.1016/j.patrec.2019.11.028.

R. Dey and R. C. Balabantaray, “An efficient feature representation strategy for offline computer synthesized font characters using multivariate filter based feature selection technique,†International Journal of Information Technology (Singapore), vol. 13, no. 5, pp. 1733–1743, Oct. 2021, doi: 10.1007/s41870-021-00780-1.

S. Beier and C. A. T. Oderkerk, “High letter stroke contrast impairs letter recognition of bold fonts,†Appl Ergon, vol. 97, Nov. 2021, doi: 10.1016/j.apergo.2021.103499.

N. R. Soora and P. S. Deshpande, “A novel local skew correction and segmentation approach for printed multilingual Indian documents,†Alexandria Engineering Journal, vol. 57, no. 3, pp. 1609–1618, Sep. 2018, doi: 10.1016/j.aej.2017.06.010.

J. B. Bernard and E. Castet, “The optimal use of non-optimal letter information in foveal and parafoveal word recognition,†Vision Res, vol. 155, pp. 44–61, Feb. 2019, doi: 10.1016/j.visres.2018.12.006.

A. Lawgali, “A Survey on Arabic Character Recognition,†International Journal of Signal Processing, Image Processing and Pattern Recognition, vol. 8, no. 2, pp. 401–426, Feb. 2015, doi: 10.14257/ijsip.2015.8.2.37.

I. S. I. Abuhaiba, “Skew Correction of Textural Documents,†J. King Saud Univ, vol. 15, pp. 67–86, 2003.

A. Boukharouba, “A new algorithm for skew correction and baseline detection based on the randomized Hough Transform,†Journal of King Saud University - Computer and Information Sciences, vol. 29, no. 1, pp. 29–38, Jan. 2017, doi: 10.1016/j.jksuci.2016.02.002.

A. A. A. Ali and S. Mallaiah, “Intelligent handwritten recognition using hybrid CNN architectures based-SVM classifier with dropout,†Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 6, pp. 3294–3300, 2022, doi: 10.1016/j.jksuci.2021.01.012.

S. A. B. Haji, A. James, and S. Chandran, “A Novel Segmentation and Skew Correction Approach for Handwritten Malayalam Documents,†in Procedia Technology, Elsevier BV, 2016, pp. 1341–1348. doi: 10.1016/j.protcy.2016.05.140.

H. el Abed and V. Märgner, “Comparison of Different Pre-processing and Feature Extraction Methods for Offline Recognition of Handwritten Arabic Words.†[Online]. Available: www.ifnenit.com

M. M. Jadoon, Q. Zhang, I. U. Haq, S. Butt, and A. Jadoon, “Three-Class Mammogram Classification Based on Descriptive CNN Features,†Biomed Res Int, pp. 1–12, 2017.

H. Li, L. Zhang, M. Jiang, and Y. Li, “Multi-focus image fusion algorithm based on supervised learning for fully convolutional neural network,†Pattern Recognit Lett, vol. 141, pp. 45–53, Dec. 2020, doi: 10.1016/j.patrec.2020.11.014.

C. Zuo, L. Han, P. Tao, and X. L. Meng, “Livestock Detection Based on Convolutional Neural Network,†in ACM International Conference Proceeding Series, Association for Computing Machinery, Aug. 2020, pp. 1–6. doi: 10.1145/3425577.3425578.

H. Pesch, M. Hamdani, J. Forster, and H. Ney, “Analysis of pre-processing techniques for Latin handwriting recognition,†in Proceedings - International Workshop on Frontiers in Handwriting Recognition, IWFHR, 2012, pp. 280–284. doi: 10.1109/ICFHR.2012.179.

S. Eswar, Noise Reduction and Image Smoothing Using Gaussian Blur. Electrical Engineering, 2015.

S. Uyun, S. Hartati, A. Harjoko, and L. Choridah, “A Comparative Study of Thresholding Algorithms on Breast Area and Fibroglandular Tissue,†IJACSA) International Journal of Advanced Computer Science and Applications, vol. 6, no. 1, pp. 120–124, 2015, [Online]. Available: www.ijacsa.thesai.org

Y. Siti Ambarwati and S. Uyun, “Feature Selection on Magelang Duck Egg Candling Image Using Variance Threshold Method,†in 2020 3rd International Seminar on Research of Information Technology and Intelligent Systems, ISRITI 2020, 2020. doi: 10.1109/ISRITI51436.2020.9315486.

X. Yang, X. Shen, J. Long, and H. Chen, “An Improved Median-based Otsu Image Thresholding Algorithm,†AASRI Procedia, vol. 3, pp. 468–473, 2012, doi: 10.1016/j.aasri.2012.11.074.

S. L. S. Abdullah, H. Hambali, and N. Jamil, “Segmentation of natural images using an improved thresholding-based technique,†Procedia Eng, vol. 41, no. Iris, pp. 938–944, 2012, doi: 10.1016/j.proeng.2012.07.266.

U. Pal, S. Sinha, and B. B. Chaudhuri, “Multi-Oriented Text lines Detection and Their Skew Estimation,†in Third Indian Conferenceon Computer Vision, Graphics and Image Processing, 2002, pp. 270–275.

P. S. Vikhe and V. R. Thool, “Mass Detection in Mammographic Images Using Wavelet Processing and Adaptive Threshold Technique,†J Med Syst, vol. 40, no. 4, pp. 1–16, 2016, doi: 10.1007/s10916-016-0435-3.

W. Wiharto, E. Suryani, and Y. R. Putra, “Classification of blast cell type on acute myeloid leukemia (AML) based on image morphology of white blood cells,†Telkomnika (Telecommunication Computing Electronics and Control), vol. 17, no. 2, pp. 645–652, Apr. 2019, doi: 10.12928/TELKOMNIKA.V17I2.8666.

P. Saragiotis and N. Papamarkos, “Local skew correction in documents,†Intern J Pattern Recognit Artif Intell, vol. 22, no. 4, pp. 691–710, Jun. 2008, doi: 10.1142/S0218001408006417.

A. E. Minarno and N. Suciati, “Batik Image Retrieval Based on Color Difference Histogram and Gray Level Co-Occurrence Matrix,†TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 12, no. 3, pp. 597–604, 2014, doi: http://dx.doi.org/10.12928/telkomnika.v12i3.80.

S. Uyun and E. Sulistyowati, “Feature selection for multiple water quality status: Integrated bootstrapping and SMOTE approach in imbalance classes,†International Journal of Electrical and Computer Engineering, vol. 10, no. 4, pp. 4331–4339, 2020, doi: 10.11591/ijece.v10i4.pp4331-4339.

A. Alaei, P. Nagabhushan, U. Pal, and F. Kimura, “An Efficient Skew Estimation Technique for Scanned Documents:An Application of Piece-wise Painting Algorithm,†Journal of Pattern Recognition Research , vol. 1, pp. 1–14, 2016, [Online]. Available: www.jprr.org

T. A. Jundale and R. S. Hegadi, “Skew detection and correction of Devanagari script using Hough transform,†in Procedia Computer Science, Elsevier B.V., 2015, pp. 305–311. doi: 10.1016/j.procs.2015.03.147.

U. Pal, M. Mitra, and B. B. Chaudhuri, “Multi-skew detection of Indian script documents,†in International Conference on Document Analysis and Recognition, 2001, pp. 292–296. [Online]. Available: www.jatit.org

T. Jundale and R. Hegadi, “Offline Handwritten Signature Recognition View project International Conference on Recent Trends in Image Processing & Pattern Recognition View project Research Survey on Skew Detection of Devanagari Script,†NCKITE, 2015. [Online]. Available: https://www.researchgate.net/publication/280076252