Automatic Weight of Color, Texture, and Shape Features in Content-Based Image Retrieval Using Artificial Neural Network

Akmal Akmal - Institut Teknologi Bandung, Bandung, 40132, Indonesia
Rinaldi Munir - Institut Teknologi Bandung, Bandung, 40132, Indonesia
Judhi Santoso - Institut Teknologi Bandung, Bandung, 40132, Indonesia


Citation Format:



DOI: http://dx.doi.org/10.30630/joiv.7.3.1184

Abstract


Image retrieval is the process of finding images in the database that are similar to the query image by measuring how close the feature values of the query image are to other images. Image retrieval is currently dominated by approaches that combine several different representations or features. The optimal weight of each feature is needed in combining the image features such as color features, texture features, and shape features. In this study, we use a multi-layer perceptron artificial neural network (MLP) method to obtain feature weights automatically and simultaneously look for optimal weights. The color moment is used to find nine color features, Gray Level Co-occurrence Matrix (GLCM) to find four texture features, and Hu Moment to find seven shape features totaling 20 neurons and all of these features become the input layer in our MLP network. Three neurons in output layers become the automatic weight of each feature. These weights are used to combine each feature's role in obtaining the relevant image. Euclidean Distance is used to measure similarity. The average precision values obtained using automatic feature weights are 93.94% for the synthetic dataset, 91.19% for the Coil-100 dataset, and 54.31% for the Wang dataset. These results have an average difference of 5.06% with the target so automatic feature weighting works well. This value is obtained at a hidden layer size of 11 and a learning rate of 0.1. In addition, the use of automatic feature weighting gives more accurate results compared to manual feature weighting.

Keywords


image retrieval; color moment; hu moment; gray level co-occurrence matrix; multi-layer perceptron

Full Text:

PDF

References


A. Latif et al., “Content-based image retrieval and feature extraction: A comprehensive review,†Math. Probl. Eng., vol. 2019, 2019, doi: 10.1155/2019/9658350.

S. Chavda and G. Mahesh, “Content-Based Image Retrieval: The State of the Art,†Int. J. Next-Generation Comput., vol. 10, no. December, pp. 193–212, 2019.

I. M. Hameed, S. H. Abdulhussain, and B. M. Mahmmod, “Content-based image retrieval: A review of recent trends,†Cogent Eng., vol. 8, no. 1, 2021, doi: 10.1080/23311916.2021.1927469.

H. Ji-Zhao, G.-H. Liu, and S.-X. Song, “Content-based image retrieval using color volume histograms,†Int. J. Pattern Recognit. Artif. Intell., vol. 33, no. 9, p. Article ID 6283987, 2019.

B. Patel, K. Yadav, and D. Ghosh, “Current Trend and Methodologies of Content-Based Image Retrieval: Survey,†2021, doi: https://doi.org/10.1007/978-981-15-6707-0_64.

X. Zenggang, T. Zhiwen, and C. Xiaowen, “Research on Image Retrieval Algorithm Based on Combination of Color and Shape Features,†J. Signal Process. Syst., vol. 93, no. 2, pp. 139-146., 2019.

J. Pradhan, A. K. Pal, H. Banka, and P. Dansena, “Fusion of region based extracted features for instance-and class-based CBIR applications,†Appl. Soft Comput., vol. 102, p. 107063, 2021.

K. T. Ahmed, S. Ummesafi, and A. Iqbal, “Content based image retrieval using image features information fusion,†Inf. Fusion, vol. 51, pp. 76–99, 2019.

H. Wang, Z. Li, Y. Li, B. Gupta, and C. Choi, “Visual saliency guided complex image retrieval,†Pattern Recognit. Lett. Elsevier, vol. 130, pp. 64–72, 2020.

D. Niu, X. Zhao, X. Lin, and C. Zhang, “A novel image retrieval method based on multi-features fusion,†Signal Process. Image Commun., vol. 87, p. 115911, 2020, doi: 10.1016/j.image.2020.115911.

M. K. Alsmadi, “Content-Based Image Retrieval Using Color, Shape and Texture Descriptors and Features,†Arab. J. Sci. Eng., vol. 45, no. 4, pp. 3317–3330, 2020, doi: 10.1007/s13369-020-04384-y.

N. Kayhan and S. Fekri-Ershad, “Content based image retrieval based on weighted fusion of texture and color features derived from modified local binary patterns and local neighborhood difference patterns,†Multimed. Tools Appl., vol. 80, no. 21, pp. 32763–32790, 2021.

N. Varish, A modified similarity measurement for image retrieval scheme using fusion of color, texture and shape moments, vol. 81, no. 15. Multimedia Tools and Applications, 2022.

K. Chu and G. H. Liu, “Image Retrieval Based on a Multi-Integration Features Model,†Math. Probl. Eng., vol. 2020, 2020, doi: 10.1155/2020/1461459.

S. Tena, R. Hartanto, and I. Ardiyanto, “Content-based image retrieval for fabric images: A survey,†Indones. J. Electr. Eng. Comput. Sci., vol. 23, no. 3, pp. 1861–1872, 2021, doi: 10.11591/ijeecs.v23.i3.pp1861-1872.

U. A. Khan, A. Javed, and R. Ashraf, “An effective hybrid framework for content based image retrieval (CBIR),†Multimed. Tools Appl., vol. 80, no. 17, pp. 26911–26937, 2021, doi: 10.1007/s11042-021-10530-x.

M. Garg and G. Dhiman, “A novel content-based image retrieval approach for classification using GLCM features and texture fused LBP variants,†Neural Comput. Appl., vol. 33, no. 4, pp. 1311–1328, 2021.

M. Subramanian, V. Lingamuthu, C. Venkatesan, and S. Perumal, “Content-Based Image Retrieval Using Colour, Gray, Advanced Texture, Shape Features, and Random Forest Classifier with Optimized Particle Swarm Optimization,†Int. J. Biomed. Imaging, vol. 2022, 2022, doi: 10.1155/2022/3211793.

S. Jardim, J. António, C. Mora, and A. Almeida, “A Novel Trademark Image Retrieval System Based on Multi-Feature Extraction and Deep Networks,†J. Imaging, vol. 8, no. 9, 2022, doi: 10.3390/jimaging8090238.

N. K. Rout, M. Atulkar, and M. K. Ahirwal, “A review on content-based image retrieval system: Present trends and future challenges,†Int. J. Comput. Vis. Robot., vol. 11, no. 5, pp. 461–485, 2021, doi: 10.1504/IJCVR.2021.117578.

S. Lu and B. Wang, “An image retrieval algorithm based on improved color histogram,†2019, doi: 10.1088/1742-6596/1176/2/022039.

R. M. Bommisetty, O. Prakash, and A. Khare, Keyframe extraction using Pearson correlation coefficient and color moments, vol. 26, no. 3. Springer Berlin Heidelberg, 2020.

G. Xie, B. Guo, Z. Huang, Y. Zheng, and Y. Yan, “Combination of Dominant Color Descriptor and Hu Moments in Consistent Zone for Content Based Image Retrieval,†IEEE Access, vol. 8, pp. 146284–146299, 2020, doi: 10.1109/ACCESS.2020.3015285.

S. Deniziak and T. Michno, “New content based image retrieval database structure using query by approximate shapes,†Proc. 2017 Fed. Conf. Comput. Sci. Inf. Syst. FedCSIS 2017, vol. 11, pp. 613–621, 2017, doi: 10.15439/2017F457.

Z. Huang and J. Leng, “Analysis of Hu’s moment invariants on image scaling and rotation,†ICCET 2010 - 2010 Int. Conf. Comput. Eng. Technol. Proc., vol. 7, no. May 2010, 2010, doi: 10.1109/ICCET.2010.5485542.

A. Humeau-Heurtier, “Texture feature extraction methods: A survey,†IEEE Access, vol. 7, no. c, pp. 8975–9000, 2019, doi: 10.1109/ACCESS.2018.2890743.

A. Eleyan and H. Demirel, “Co-occurrence matrix and its statistical features as a new approach for face Recognition,†Turkish J. Electr. Eng. Comput. Sci., vol. 19, no. 1, 2011, doi: 10.3906/elk-0906-27.

Haralick, R.M and K. Shanmugam, “Texture features for image classification,†EEETrans. Syst. Man Cybern. SMC, vol. 3, no. 6, pp. 610–621, 1973.

M. Huang, H. Shu, Y. Ma, and Q. Gong, “Content-based image retrieval technology using multi-feature fusion,†Opt. - Int. J. Light Electron Opt., vol. 126, no. 19, pp. 2144–2148, 2015, doi: 10.1016/j.ijleo.2015.05.095.

O. I. Abiodun et al., “Comprehensive Review of Artificial Neural Network Applications to Pattern Recognition,†IEEE Access, vol. 7, pp. 158820–158846, 2019, doi: 10.1109/ACCESS.2019.2945545.

V. Ojha, A. Abraham, and V. Snášel, “Metaheuristic design of feedforward neural networks: a review of two decades of research,†Eng Appl Artif Intell, vol. 60, pp. 97–116, 2017.

A. A. Heidari, H. Faris, I. Aljarah, and S. Mirjalili, “An efficient hybrid multilayer perceptron neural network with grasshopper optimization,†Soft Comput., vol. 23, no. 17, pp. 7941–7958, 2019, doi: 10.1007/s00500-018-3424-2.