Data-Centric Learning Method for Synthetic Data Augmentation and Object Detection

Yeseong Park - Testworks, AI Research, 42-19, Baekjegobun-ro 41-gil, Songpa-gu, Seoul, Republic of Korea
Hyeongbok Kim - Testworks, AI Research, 42-19, Baekjegobun-ro 41-gil, Songpa-gu, Seoul, Republic of Korea
Yoon Jung Park - Testworks, AI Research, 42-19, Baekjegobun-ro 41-gil, Songpa-gu, Seoul, Republic of Korea
Changsin Lee - Testworks, AI Research, 42-19, Baekjegobun-ro 41-gil, Songpa-gu, Seoul, Republic of Korea
Jinsuk Lee - Testworks, AI Research, 42-19, Baekjegobun-ro 41-gil, Songpa-gu, Seoul, Republic of Korea


Citation Format:



DOI: http://dx.doi.org/10.30630/joiv.6.1-2.939

Abstract


This paper proposes a deep learning framework for decreasing large-scale domain shift problems in object detection using domain adaptation techniques. We have approached data-centric domain adaptation with Image-to-Image translation models for this problem. It is one of the methodologies that changes source data to target domain's style by reducing domain shift. However, the method cannot be applied directly to the domain adaptation task because the existing Image-to-Image model focuses on style translation. We solved this problem using the data-centric approach simply by reordering the training sequence of the domain adaptation model. We defined the features to be content and style. We hypothesized that object-specific information in images was more closely tied to the content than the style and thus experimented with methods to preserve content information before style was learned. We trained the model separately only by altering the training data. Our experiments confirmed that the proposed method improves the performance of the domain adaptation model and increases the effectiveness of using the generated synthetic data for training object detection models. We compared our approach with the existing single-stage method where content and style were trained simultaneously. We argue that our proposed method is more practical for training object detection models than others. The emphasis in this study is to preserve image content while changing the style of the image. In the future, we plan to conduct additional experiments to apply synthetic data generation technology to various other application areas like indoor scenes and bin picking.

Keywords


Domain adaptation; object detection; image-to-image translation; GAN.

Full Text:

PDF

References


Ian J. Goodfellow, Jean Pouget-Abadie, "Generative adversarial nets," Advances in neural information processing systems 27, June 2014

Poojan Oza, Vishwanath A. Sindagi, "Unsupervised domain adaptation of object detectors: A survey," May 2021

Jun-Yan Zhu, Taesung Park, "Unpaired image-to-image translation using cycle-consistent adversarial networks," Proceedings of the IEEE international conference on computer vision, March 2017

Ming-Yu Liu, Thomas Breuel, and Jan Kautz. "Unsupervised image-to-image translation networks," Advances in neural information processing systems 30, March 2017

Kyungjune Baek, Yunjey Choi, "Rethinking the truly unsupervised image-to-image translation," Proceedings of the IEEE/CVF International Conference on Computer Vision, June 2021

Sheng-Wei Huang, Che-Tsung Lin, "Auggan: Cross domain adaptation with gan-based data augmentation," Proceedings of the European Conference on Computer Vision (ECCV), 2018

Che-Tsung Lin, Sheng-Wei Huang, "GAN-based day-to-night image style transfer for nighttime vehicle detection," IEEE Transactions on Intelligent Transportation Systems 22.2, 2020, pp. 951-963.

Xue Bin Peng, Marcin Andrychowicz, "Sim-to-real transfer of robotic control with dynamics randomization," 2018 IEEE international conference on robotics and automation (ICRA), IEEE, October 2018

Han-Kai Hsu, Chun-Han Yao, "Progressive domain adaptation for object detection," Proceedings of the IEEE/CVF winter conference on applications of computer vision, October 2019

Zhiqiang Shen, Mingyang Huang, "CDTD: A large-scale cross-domain benchmark for instance-level image-to-image translation and domain adaptive object detection," International Journal of Computer Vision 129.3, November 2021, pp. 761-780

Chaoqi Chen, Zebiao Zheng, "Harmonizing transferability and discriminability for adapting object detectors," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, March 2020

Shan Lin, Fangbo Qin, "Lc-gan: Image-to-image translation based on generative adversarial network for endoscopic images," 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, March 2020

Zhenwei He, and Lei Zhang "Multi-adversarial faster-rcnn for unrestricted object detection," Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019

Kuniaki Saito, Yoshitaka Ushiku, "Strong-weak distribution alignment for adaptive object detection," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, April 2019

Peng Su, Kun Wang, "Adapting object detectors with conditional domain normalization," European Conference on Computer Vision. Springer, Cham, July 2020

Yangtao Zheng, Di Huang, "Cross-domain object detection through coarse-to-fine feature adaptation," Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, March 2020

Chang-Dong Xu, Xing-Ran Zhao, "Exploring categorical regularization for domain adaptive object detection," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, March 2020

Chaoqi Chen, Zebiao Zheng, "I3net: Implicit instance-invariant network for adapting one-stage object detectors," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, March 2021

Vibashan VS, Vikram Gupta, "Mega-cda: Memory guided attention for category-aware unsupervised domain adaptive object detection," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, March 2021

Yuhua Chen, Haoran Wang, "Scale-aware domain adaptive faster r-cnn," International Journal of Computer Vision 129.7, May 2021, pp. 2223-2243

Zhenwei He, and Lei Zhang, "Domain adaptive object detection via asymmetric tri-way faster-rcnn," European conference on computer vision. Springer, Cham, July 2020

Mehran Khodabandeh, Arash Vahdat, "A robust learning approach to domain adaptive object detection," Proceedings of the IEEE/CVF International Conference on Computer Vision, November 2019

Aruni RoyChowdhury, Prithvijit Chakrabarty, "Automatic adaptation of object detectors to new domains using self-training," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, April 2019

Seunghyeon Kim, Jaehoon Choi, "Self-training and adversarial background regularization for unsupervised domain adaptive one-stage object detection," Proceedings of the IEEE/CVF International Conference on Computer Vision, September 2019

Ganlong Zhao, Guanbin Li, "Collaborative training between region proposal localization and classification for domain adaptive object detection," European Conference on Computer Vision. Springer, Cham, September 2020

Petru Soviany, Radu Tudor Ionescu, "Curriculum self-paced learning for cross-domain object detection," Computer Vision and Image Understanding 204, 103166, January 2021

Jinhong Deng, Wen Li, "Unbiased mean teacher for cross-domain object detection," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2021

Taekyung Kim, Minki Jeong, "Diversify and match: A domain adaptive representation learning paradigm for object detection," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, May 2019

Adrian Lopez Rodriguez, and Krystian Mikolajczyk, "Domain adaptation for object detection via style consistency," November 2019

Qi Cai, Yingwei Pan, "Exploring object relation in mean teacher for cross-domain detection," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, December 2019

Minghao Xu, Hang Wang, "Cross-domain detection via graph-induced prototype alignment," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, March 2020

Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. "Image style transfer using convolutional neural networks," Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.

Xun Huang, and Serge Belongie "Arbitrary style transfer in real-time with adaptive instance normalization," Proceedings of the IEEE international conference on computer vision, July 2017

Tero Karras, Samuli Laine, and Timo Aila, "A style-based generator architecture for generative adversarial networks," Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, March 2019

Tero Karras, Samuli Laine, "Analyzing and improving the image quality of stylegan," Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, March 2020

Hyeongkeun Lee, Kyungmin Lee, "Applying FastPhotoStyle to Synthetic Data for Military Vehicle Detection," 2020 20th International Conference on Control, Automation and Systems (ICCAS). IEEE, 2020

David Kadish, Sebastian Risi, and Anders Sundnes Løvlie. "Improving object detection in art images using only style transfer," 2021 International Joint Conference on Neural Networks (IJCNN), IEEE, May 2021

Laurens van der Maaten, Geoffrey Hinton, "Visualizing Data using t-SNE," Journal of Machine Learning Research 9, November 2008

Shaoqing Ren, Kaiming He, "Faster r-cnn: Towards real-time object detection with region proposal networks," Advances in neural information processing systems 28, January 2016, pp. 91-99