Real-Time Digital Assistance for Exercise: Exercise Tracking System with MediaPipe Angle Directive Rules
DOI: http://dx.doi.org/10.62527/joiv.8.4.2993
Abstract
This paper focuses on developing an exercise tracking system capable of recognizing simple exercises, such as push-ups, pull-ups, and sit-ups, with high accuracy, leveraging human pose estimation techniques to enhance prediction performance. Exercise tracking can help users to perform workouts correctly and improve overall physical and mental health. The system utilizes the HSiPu2 dataset for training and evaluation, employing MediaPipe as the human pose estimation input and a Multi-Layer Perceptron (MLP) model for exercise recognition. Initially, a baseline MLP with three layers was implemented, followed by an improved expand-shrink MLP architecture designed to enhance model performance. The results demonstrate that the expand-shrink MLP model has achieved a 16% higher accuracy than the baseline, showcasing its effectiveness in accurately recognizing simple exercises based on pose estimation data. This advancement highlights the potential of the model to support a broader range of exercise types, offering a robust solution for monitoring workouts. The system provides meaningful feedback to users by ensuring accurate exercise recognition and promoting safe and effective physical activity. Future research can explore integrating this system with real-time feedback mechanisms, enabling users to receive immediate corrections during workouts. Expanding the dataset to include diverse exercise routines, including complex and dynamic movements, could enhance the system’s applicability. These developments would pave the way for more comprehensive and practical exercise-tracking solutions, supporting individuals to maintain a healthy lifestyle and improving the accessibility of fitness technologies.
Keywords
Full Text:
PDFReferences
A. K. Srivastav, S. Khadayat and A. J. Samuel, "Mobile-based health apps to promote physical activity during COVID-19 lockdowns," J Rehabil Med Clin Commun, vol. 4, no. 2, pp.1-3 2021. doi: 10.2340/20030711-1000051
A. Bhatnagar and M. A. Nystoriak, "Cardiovascular Effects and Benefits of Exercise," Front Cardiovasc Med, vol. 5, no. 135, pp. 1-111, 2018. doi: 10.3389/fcvm.2018.00135
S. A. Saeed, K. Cunningham and R. M. Bloch, "Depression and Anxiety Disorders: Benefits of Exercise, Yoga, and Meditation," Am Fam Physician, vol. 99, pp. 620-627, 2019. doi: 10.12811/kber.2019.10135.
S. Oetoro, I. Permadhi and E. Sumarliah, "The impacts of mHealth technology and healthcare e-consultation on workout levels among obese and overweight people post-COVID-19," Kybernetes, vol. 52, no. 7, pp. 2288-2304, 2023. doi: 10.1108/K-08-2022-1099
F. Tlili, R. Haddad, R. Bouallegue and N. Mezghani, "A Real time Posture Monitoring System Towards Bad Posture," Wirel Pers Commun, vol. 120, pp. 1207-1227, 2021. doi: 10.1007/s11277-021-08511-2
Z. Ding, W. Li, P. Ogunbona and L. Qin, "A real-time webcam-based method for assessing upper-body postures," Mach Vis Appl, vol. 30, pp. 833-850, 2019. doi: 10.1007/s00138-019-01032-x
M. Al-Faris, J. Chiverton, D. Ndzi and A. I. Ahmed, "A Review on Computer Vision-Based Methods for Human Action Recognition," J Imaging, vol. 6, no. 6, pp. 1-46, 2020. doi: 10.3390/jimaging6060046
Y. Zhu, D. Wang, R. Zhao, Q. Zhang and A. Huang, "FitAssist: Virtual Fitness Assistant Based on WiFi," in 16th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous), New York, NY, USA, pp. 1-11; 2019. doi: 10.1145/3360774.3360784
N. Sonwani and A. Pegwar, "Auto_Fit: Workout Tracking using Pose-Estimation and DNN," Int J Eng Appl Sci Technol, vol. 5, no. 1, pp. 167-173, 2020. doi: 10.33564/IJEAST.2020.v05i01.033
B. Ferreira, P. M. Ferreira, G. Pinheiro, N. Figueiredo, F. Carvalho, P. Menezes and J. Batista, "Deep learning approaches for workout repetition counting and validation," Pattern Recogn Lett, vol. 151, pp. 259-266, 2021. doi: 10.1016/j.patrec.2021.08.009
A. Patil, D. Rao, K. Utturwar, T. Shelke and E. Sarda, "Body Posture Detection and Motion Tracking using AI for Medical Exercises and Recommendation System," in International Conference on Automation, Computing and Communication, pp. 1-6, 2022. doi: 10.1051/itmconf/20224403043
T. Khan, "An Intelligent Baby Monitor with Automatic Sleeping Posture Detection and Notification," AI, MDPI, Basel, Switzerland, vol. 2, pp. 290-306, 2021. doi: 10.3390/ai2020016
S. Dhulipala, F. F. Adedoyin and A. Bruno, "Sign and Human Action Detection Using Deep Learning," J Imaging, vol. 8, p. 1-34, 2022. doi: 10.3390/jimaging8070192
Y. Kong and Y. Fu, "Human Action Recognition and Prediction: A Survey," Int J Comput Vis, vol. 130, pp. 1366-1401, 2022. doi: 10.1007/s11263-022-01645-3
H. Hwang, C. Jang, G. Park, J. Cho and I. J. Kim, "ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications," IEEE Access, vol. 11, pp. 9279 - 9294 ,2021. doi: 10.1109/ACCESS.2021.3119050
X. B. Fu, S. L. Yue and D. Y. Pan, "Camera-based Basketball Scoring Detection Using Convolutional Neural Network," Int J Autom Comput, vol. 18(2), pp. 266-276, 2021. doi: 10.1007/s11633-020-1259-7
G. Taware, D. R. Kharat and P. Dhende, "AI-Based Workout Assistant and Fitness Guide," in 6th International Conference On Computing, Communication, Control and Automation, Pune, India, 2022, pp.1-9. doi: 10.26438/ijcse/v12i8.19
G. Güney, T. S. Jansen, S. Dill, J. B. Schulz, M. Dafotakis, C. H. Antink and A. K. Braczynski, "Video-Based Hand Movement Analysis of Parkinson Patients before and after Medication Using High-Frame-Rate Videos and MediaPipe," Sensors, vol. 22, no. 20, 7992, pp.1-15, 2022. Doi: 10.3390/s22207992
Y. L. Chang, C. S. Chan and P. Remagnino, "Action recognition on continuous video," Neural Comput Appl, vol. 33, pp. 1233-1243, 2021. doi: 10.1007/s00521-020-05405-x.
S. H. Cheng, M. A. Sarwar, Y. A. Daraghmi, T. U. İk and Y. L. Li, "Periodic Physical Activity Information Segmentation, Counting and Recognition From Video," IEEE Access, vol. 11, pp. 23019-23031, 2023. doi: 10.1109/ACCESS.2023.3254015
A. Depari, P. Ferrari, A. Flammini, S. Rinaldi and E. Sisinni, "Lightweight Machine Learning-Based Approach for Supervision of Fitness Workout," in IEEE Xplore Sensors Applications Symposium (SAS), Brescia, Italy, 2019. doi: 10.1016/j.measurement.2019.07.038
J. S. Suraiya Parveen, "A Motion Detection System in Python and Opencv," in IEEE Xplore Third International Conference on Intelligent Communication Technologies and Virtual Mobile Networks, New Delhi, India, 2021. doi: 10.1109/icicv50876.2021.9388404 (sci-hub).
J. T. Monsalve, D. Arnold, W. J. Yi and J. Saniie, "Design Flow of Wearable Internet of Things (IoT) Smart Workout Tracking System," in IEEE Xplore, Chicago IL, U.S.A., 2019. doi: 10.1109/EIT.2019.8833917
A. Nagarkoti, R. Teotia, A. K. Mahale and P. K. Das, "Realtime Indoor Workout Analysis Using Machine Learning & Computer Vision," in IEEE Xplore 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2019. pp. 1440-1443, doi: 10.1109/EMBC.2019.8856547
Z. Wu and H. Du, "Research on Human Action Feature Detection and Recognition Algorithm Based on Deep Learning," Mob Inf Syst, vol. 2022, pp. 1-12, 2022. doi:10.1155/2022/4652946
C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee, W.-T. Chang, W. Hua, M. Georg and M. Grundmann, “MediaPipe: A Framework for Building Perception Pipelines,”, arXiv, vol. 1906.08172v1, 2019. doi: 10.48550/arXiv.1906.08172
R. K. Y. Chang, S. H. Lau, K. S. Sim and M. S. M. Too, "Kinect-based framework for motor rehabilitation," 2016 International Conference on Robotics, Automation and Sciences (ICORAS), Melaka, Malaysia, 2016, pp. 1-4, doi: 10.1109/ICORAS.2016.7872606.
K.L. Lew, K.S. Sim, S.C. Tan, and Fazly Salleh Abas, "Virtual reality post stroke upper limb assessment using Unreal Engine 4," Eng Lett, vol. 29, no. 4, pp. 1511-1523, 2021. doi: 10.1111/j.1756-2228.2020.00029.x.
C.C. Lim, K.S. Sim, and C.K. Toa, "Development of Visual-based Rehabilitation Using Sensors for Stroke Patient," Int. J. Robot. Autom. Sci., vol. 2, no. 1, pp. 25-30, MMU Press, 2020. Doi: 10.33093/ijoras.2020.2.4
M. Elsayed, K.S. Sim, and S.C. Tan, "A Novel Approach to Objectively Quantify the Subjective Perception of Pain through Electroencephalogram Signal Analysis," IEEE Access, vol. 8, no. 1, pp. 199920-199930, 2020. doi: 10.1109/ACCESS.2020.3032153.
K. L. Lew, K. S. Sim, and Z. Ting, "Deep Learning Approach EEG Signal Classification," Int. J. Inform. Vis., vol. 8, no. 3-2, pp. 1693-1702, Nov. 2024,doi: 10.62527/joiv.8.3-2.2959
S. Ranjit, K. S. Sim, R. Besar, and C.P. Tso, "Motion Estimation in Medical Imaging," in 4th Kuala Lumpur International Conference on Biomedical Engineering 2008: BIOMED 2008, 25-28 June 2008, Kuala Lumpur, Malaysia, pp. 603-606, Springer Berlin Heidelberg, 2008. doi: 10.1007/978-3-540-69139-6_151
J.-P. Cheng and S.-C. Haw, “Mental Health Problems Prediction Using Machine Learning Techniques”, Int. J. Robot. Autom. Sci., vol. 5, no. 2, pp. 59–72, Sep. 2023. doi: 10.33093/ijoras.2023.5.2.7
C. Palanisamy, "Smart Manufacturing with Smart Technologies – A Review," Int. J. Robot. Autom. Sci., vol. 5, no. 2, pp. 85-88, Sep. 2023. doi: 10.33093/ijoras.2023.5.2.10
Y. S. Bong and G. C. Lee, “A Contactless Visitor Access Monitoring System”, Int. J. Robot. Autom. Sci. , vol. 3, pp. 34–42, Mar. 2024. doi: 10.33093/ijoras.2021.3.6
K. Xia, H. Wang, M. Xu, Z. Li, S. He and Y. Tang, “Racquet sports recognition using a hybrid clustering model learned from integrated wearable sensor. Sensors, 20(6), p.1638. 2020, doi: https://doi.org/10.3390/s20061638
K.S. Sim, N.S. Kamel and H.T. Chuah, “A real-time image dynamic range compensation for scanning electron microscope imaging system,” Scanning, 27, pp.199-207. 2005, doi: 10.1002/sca.4950270407
C. C. Lim, K. S. Sim, C. K. Toa, “Development of Visual-based Rehabilitation Using Sensors for Stroke Patient,” Int. J. Robot. Autom. Sci., vol. 2, pp. 25–30, Oct. 2020, doi: 10.33093/ijoras.2020.2.4.
K. B. Gan, C. H. Chen, and N. A. Abd Aziz, “Upper Limbs Extension and Flexion Angles Calculation and Visualization Using Two Wearable Inertial Measurement Units,” Int. J. Robot. Autom. Sci. , vol. 4, pp. 1–7, Jul. 2022, doi: 10.33093/ijoras.2022.4.1.
R. G. Candraningtyas, A. P. Yunus, and Y. H. Choo, "Human fall motion prediction: A review," Int. J. Robot. Autom. Sci., vol. 6, no. 2, pp. 52–58, 2024, doi:10.33093/ijoras.2024.6.2.8
J. He, J. He, S.L. Baxter, J. Xu, X. Zhou and K. Zhang (2021). The Practical Implementation of Artificial Intelligence Technologies in Medicine. Nat Med, 25(1), 30-36. doi: 10.1038/s41591-018-0307-0
J. Armando Vicente-Martínez, M. Márquez-Olivera, A. García-Aliaga, and V. Hernández-Herrera, “Adaptation of YOLOv7 and YOLOv7_tiny for Soccer-Ball Multi-Detection with DeepSORT for Tracking by Semi-Supervised System,” Sensors, 2023, 23(21), 8693; pp. 1-29; doi: 10.3390/s23218693
M. Too, S. H. Lau, C. K. Tan, “Validity and Reliability of a Conceptual Framework on Enhancing Learning for Students via Kinect: A Pilot Test,”, Int. J. Robot. Autom. Sci., vol. 6, no. 1 ,pp. 59 – 63, 2024, doi: 10.33093/ijoras.2024.6.1.8
Z. Y. Lim, C. K. Toa, E. Rao, and K. S. Sim, “Development of Augmented Reality Based Applications for Brain Memory Training,” Int. J. Robot. Autom. Sci., vol. 5, no. 1, pp. 13–20, Apr. 2023, doi: https://doi.org/10.33093/ijoras.2023.5.1.3.
F. Giakoni-Ramírez, A. Godoy-Cumillaf, S. Espoz-Lazo, D. Duclos-Bastias, and P. del Val Martín, “Physical Activity in Immersive Virtual Reality: A Scoping Review,” Healthcare 2023, 11(11), 1553, pp.1-15, doi: 10.3390/healthcare11111553
W. X. Lim, C. K. Toa, and K. S. Sim, “The Application of Augmented Reality Platform for Chemistry Learning,” Int. J. Robot. Autom. Sci., vol. 5, no. 2, pp. 101–110, Sep. 2023, doi: 10.33093/ijoras.2023.5.2.13.
W. S. Lim, K. J. D, Lai, S. T. Lim, B. C. Yeo, “Vision-based Egg Grading System using Support Vector Machine,” Int. J. Robot. Autom. Sci., vol. 6, no. 1, pp.13–19, doi: 10.33093/ijoras.2024.6.1.3
Z. Quan and L. Pu, “An improved accurate classification method for online education resources based on support vector machine (SVM): Algorithm and experiment,” Educ Inf Technol (Dordr), vol. 28, no. 7, pp. 8097–8111, Jul. 2023, doi: 10.1007/s10639-022-11514-6
H. A. Goh, C. W. Chong, R. Besar, F. S. Abas, K. S. Sim, 2009, “Translation and scale invariants of Hahn moments,” Int J Image Graph, Volume 9, Issue 2, pp. 271-285, World Scientific Publishing Company. doi: 10.1142/S0219467809003435
R. G. Candraningtyas, A. P. Yunus, and Y. H. Choo, "Human fall motion prediction: A review," Int. J. Robot. Autom. Sci., vol. 6, no. 2, pp. 52–58, 2024. doi: 10.33093/ijoras.2024.6.2.8
F. Abuhoureyah, K. S. Sim and Y. Chiew Wong, "Multi-User Human Activity Recognition Through Adaptive Location-Independent WiFi Signal Characteristics," in IEEE Access, vol. 12, pp. 112008-112024, 2024, doi: 10.1109/ACCESS.2024.3438871