ON INFORMATICS VISUALIZATION A Framework for Personalized Training at Home Based on Motion Capture

— As the number of single-person households continues to increase, contents related to exercise at home are increasing. Therefore, this paper proposes a framework for efficient home fitness. The home fitness framework proposed in this paper is based on a method of acquiring expert motion information and providing it to users. To this end, content creation and provision are implemented by using Kinect and Unreal game engine. In addition, it is possible to provide customized home fitness in consideration of the user's athletic ability. The proposed system first captures the expert's motion and stores it. We propose a method that can efficiently store stop motion and dynamic motion storage. In the case of each user, since there is a difference in exercise ability for each individual, an adverse effect may occur if an excessively accurate motion is requested. Therefore, to solve these problems, we present the parts that can be considered for each joint. For the exercise motion provided by the expert, a method was provided that allows the user to adjust the degree of matching for each joint in consideration of user's own exercise ability. That is, joint parts that do not require exact matching could be completely excluded from matching. For parts that would be subject to large changes, a range of errors is specified. As the training progresses, the error range is reduced, and the excluded parts are presented to be matched gradually. Such adjustments are made based on expert feedback. In this way, it was possible to improve the exercise effect gradually. This paper proposes an effective method for personalized home fitness according to the user's athletic ability. This will apply to various fields besides home fitness.


I. INTRODUCTION
As the number of single-person households increases [1], the popularity of home fitness, an individual exercise, is increasing [2]- [6]. Various devices for personal exercise are emerging [7]- [9], and various contents are being produced accordingly [10]- [13]. In order to increase the effect of exercise, game-type content is mainly produced [14]- [16], and it is more important to personal progress rather than correct posture. Even if the posture presented by the content and the posture of the user are different, the content proceeds by being similar or in response to a sensor value of a specific device. In addition, since each user proceeds regardless of their physical ability, it is difficult to see the exercise effect, and in some cases, adverse effects may occur for each part. Correct posture is necessary for proper exercise [16], [17]. For this, professional trainers recommend receiving personal PT (Personal Training).
However, most are faced with expensive costs and the time problem of meeting an expert. In order to solve the above problems, we propose a framework for personalized home fitness. Motions of 25 joints of the user are captured by using Kinect [18] and home fitness is conducted based on this data. By this, user can exercise by comparing an individual's posture with an expert in detail, and user can maximize the effect of exercise for each part. The error range can be adjusted according to the individual's different athletic abilities, so users can exercise according to their own abilities.
In addition, online PT is possible by storing user data, sharing with experts, and receiving feedback. Furthermore, using this, it provides an environment where users and users can share data and practice various postures such as exercise, acting, dancing, and yoga [19]- [21]. It can be extended to various fields, and training at home is possible regardless of cost, time, or space. In addition, users and experts can interact by sharing sessions in the server to share data and receive realtime feedback [22], [23].

II. MATERIALS AND METHODS
This section describes in detail the method for accurate motion capture regardless of the distance and angles of the camera. Data sharing in the proposed framework is also described.

A. Motion Capture System
Devices manufactured for motion capture include Vicon, Xsens, Capture.U, Kinect v2, Azure Kinect, and Webcam (OpenCV). In the case of Vicon, Xsens, and Capture.U, it is expensive for individuals to use in terms of cost, and there is a hassle of wearing a device. In addition, the data is heavy due to the tracking of many joints, requiring high specifications.
Motion capture system using RGB cameras such as webcam provides unsafe data in postures other than the front and has fewer traceable bones. Also, it is difficult to output precise data because FPS is much lower than that of PC specifications. Azure Kinect covers the problems of the above systems [24], but its penetration rate is low. Kinect v2 shown in the Fig. 1, is a device that conforms to the motion capture system of the personalized home fitness framework proposed in this paper when considering price, convenience, specifications, data stability, and penetration rate [25], [26].  Real-time motion capture data must be saved in order for experts and users to share data. To this end, this paper proposes the array value method, which is a fixed data storage method to maintain a specific posture, and the record animation method [30], which is a storage method that the continuous change of posture must follow. The array value storage method is used for fitness and yoga, where a fixed posture needs to be maintained. Instead of comparing the posture of the expert and the user's posture in real-time, the goal is to maintain the posture for the purpose of a specific posture. Accordingly, since there is no need to store all data, the joint transform value of a specific posture of the expert is stored and compared with the user.
For the record animation method, the proposed framework uses the sequence record system supported by Unreal Engine 4, as shown in Fig. 5. At first, the skeletal mesh of the character to be captured is added to the sequence record. A class variable of skeletal mesh component must exist among record variables in the value of class and properties to record.
The recoded skeletal mesh data is saved in animation format during the recoding period. This animation file is compared with the user's real-time motion capture data and used to compare dynamic posture data such as dancing, acting, and yoga. The saved animation can be viewed by playing, as shown in Fig. 6. When exercising, each individual has areas that are uncomfortable to exercise. If the user forcefully exercises in an area that is uncomfortable to exercise in order to maintain a professional-like posture, adverse effects can occur. For this case, comparing joint checking option is provided as shown in Fig. 7. This option allows the user to designate the necessary and uncomfortable exercise areas, so the designated exercise area is excluded from data comparison when comparing postures. Users freely exercise the excluded areas and prevent adverse effects. As shown in Fig. 8, the joints (yellow circles on the left side) excluded from the trainer motion were made to be recognized by the system as correct posture even if the user did not perform an exact match (blue circles on the right side). 2) Motion data compensation: When an expert or user performs motion capture by using Kinect, an error occurs in the data comparison value depending on the environment, such as the distance, angle, and location of the camera. It can be solved by perfectly matching the camera positions of users and experts, but it is inefficient in terms of differences in personal space installation location and convenience. In order to solve this problem, we propose a method of solving the problem according to the camera environment by using the zaxis (height) during real-time comparison.
When an expert and a user install a motion capture camera, the distance between the user and camera may differ from that of an expert case. As the camera distance gets closer, the angle of the joint recognized by the camera widens, making precise data comparison impossible, as shown in Fig. 9. Even if the expert shares the correct posture, many errors occur in data comparison if the user is not on the same distance line as the expert. When the camera's distance is very close, it is difficult to compare the data because the motion capture data is shaken. In addition, if an expert installs a camera closely and shares data, the ratio of a person's arm and leg may increase excessively, resulting in a state where data cannot be compared.
Even if the distance between the camera installed by the expert and the user is the same, if the camera angle is different, the angle of the joint narrows and data error occurs, as shown in Fig. 10. Joint tracking becomes impossible if one arm or leg is invisible due to a different camera angle, so random data values can be output. In addition, if an expert changes the camera angle to store and share data, the user must match the joint through excessive movement, which can adversely affect. To resolve the factors of camera distance and user position, the proposed system fixes the position of the Unreal Engine character. The highest route of the character skeleton is fixed to 0,0,0 on the world(map), and the location value of the retargeted character is constantly output during motion capture. A method using the Z-axis height is used to solve the problem of camera angle. With the highest root of Unreal Engine 4 fixed, the location value of the Z-axis compares the data of the expert and the user to solve the location data error of the X and Y axes according to the angle, as shown in Fig.  11. It may be difficult for the user to follow the expert's actions completely, and if the user forcefully follows the movement regardless of the difference in individual physical ability, the adverse effect of exercise may occur. Therefore, for usercustomized fitness training, it was allowed to set the error range according to the user's ability, as shown in Fig. 12. Fig. 12 Specifying motion error range In the proposed system, the user can follow the expert's motion and check the difference in motion with a graph. When specifying the margin of error, the difference in physical abilities between experts and individuals is considered. Using this, the user increases the exercise effect by matching the insufficient area and gradually reduces the error range to increase the detail of the motion.

B. Data Sharing Framework
Experts and users register their ID in the server database through membership registration. Experts upload their motion capture data and share the data for training with users. For personal training, users receive motion capture data of experts, compare it with their real-time motion capture data, and train. Figure 13 shows how to share data through the server. For customized training, feedback is essential. Users can upload their motion capture data to the feedback storage space of the expert database. Experts download and analyze user data uploaded to their feedback storage space and provide feedback to the individual user. In addition, a free data sharing space is created in the database so that data can be shared among general users. Through this, the exercise methods of each user are shared, and exercise effects can be increased. Figure 14 shows the structure of the entire database for the proposed framework.
For real-time training, a multi-play session space is created [29]. Here, real-time education between experts and users and real-time data sharing between users and users can be achieved. For real-time training between experts and users, the expert becomes a server, and the room is terminated at the end of the training. Experts can open a server through their PC and set the number of people. Using the Unreal Engine character movement component, motion animation data is networked by experts and users in real-time. For safety when sharing real-time data among users, the server is implemented as a dedicated server that operates a server separately to prevent the session from being terminated. As shown in Fig. 15, the home fitness framework proposed in this paper is a method that can enhance the exercise effect by comparing the motion capture data of experts and the realtime exercise data of users. At this time, by adjusting the movement of the joints within the range of the user's physical ability, it is possible to increase the exercise effect according to the user.
In order to verify the effectiveness of the motion-capturebased customized training, users are divided into two groups, and the training results are compared. The expert's squat posture is motion captured for home fitness test and all joint values are averaged. The first group participating in the test performed the squat position 10 times while watching only the video of the expert. The second group performs 10 squats while viewing motion capture data. Both groups proceed with motion capture and average the data correspondence with experts. Experimental results are presented in Fig. 16 and 17.
From analyzing the tested data, the user who performed the squat while watching the video only needs a large number of repetitions of 8 to 9 times before maintaining the correct posture on average. However, the user who performed the squat while viewing the motion capture data needs a relatively small number of times 2-3 times to maintain the correct posture. Through this, it is proved that when training while viewing one's motion capture data during home fitness, there is a high effect on maintaining the correct posture. In addition, the proposed framework has the advantage that joints can be limited according to the user's ability, and the exercise intensity can be gradually adjusted by setting an error range for each selected joint.  It is very important to maintain a correct posture to increase the exercise effect. As the demand for home fitness increases, many exercise contents have been proposed that follow the posture of experts. However, if the posture of an expert forcibly followed without considering the different physical abilities of each individual, the adverse effect of exercise may occur. This paper proposes a personalized training framework method by using Unreal game engine and Kinect to solve this problem. The proposed framework uses the Kinect to acquire expert motion data and shares it through a network so that users can use it for home fitness. Also, two ways to save motion capture data are proposed. The array value method is effectively used for fixed movements, yoga, and fitness. The record animation method effectively stores dynamic data such as dancing and acting. In order to provide personalized training content, the user's athletic ability was also considered.
By designating the error range, exercise is recommended as much as the user's personal physical ability, reducing the error range and increasing the exercise effect. The error range is calculated for each joint, and the user can exercise while checking the areas that are lacking and matching the data values. In addition, users can check the areas users are uncomfortable with and exclude them from the data comparison. Through this, even if the posture with the expert does not perfectly match, training for each area users need is possible. It has the advantage that it is possible to provide customized training content for each user by changing the limit for individual joints and error range settings for each joint and gradually improving the exercise effect. The proposed framework can be effectively applied not only to individual movements but also to group affairs.