HUMAN GAIT CYCLE ANALYSIS USING KINECT V2 SENSOR

: Examining a human movement can provide a wealth of information about a patient’s medical condition. The examination process can be used to diagnose abnormal changes (lesions), ability development and monitor the rehabilitation process of people with reduced mobility. There are several approaches to monitor people, among other things with sensors and various imaging and processing devices. In this case a Kinect V2 sensor and a self-developed LabView based application was used, to examine the movement of the lower limbs. The ideal gait pattern was recorded in the RoboGait training machine and the measured data was used to identify the phases of the human gait. During the evaluation, the position of the skeleton model, the associated body joints and angles can be calculated. The pre-recorded ideal and natural gait cycle can be compared. With the self-developed method the pre-recorded ideal and natural gait cycle can be compared and processed for further evaluation. The evaluated measurement data confirm that a reliable and mobile solution for gait analysis has been created.


Introduction
Analyzing the gait process can provide important data about the patient's condition in many ways [1]. It can be a diagnosis or the tracking of the rehabilitation progress and ability development. Conventionally, this type of examinations is performed by 4 P. MÜLLER, Á. SCHIFFER Pollack Periodica 15, 2020, 3 professionals with many years of experience and expertise. Basic movement patterns can be observed visually, but they also require the involvement of experts or physiotherapists [2]. To detect advanced patterns of movement, multiple data sources are required. The currently available solutions use various sensors. Certain measurement methods apply visual markers attached to the body or use wearable pressure-sensitive sensors or measure the muscle work [3], [4].
The recorded ElectroMyoGraphy (EMG) signals can be used for creating a wired solution for monitoring the motion of the human body. With developed solution the upper and lower extremities can be measured as well. By using this type of information, the researchers are able to create a decision support system, combined with machine learning [5], [6].
In the case of body-attached sensors, the sensors have to be fastened on specific positions of the body parts. The sensors are connected by wires to each other and the central processing device. The disadvantage of this kind of method is, that the examined person is restricted in movement by the attached devices; therefore the intended patterns of movement can be distorted and misobserved [4]. It is also possible, that the Signal to Noise Ratio (SNR) rises significantly, due loose wire interconnections. For these problems optical sensors can provide a solution, as they can identify the body joints based on visual markers [3]. For wireless, visual-only measurements an image sensor with high frame-rate is required, as well as to use of depth sensitive cameras. Measurements performed in this way, require special preparations and are excessively expensive.

Methodology of the measurement
During the measurement, a widely available Kinect V2, also known as Microsoft Kinect ONE sensor was used [7]. This type of sensor transmits 11 data streams in real time. The stream includes Red-Green-Blue (RGB) color video [8], depth image data and a skeleton stream for up to 6 people at the same time. The skeleton model, captured by the Kinect's cameras, consists of 25 connection points (joints) for each person. The geometric properties, of each connection points determine the position of the person in space. In addition to spatial data, the sensor also can determine the speed of the moving objects and the distance between them. During the measurement the sensor was fixed. The reason for this restriction is that the device would lose focus otherwise, which would change the reference value. The Kinect sensor has its own technical bottlenecks, for example it needs reference points to identify people and to calculate the distance. To preserve the reference points, it requires a static background and constant light conditions [8].

The measurement layout
The scheme of the measuring system can be seen in Fig. 1. The main component of the system is a stationary positioned Kinect sensor, which is connected to the central processing unit. As it is shown in Fig. 1, the sensor is positioned opposite the patient and the horizontal distance of the subject is represented on the Z-axis. If the sensor is HUMAN GAIT CYCLE ANALYSIS USING KINECT V2 SENSOR 5 Pollack Periodica 15, 2020, 3 not able to detect all of the body points, it estimates the joint's spatial position. The placement of the sensor is justified by the fact, that in this way the sensor is able to measure the points, instead of estimating them. For example, estimation may occur when the sensor is directed to the side of the body, looking in the X direction, where one limb may block out the points of the body or the other limb. The ideal distance for the measurement is between 0.5 meter and 3.5 meters. Two different scenarios are laid out. In the first arrangement, the subject was in the RoboGait [9], which is a locomotor therapy system. In this case, the person is secured to the device with an elastic harness, at multiple points. It follows that the displacement of the center of the body-mass is not significant, less than a centimeter. In the second arrangement (non-secured arrangement) the subject moves freely on a treadmill. To analyze the individual phases of the gait cycle the ideal movement pattern has to be established. For this reason, the patient is solely moved by the RoboGait, while being recorded.
In case if it is not possible to walk on a treadmill, the measuring range can be adjusted for recording 4-5 complete cycles.

Real-time monitoring system
The self-developed computer-based monitoring system is capable of displaying data in real time and storing the measured data for processing later. For this purpose, a LabView based measuring and data recording application has been developed, which is directly connected to the sensor (see Fig. 2). The application establishes the communication between the sensor and computer. After the verification process is done, the measurement should be started. In any other case, the application sends short messages about the warning state. The application provides a graphical user interface with real-time data representation and gives feedback to experts or to the patient. For this reason, the joints to be examined can be selected before the beginning of the 6 P. MÜLLER, Á. SCHIFFER Pollack Periodica 15, 2020, 3 measurement, of which the distance or calculated angle or angular velocity can be continuously monitored. This way the system is capable of verifying rehabilitation processes and repetitive movement tasks prescribed by a movement therapist; an example is the inspection of the raising arm, where the magnitude and change rate of the angle measured at the shoulders can be tracked in real time.
During the measurements the application store all recorded information in a Comma Separated Values (CSV) file, what consists of the timestamps, and the joints coordinate in spatial space.   Pollack Periodica 15, 2020, 3

Post-processing of measurement data
The phases of the gait can be identified during post-processing. The self-developed data acquisition and processing application, does not only derive the data relevant to the selected joints, but can also calculate and store all data of the whole measurement cycle into the same single file. The data format of this file allows for the recording of the X, Y, Z coordinates of the 25 tracked joints simultaneously with timestamps. The processing continues with a script, which serves multiple purposes. The first step of the evaluation process is to filter the samples with a low-pass filter and to determine the point in time from which the gait cycle can be considered periodic. The gait cycle starts with a transient phase and the periodic movement starts only after a few steps. The data points after this can be used to identify the gait cycle phases. The flattening and averaging filter serves the purpose of eliminating the inaccuracies of the measurement system and helps to reduce the noise especially in case of a handicapped person [9].
To analyze the individual phases of the gait cycle the ideal movement pattern has to be established. For this reason, the patient is solely moved by the RoboGait, while being recorded.

Identifying gait cycles
To characterize the gait pattern the kinematic properties describing the temporal nature of the gait can be used, such as the step time, cycle time, support phase duration, swing phase duration and dual support phase duration. Another group of kinematic features are the angular parameters used to characterize the joint's movements. In the literature a lot of models for gait cycle identification can be found: for example, it can be divided into eight phases [10], but in that case the patients use orthoses and rehabilitation aids, and the smaller signs are difficult to identify. In this paper the normal gait cycle is divided into 4 main phases [11], where the step states are named after the spatial position of the foot at the beginning of the phase (see Fig. 4 The ratios of the times spent in the different phases of the gait cycle [11] are shown in Table I. The percentage distribution shows the healthy gait cycle and phases, but patients with moving disabilities have different percentage distribution. This means, the certain phases may be reduced radically or even in the worst case completely [12].    Fig. 6 shows the measured and calculated time-independent and time-dependent values of a RoboGait measurement session for the three joints (ankle, knee, hip), which are indispensable to the analysis of the gait cycle. The measured quantitates are the following: ○ (circle) belongs to knee distance from the sensor, ☐ (square) is the ankle distance and ✳ (star) is the foot distance. Fig. 6a represents the absolute distance between the sensor and the points of the knee (V. point), the ankle (IV. point), and the peg[feet] (III. point), and Fig. 6b shows the angles α, β and γ respectively, where ○ (circle) belongs to knee angle, ☐ (square) is the ankle angle and the angle at hip is the ✳ (star). In Fig. 7 a steady gait cycle can be seen, measured in RoboGait. The vertical lines in the figures follow the phases of the gait cycle derived from the movement analysis (HS, HO, TO, MS).

Fig. 7. Stationary step cycle measured in RoboGait
The distribution of the identified phases, over a single step-cycle is shown in Fig. 8. The distribution demonstrated in the figure provides a good match to the well-known 60%-40% ratio, where 60% belongs to the stance phase, what is including the Heel Strike, Heel Off and the Toe Off, the rest to the swing phase [11].

Evaluate the measurement
The period of a steady step of a non-fastened, normal walk can be seen in Fig. 9. Comparing the results in Fig. 7 and Fig. 9 it can be stated that the number of samples corresponding to a single cycle is different. The reason for this is that RoboGait was primarily developed for rehabilitation and developing movement, where the movement speed is about 3 times slower than in the case of natural (non-fastened) walking, therefore it may occur that the two examined cases provide differing number of samples.   Table II and Table III show the similarities between the step phases of RoboGait and normal walking with respect to the left and right knee joints with their deviation from each other and the calculated correlation coefficient between the two legs. The difference between the left and right gait cycle phases are explained by the fact that the person was an athlete with either dominant leg or injury. This is the reason why only the correlation of the left lower extremities is shown.

Summary
The measurement method described above identified the gait phases with sufficient accuracy, compared to what is described in the literature. The accuracy of the method is further enhanced by the fact, that the correlation between the left and the right walking phases in the investigated scenarios is 99.4%.
The application developed for the measurement process gives rooms for a real time support and expert system. The support system was designed to aid the work of a physiotherapist by providing continuous and real-time measurable parameters and calculated data.
The remaining part of the research is the development of an expert system that is capable of autonomous decision-making regarding the identification of movement phases. For example, by detecting differences between the phases of movement, it is able to suggest a prognosis or to provide opportunities for further higher levels of rehabilitation activities, which can be personalized for patients individually.