Making the Discrimination in the Walking Parameters of Individuals with Multiple Sclerosis and Parkinson’s Disease with Machine Learning

Abstract

Objective: To determine the contribution of gait analysis to the differentiation and diagnosis of these diseases by examining the walking videos of individuals diagnosed with multiple sclerosis (MS) and Parkinson’s disease (PD) using the deep learning method.

Materials and Methods: A hybrid system based on Convolutional Neural Networks was developed for the detection of MS and PD based on gait analysis. The patients were walked on a flat surface of approximately 14 meters and video recordings were taken from the front, back and both sides during walking. Videos of a total of 28 patients, 12 PD and 16 MS patients, were used in the study.

Results: In the study, the data was analyzed using machine learning techniques and the best accuracy score was obtained as 87.5%.

Conclusion: The accuracy rate of machine learning models in the diagnosis, follow-up and treatment process of patients such as MS, PD and other neurological diseases has been examined and it has been concluded that it is inevitable that these methods will be used much more over time.

Introduction

The number of steps an average person takes in a day is between 5,000 and 15,000, or 2-5 million per year (1). This number decreases with age. Men tend to walk more steps than women. When we look at gait types, there is no uniform gait type among individuals with neurological involvement, and gait types vary depending on the affected area of the brain (2).

Gait disorder is a finding that can accompany almost all neurological diseases and can present with different clinical patterns depending on the affected neurological system. Patients can be admitted to clinics with different types of gait disorders according to pyramidal, sensory, deep sensory, cerebellar, or muscle involvement. Depending on the affected anatomical location, patients may have first motor neuron, second motor neuron, basal ganglion, or cerebellar findings, as well as different degrees of muscle weakness and muscle tone changes related to this involvement. As a result of these interactions, deterioration in the quality of life may occur because of the physical effects of the natural course of the diseases. The evaluation of physical function and gait in neurological diseases is important for the diagnosis of the disease, its course, and the effectiveness of the treatment applied. In the disease progression, losses in physical functions indicate key findings, and different tests are applied for the relevant disease to show these effects in the follow-up of its clinical course. In addition to these tests, evaluations are made by performing machine learning with various gait analysis methods (3,4). Studies in this field are being conducted on cerebral palsy subtypes, Parkinson’s disease (PD), and other neurological diseases (5). These studies aim to perform kinetic and kinematic analysis on joints supported by wearable technologies in the hospital environment (6,7). One of our aims is to analyze the videos the individual records in his/her environment. Using this system, which we can call preliminary diagnosis pointers, it will be possible to apply to the clinic with a report that provides objective data about the symptoms of the disease, which department to apply to, and a percentage chance of success.

Physical functions in patients with multiple sclerosis (MS) or PD can be evaluated using disease-specific tests, such as the 25-step walking test, walking and sit-to-stand test, Unified Parkinson’s Disease Rating Scale (UPDRS), Expanded Disability Status Scale (EDSS), and balance test. Gait analysis is a method of analyzing all movements performed by the patient while walking visually (kinematics) and in terms of energy spent (kinetics). This analysis method should be used for diseases that cause movement disorders. Since the analyses are performed with precise data, it is an effective method to measure treatment effectiveness, disease progression, and patient follow-up. Since it is a costly and not easily accessible method, its availability may not be possible for every patient (8). These evaluations can make a substantial contribution to the diagnosis and differential diagnosis of diseases. In addition, by detecting possible physical effects early in the natural course of diseases, the most effective neurorehabilitation practices can be implemented for the individual.

In this study, we aimed to contribute to the clinical discrimination of patients with PD and those with MS using machine learning-based gait analysis and to develop an artificial intelligence model that could make practical contributions in this field.

Material and Methods

Patients with gait problems who were followed up for PD or MS at Ondokuz Mayıs University Faculty of Medicine, Department of Neurology, were included in the study. Within the scope of the project, the neurorehabilitation room created within the Department of Neuroscience of the Graduate Institute was used. The patients walked on a flat surface of approximately 14 m, and video recordings were taken from the front, back, and both sides as they walked. The resulting video recordings were recorded on a computer. A dataset was created using videos obtained from the patients. The study protocol for this dataset was approved by the Ondokuz Mayıs University Clinical Research Ethics Committee (decision no: 2021-544, date: 24.11.2021). Written informed consent forms were obtained from the participants in the study group, and patient identifiers were removed to ensure anonymity.

A hybrid system based on convolutional neural networks (CNNs) was developed for the detection of MS and PD based on gait analysis. The general application steps of this model are given in Figure 1.

Figure 1. General flow diagram of the proposed model
MS: Multiple sclerosis

Figure 1 illustrates the proposed system, which comprises four stages: pre-processing, feature extraction, feature fusion, and classification. These stages can be summarized as follows:

Step 1. Pre-processing phase: In the pre-processing phase, videos were sequentially divided into a specified number of frames. In the current study, the number of frames was set to 10.

Step 2. Feature extraction phase: In the first step, the pretrained VGG16 architecture, developed on deep CNNs, was used for each frame image obtained from the patients’ videos (9). This architecture, developed by Simonyan and Zisserman (9), can be defined as an ESA and comprises 16 layers arranged sequentially. Most of these layers comprise small-sized convolution layers with 3x3-dimensional filters. Maximum pooling was applied to these filters before more convolution layers were applied. This process further abstracted features, enabling the network to learn more complex objects and features. Additionally, this architecture included two fully connected layers. These layers smoothed the inputs after feature extraction and were then used for classification. The last layer used the softmax activation function, and its outputs represented the probabilities of possible class labels. Finally, this architecture was trained and tested using the ImageNet dataset containing 1,000 classes (9,10).

In this study, this architecture was preferred because the pretrained deep architecture had less data and generally had higher performance than models developed from scratch. As shown in Figure 2, 4,096 features were extracted for each frame from the fully connected layer called fc6 of the VGG16 architecture (11).

Figure 2. Structure of the deep VGGNet architecture used in the current study

In Equation 1, N represents the number of frames.

Step 3. Feature fusion phase: The deep features obtained for each frame using Equation 1 were combined using the global mean pooling approach in Equation 2:

Step 4. Classification phase: The combined features were the input of the fully connected layer. Subsequently, a model based on disease diagnosis was developed by following this layer with the rectified linear unit layer, then the fully connected layer, and finally the softmax layer.

An experiment was conducted by examining previously used models on PD (12). In these studies, gait analysis algorithms used for diagnosis revealed a balanced accuracy range between 83.5% and 100%, sensitivity between 83.3% and 100%, and specificity between 82% and 100%. Gait analysis algorithms for motor state discrimination revealed a balanced accuracy range of 90.8-100%, sensitivity of 92.5-100%, and specificity of 88-100% (13).

In the experimental studies, the MATLAB (R2022a) software platform was used. These studies were performed using a computer with an NVIDIA Quadro P4000 GPU card, 32 GB RAM, and an Intel Xeon Silver 2.19 GHz processor. In all experimental studies, 80% of the dataset was used for the training phase and the remaining 20% for the testing phase.

To evaluate the performance of the proposed system, videos obtained from 28 patients, 12 with PD and 16 with MS, were used. Each video had an average duration of 15-25 s. The video of each patient was divided into three equal parts to produce 84 videos in total. Finally, 10 frames were randomly extracted from each video and input into the proposed system. In the training phase of the proposed architecture, the mini-batch size was set to 8, the epoch value was set to 50, and the learning rate was set to 0.0001. Additionally, the performances of different approaches based on three different optimization algorithms (adam, rmsprop, and sgdm) were calculated.

Results

The performance results obtained from these studies are given in Table 1. The optimization algorithm with the best accuracy score based on the proposed architecture was 87.5% with adam. The sgdm optimization method produced an accuracy score of 81.25%, whereas a score of 75% was obtained using the rmsprop method. The confusion matrices of the proposed architecture based on the three optimization algorithms are given in Figure 3.

Figure 3. Confusion matrices of the proposed model: a) sgdm, b) adam, c) rmsprop
MS: Multiple sclerosis

As illustrated in Figure 3b, one patient with MS and one patient with PD were misclassified using the proposed model based on the adam optimization method. In addition, three and four classes were detected incorrectly by the sgdm and rmsprop optimization methods, respectively. Thus, the model developed based on CNNs achieved promising results.

Discussion

We observed that the data obtained from the study contributed to the diagnosis of two different neurological diseases, MS and PD, and to the grading of the physical disabilities of the diagnosed patients. These two neurological diseases were distinguished from each other with a high percentage of success using the CNN model obtained using the walking images of the patients. In new studies, it will be important to label the affected neurological systems in detail and report the disabilities of the diseases more quantitatively to ensure objective data.

Machine learning provides substantial knowledge transfer and contributes to the prognostic process. Gait may be affected, especially in diseases where bradykinesia is prominent, such as PD, or in which weakness and ataxia may be observed, such as MS. Parkinsonian gait is characterized by short step intervals, which are initially very slow because of difficulty in starting the movement but gradually gain speed, and arm oscillations are minimal. Weakness and ataxia due to pyramidal findings are prominent in MS, and gait progresses with dispersed parameters and is affected by fatigue. Balance functions are affected in the foreground, and step length is not distributed homogeneously.

Experts can distinguish the gait type of these two disease groups with the naked eye in most, if not all, patients. To make this distinction using engineering, the disease must be defined and the changes in speed, walking distance, arm oscillations, and step length taught to the machine one by one. By correctly labeling the data and teaching the machine the diagnoses of the patients included for gait evaluation, the machine can determine which disease group the test data belongs to. We understood that the performance rate was high because of the visible difference in the gait types of the two disease groups and the correct teaching of the machine. It is estimated that the success rate will increase as data from the same two patient groups increases. This study was conducted to report that machine learning is effective. It is expected that the success in the discrimination between diseases will increase with the continuation of our study.

With the qualitative and quantitative increase in patient data, machines may be used to determine the differential diagnosis and prognostic process of patients with primary motor neuron findings, such as motor neuron disease, myelitis, MS, and cerebrovascular disease, as well as patient groups with bradykinesia, such as PD and normal pressure hydrocephalus. More contributions can be made with artificial intelligence models based on machine learning.

Our study also aims to provide objective data in the future about the patients’ parameters, such as EDSS, UPDRS, and degree of paresis. The model developed based on the CNNs has achieved promising results. We aim to increase the classification performance by increasing the dataset. Additionally, we will investigate time series such as long short-term memory and gated recurrent unit and plan to adapt them for the proposed model.

Conclusion

The artificial intelligence-based analysis of gait can be considered a paraclinical method that will make a vital contribution to the diagnosis, differential diagnosis, and determination of the prognostic process for patients with neurological diseases who have undergone neurological examination and related tests. By adding radiological and laboratory data to machine learning, the contribution to the accurate artificial intelligence-based diagnosis of neurological diseases can be further increased. When sufficient performance is achieved through studies with larger patient groups, an artificial intelligence model that can be used in the classification of certain neurological diseases will be created.

The study was approved by the Ondokuz Mayıs University Clinical Research Ethics Committee (decision no: 2021-544, date: 24.11.2021).

Consent form was filled out by all participants.

Externally peer-reviewed.

Concept: S.G., M.T., Design: S.G., E.S., Data Collection or Processing: S.G., M.T., K.A.K., Analysis or Interpretation: S.G., E.S., M.T., M.Tü., K.A.K., Literature Search: K.A.K., Writing: S.G., E.S., K.A.K.

No conflict of interest was declared by the authors.

The authors declared that this study received no financial support.

References

1. Silva M, Shepherd EF, Jackson WO, Dorey FJ, Schmalzried TP. Average patient walking activity approaches 2 million cycles per year: pedometers under-record walking activity. J Arthroplasty 2002;17:693-697.
2. Nelson AJ. Functional ambulation profile. Phys Ther 1974;54:1059-1065.
3. Khera P, Kumar N. Role of machine learning in gait analysis: a review. J Med Eng Technol 2020;44:441-467.
4. De Vos M, Prince J, Buchanan T, FitzGerald JJ, Antoniades CA. Discriminating progressive supranuclear palsy from Parkinson’s disease using wearable technology and machine learning. Gait Posture 2020;77:257- 263.
5. MacWilliams BA, Carroll KL, Stotts AK, Kerr LM, Schwartz MH. Discrimination between hereditary spastic paraplegia and cerebral palsy based on gait analysis data: a machine learning approach. Gait Posture 2022;98:34-38.
6. Landolfi A, Ricciardi C, Donisi L, et al. Machine learning approaches in Parkinson’s disease. Curr Med Chem 2021;28:6548-6568.
7. Slemenšek J, Fister I, Geršak J, et al. Human gait activity recognition machine learning methods. Sensors (Basel) 202323:745.
8. Sequeira MM, Rickenbach M, Wietlisbach V, Tullen B, Schutz Y. Physical activity assessment using a pedometer and its comparison with a questionnaire in a large population survey. Am J Epidemiol 1995;142:989- 999.
9. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv 2014;1409.
10. Turkoglu M. COVIDetectioNet: COVID-19 diagnosis system based on X-ray images using features selected from pre-learned deep features ensemble. Applied Intelligence 2021;51:1213-1226.
11. Schumann P, Scholz M, Trentzsch K, et al T. Detection of Fall Risk in Multiple sclerosis by gait analysis-an innovative approach using feature selection ensemble and machine learning algorithms. Brain Sci 2022;12:1477.
12. Severson KA, Chahine LM, Smolensky LA, et al. Discovery of Parkinson’s disease states and disease progression modelling: a longitudinal data study using machine learning. Lancet Digit Health 2021;3:e555-e564.
13. di Biase L, Di Santo A, Caminiti ML, et al. Gait analysis in Parkinson’s disease: an overview of the most accurate markers for diagnosis and symptoms monitoring. Sensors (Basel) 2020;20:3529.