Scientists at MIT have developed a neural network that analyzes video recordings of patients with motor or neurological disorders and assesses their clinical condition in real time. This tool operates on most mobile devices and significantly reduces the time spent by both patients and physicians on regular check-ups.
A visit to the doctor can be a laborious and time-consuming process for parents of children with motor impairments such as cerebral palsy, as the physician must regularly conduct in-person examinations lasting about an hour. By combining computer vision and machine learning methods, the algorithm analyzes video recordings of patients and provides a clinical assessment of their motor function.
Method Details
Researchers tested the method on video recordings of 1000 children with cerebral palsy. The method’s evaluation yielded an accuracy of over 70%, which closely matches what a doctor determined during a physical visit. Scientists utilized a publicly available skeleton pose dataset prepared by the neuromuscular biomechanics laboratory at Stanford University. This dataset included video recordings of over 1000 children with cerebral palsy, each displaying a child performing a series of exercises in clinical conditions, and each video was annotated with a GMFCS rating assigned by a physician after an in-person evaluation.
Subsequently, researchers developed methods for automatic decoding of patterns in cerebral palsy data specific to each clinical level of motor function. They began with a spatial-temporal graph convolutional neural network. Before applying the neural network to cerebral palsy, scientists employed a model pre-trained on a broader dataset containing video recordings of healthy adults performing various everyday actions like walking, running, sitting, and shaking hands. They adapted this pre-trained model and added a new classification layer specific to clinical indicators related to cerebral palsy.
The researchers fine-tuned the network to recognize distinctive patterns in the movements of children with cerebral palsy and accurately classify them within the primary levels of clinical assessment. They discovered that the pre-trained network learned to classify the mobility levels of children more accurately than if it had been trained solely on data related to pediatric cerebral palsy.
Currently, the researchers are developing an application that parents and patients can use to automatically analyze patient video recordings taken in the comfort of their own homes. Subsequently, the results can be sent to a physician for further evaluation. The team also plans to adapt this method for assessing other neurological disorders.
