By Dave DeFusco

Researchers have developed a series of algorithms using Siamese networks, a type of artificial intelligence, to better identify and track the body movements of stroke patients in order to assist in patient treatment and recovery. Their findings, outlined in the white paper, “Accurate Body Pose Matching for Individuals with Stroke Using Siamese Networks,” were presented in June at the IEEE/ACM international conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE). 

Stroke is one of the leading causes of long-term disability in the United States, affecting over 800,000 people annually. For those recovering from a stroke, regaining motor function is a critical step toward independence. Among the latest advancements in stroke rehabilitation are exoskeletons and game-based therapies, which are designed to help patients recover movement and coordination. 

“A key aspect of these therapies is accurately tracking the body movements of stroke patients,” said Ruslan Gokhman, lead author of the paper and a master’s student in the M.S. in Artificial Intelligence at the Katz School, “however, existing machine learning methods have struggled to precisely match the movements of stroke patients with ideal movement patterns, known as ground truths.”  

The new methods proposed by the researchers, who were supported by a 2023-2024 Dean's Faculty Research Initiative (FRI) grant, have shown significant improvement in accuracy compared to traditional approaches. The system collects movement data from stroke patients using a Kinect sensor, which recorded the positions of 25 different joints in the body from 10 healthy individuals and nine stroke patients during rehabilitation exercises. The researchers used Siamese networks, including Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN), to compare the recorded movements with ideal poses.  

These networks are trained to recognize and match similar movements accurately. The models are tested and validated using a public database of movement data from both healthy individuals and stroke patients. The Siamese-based models showed an improvement in identification accuracy compared to traditional methods. 

Accurate pose matching is crucial for effective rehabilitation. If the system can correctly identify and assess a patient’s movements, doctors can tailor the therapy to the patient’s specific needs, leading to better recovery outcomes. 

“Because of the rapid advances in technology, conducting comprehensive and regular research is imperative to identify gaps and develop a model that best addresses an unmet need,” said Dr. Rana Khan, a co-author of the paper and program director of the M.S. in Biotechnology Management and Entrepreneurship.  

The Siamese LSTM Model processes sequences of movements and can handle long-term dependencies in the data, making it effective for tracking the progression of rehabilitation over time, and the Siamese CNN Model excels at recognizing patterns in successive movements, offering higher accuracy in matching poses. 

The effectiveness of the proposed models was validated through rigorous testing and comparison with existing methods. The Siamese CNN model, in particular, exhibited remarkable precision in minimizing erroneous matches, making it an ideal choice for applications where the accuracy of pose matching is critical. 

“These advanced algorithms are not only beneficial for stroke rehabilitation but also have potential applications in other areas,” said Ramana Vinjamuri, associate professor of computer science and electrical engineering at the University of Maryland, Baltimore County, “such as virtual reality games and autonomous driving, where precise body pose estimation is important.” 

In stroke rehabilitation, the new system can help design better therapy sessions. For example, serious, or interactive, games used in therapy can be more effectively tailored to each patient’s needs, making the exercises more engaging and beneficial. These games often involve movements scored on smoothness and accuracy, which are essential for recovering motor function. 

Future research will focus on validating these models with new datasets and examining their direct impact on rehabilitation outcomes. The goal is to create even more effective and personalized rehabilitation plans for stroke patients, ultimately improving their quality of life. 

“The introduction of Siamese-based models for body pose matching represents a significant advancement in stroke rehabilitation,” said Dr. Sai Praveen Kadiyala, a co-author of the paper, lead researcher of the Stroke Rehabilitation Project and postdoctoral research fellow in the Katz School’s Department of Graduate Computer Science and Engineering. “By improving pose identification accuracy, these models can enhance the assessment and planning of rehabilitation strategies, ultimately contributing to more effective recovery for stroke patients.” 

Other researchers involved in the study were Talya Sawdayi, a recent graduate of the M.S. in Biotechnology Management and Entrepreneurship; and Dr. Ashwin Satyanarayana, an associate professor at CUNY.