Siberian Federal University: Russian Scientists are Developing a Helper App for People with Aphasia

A team of young scientists from SibFU is developing an app that will help people with complex motor aphasia to restore their verbal skills either on their own or with their relatives’ assistance.

Aphasia is the absence or disorder of an already formed speech with impaired perception of it caused by a stroke, brain injury, brain tumours or brain surgeries. People with motor aphasia have frequent permutations of sounds and syllables in their speech, impaired understanding of oral speech, or meanings of single words or instructions, as well as impaired written speech.

According to various sources, in 2017, 428 thousand cases of stroke were recorded in Russia, the death rate was about 136 thousand. The National Aphasia Association (USA) has recorded that about 25–40% of patients who have undergone a stroke or various neurosurgical interventions suffer from some of the speech disorders. According to Yandex.Wordstat, an average of 400 search inquiries for “aphasia treatment” are recorded monthly.

According to the SibFU development team, an accessible cross-platform app with special speech restoration exercises will be created in Krasnoyarsk in a couple of years. A design tool for exercises is already in development, and so is the main software, and a set of scenarios is being compiled for the formed groups based on the methods of rehabilitation of people with motor aphasia.

“Our application will enable solving the isseus of rehabilitation independently or with the help of family members, moreover, recovery will be facilitated by doing special exercises while playing and through playing everyday life situations. The application is always available to the consumer, it will remind of the time and frequency of exercises during the day. It is compatible with all the main platforms — Windows, Android and iOS,” said Igor Gimazdinov, project lead, student of the School of Space and Information Technology.

According to the developers, the treatment of motor aphasia takes one to two years on average, while working with speech therapists involves significant financial costs. Many patients feel embarrassed about their condition, do not feel progress and refuse the help of specialists. The automated app is less subject to the human factor – it provides comprehensive statistics on the regularity and productivity of exercises, applies a reward system and can be used in rehabilitation centres, by professional speech therapists, and by users at home.

The application is implemented using Kivy, an open source Python library. The Yandex.SpeechKit technology is also used, which recognises and synthesises human speech. The development of exercises takes into account the methods used by speech therapists of the Centre for Neurology and Neurorehabilitation of the FMBA of Krasnoyarsk Territory. Experts from the Department of Romance Languages and Applied Linguistics of SibFU offer consultations for the project.

“Situations that give rise to positive emotions will be selected for the application scenarios: cooking together with the family, family dinner, hobbies, walking with friends. In addition, exercises and scenarios are personalized. The lexical units and set expressions that are most frequently used by representatives of the groups of users will be selected for each of them,” Igor Gimazdinov added.

The scientists emphasized that the user will be able to see their personal progress in their personal account, because the app records all the details of the rehabilitation process: the task completion time, the use of prompts, the number of correct and incorrect answers, etc. The difficulty level of the tasks adapts to each user, it changes depending on the patient’s progress. The app can advise to have rest having assessed the user’s well-being, or motivate them to continue exercising.

“It is known that in cases of speech disorder, words and speech formulas that were frequently used by the patient before the condition are most quickly restored. But the trouble is that they can no longer tell us which ones they used. However, there are age, generational, gender and other factors that can give us a hint that will help create personalized exercises for speech rehabilitation. The researchers of our department described the collective language portraits of healthy people from several risk groups for aphasia, based on them they created and clinically tested sets of exercises that will become the content for the application. Young employees of our department, as well as students take an active part in the project,” explained Anastasia Kolmogorova, head of the Department of Romance Languages ​​and Applied Linguistics of Siberian Federal University.

“Today, a demo version has been implemented for the application for restoring speech in severe complex motor aphasia cases. It shows the capability of technical implementation of the scenario core. Two groups of users were formed and a list of exercises for them was compiled. There is a team that is working on developing and promoting the product. We expect that with such helper a person will be able to independently and effectively restore their speech anywhere in the world and at any convenient time. Now we are collaborating with relatives of patients with aphasia on the Internet forums and getting feedback from them to improve the app,” the project participants say.

On 5 July 2022, the project of an app for patients with aphasia became the winner of the first stage of the Student Startup competition. The developers received a million roubles to implement their idea. This competition is held by the Ministry of Science and Higher Education of Russia and the Innovation Promotion Fund as part of the University Technological Entrepreneurship Platform federal project.