Mind-Writing: Brain-to-text communication via handwriting

Jaimie Henderson and Krishna Shenoy
Jaimie Henderson and Krishna Shenoy, these two are actively collaborating in BCI since 2005 and senior co-authors of the new research

Introduction

Advancements in Artificial Intelligence made everything possible nowadays. The art of handwriting has gone through several advancements from ancient times to the modern age. This art is considered a great contributor in pushing the brain-computer interface field to the next level. Researchers have achieved a milestone in devising a system through which a person can efficiently communicate to a computer directly with the help of his or her brain. This can be done just by imagining that the person is creating a handwritten message in their own mind.

This technique efficiently enables the communication rate two times faster as compared to the previously conducted experiment of typing-by-brain. Through this, we are just required to think and the computer will understand your activity of the brain and type it as you require.

Brain-Computer Interface:

Several researchers performed various experiments and research on this idea at Stanford University. The research was conducted on a 65-year old man, who was suffering from a spinal cord injury. They implanted an electrode array in his brain. An article was recently published in the journal “Nature” that describes the full experiment conducted by the scientists at the Stanford University.

According to Cynthia Chestek who is a biomedical engineer at the University of Michigan, the most important fact that comes out from this research is its very high speed. Cynthia Chestek was not herself involved in this research but she explained the reason why this research fact got so much popularized in such less time. She told that the following outcome of the experiment is at least halfway towards body-abled typing speed.

Researchers have been studying for finding several ways through which people can communicate directly to a computer with the help of their thoughts. There will be no involvement of eye movement, hand movement, or verbal commands in this process. This technology is very efficient in providing life-giving communication techniques for people who are unable to speak because of any disease or brainstem stroke.

The previous approaches of the BCI typing-by-brain technique included a person that imagines the movement of the cursor through a digital keyboard for selecting letters. But in the latest technology provided by the brain-computer interface includes an electrode array implanted in the brain for recording the activities of the brain and algorithms of machine learning is used for deciphering the obtained thoughts. In this way, the thought of a person is translated into a typed word. The earlier typing-by-brain process enables a person with the fastest typing speed of about 8 words per minute that is equivalent to 40 characters in a minute. This achievement is quite impressive but is comparatively slower than the speed of communication in real life. The researchers at Stanford University achieved a milestone in this field by doubling the previous speed of typing. The designed model is able to decode the activity of the brain associated with handwriting at a speed of about 18 words in a minute.

This latest technology enables a person suffering from paralysis for several years to imagine the hand movements that he makes while writing sentences. This was concluded by Frank Willet who was a collaborative researcher in the following experiment. He explained that they asked the participant to think that he is writing by moving his hand, this somatosensory illusion makes them feel like their hand is actually moving again. The implanted microelectrode array in the brain’s motor cortex is able to record every electrical activity of the neurons when the individual is trying to write something.

An algorithm of machine learning is then required for decoding the pattern mapped by the brain associated with each and every letter. A computer display is used for displaying the obtained letters on the screen.  Through this, the participant successfully achieved the fastest typing speed of about 18 words per minute that is equivalent to 90 characters in a minute. When it is compared to other typing speeds in various situations, it is considered quite an impressive achievement in this field. A healthy person with equivalent age as of the study participant was able to type 23 words per minute. An adult can type on an average of 40 words per minute on a full keyboard with their hands.

The recurrent neural network learning can be trained for a few hours for enabling an efficient recognition of the neural activities of the participants while they imagine writing sentences in English. The hidden Markov model was effectively used for labeling the relevant data in the recognition process. The algorithm and technique presented in this research can be used in various other fields of research like connecting the prosthetic hands to the person’s brain. The algorithm of the brain-computer interface is required to be customized and trained for each and every individual participant. They are also required to be recalibrated along with time as the activity response of neurons also tends to change with time. This can also be required for even a slight movement of the microelectrode array in the brain. However, researchers are trying to reduce the initial time of training and looking forward to automatic recalibrated algorithms that can be very beneficial in the future. Also, this technique has low error rates as compared to other BCIs, said Shenoy, who is also a Howard Hughes Medical Institute investigator.

Cr:
Wu Tsai Neurosciences Institute, Stanford Univ.

Conclusion

Hence, the conclusion is this efficient modern technology will be very helpful for the person who is unable to speak because of any disease or brainstem stroke. A microelectrode array is required to be implanted in the brain of the person that maps the activity of the neurons and feed the outcome as various patterns to the given machine learning algorithm. This machine-learning algorithm decodes the pattern and projects the outcome on a screen that is in the form of a typed text character. Through this advanced technology we are just required to think and the computer will understand our brain activity and type it as we require. With the help of this technology the participant is capable of achieving the fastest typing speed of about 18 words per minute that is equivalent to 90 characters in a minute.

References:

Willett, F.R., Avansino, D.T., Hochberg, L.R. et al. High-performance brain-to-text communication via handwriting. Nature 593, 249–254 (2021). https://doi.org/10.1038/s41586-021-03506-2

https://neuroscience.stanford.edu/news/software-turns-mental-handwriting-screen-words-sentences

TechThoroughFare

Share on facebook
Share on twitter
Share on linkedin

Leave a Reply

Your email address will not be published.

Science

Latest

Trending