Many of us do not consider the accessibility of writing, it is something that forms a massive part of our lives yet we often take it for granted. For people who are paralysed and unable to write, this new development may allow them access to this important aspect of our social world.
In a recent breakthrough in brain-computer interface (BCI) technology, scientific researchers have developed the technology to translate visualised handwriting to the screen. While many technologies have focused on typing in a more conventional sense – such as using keys on a visualised keyboard or through eye gaze technology – the use of handwriting is a new avenue of research.
Sharing your thoughts
Scientists with the BrainGate research collaborative have created a neural implant that, through the use of sensors, enabled a paralyzed participant to ‘write’. Having deciphered the brain activity associated with hand writing letters, the team used an algorithm to identify letters as he visualised writing them. The system then displayed the text onscreen in real-time.
‘Using signals the sensors picked up from individual neurons when the man imagined writing, a machine learning algorithm recognized the patterns his brain produced with each letter. …This so-called “Brain-to-Text” BCI is so fast because each letter elicits a highly distinctive activity pattern, making it relatively easy for the algorithm to distinguish one from another’
These significant differences in the activity patterns for each letter is part of what makes this handwriting method so much more effective than using similar technologies to ‘type’. The 65-year-old participant wrote 90 characters per minute which is similar to the mobile phone typing speed of the rest of his age group. This is an incredible feat meaning that users of the technology could participate in online conversations at the same pace as their peers. While accepting the technology is not yet at a stage to be made widely available, the team behind the technology hopes that ‘the innovation could, with further development, let people with paralysis rapidly type without using their hands’.
A member of the department of bioengineering at the University of Washington, Amy L. Orsborn, stated that ‘this study is an important and clear advance for intracortical brain-computer interfaces [and] the most significant demonstration to date of leveraging established tools in machine learning like predictive language models to improve BCIs.’ The machine learning element of the study played a massive role as it was what enabled the real-time conversion of the ‘handwriting’ into on-screen text. The importance of these predictive language models in the study cannot be overstated.
Is handwriting the best way forward?
One potential quibble with the study raised by Mijail D. Serruya, an assistant professor of neurology at Thomas Jefferson University, is whether handwriting is indeed the best method of communication to use. His concern is that, whilst ‘focusing on restoring communication via written letters is intuitive, it may not be the most efficient means of doing so.’ Though handwriting is an excellent alternative to visual keyboards, he posits that there may be more efficient alternatives.
Instead of restoring handwriting, participants could for example be taught a new series of simpler gestures, creating another language akin to sign language, shorthand, or stenography chords. The argument for alternatives such as these is that they could ‘boost the speed of communication and, crucially, decrease the mental effort and attention needed.’
Another recent and very public instance of brain interfaces is Elon Musk’s Neuralink. Neuralink seeks to make it possible for users to interface directly with their mobile devices without the need for any physical input. Their device, the Link, is designed to be implanted into the brain and charged using an external wireless charger.
The threads of the Link are so small that the device cannot be implanted by the human hand. Its creators face the secondary challenge of creating and perfecting a robotic system that neurosurgeons can use to implant the technology. The device would interface with an iOS app, a simulation of which is available on their website – though with a heavy caveat that it is not FDA-approved.
Their ‘goal is to build a system with at least two orders of magnitude more communication channels (electrodes) than current clinically-approved devices. This system needs to be safe, it must have fully wireless communication through the skin, and it has to be ready for patients to take home and use on their own. Our device, called the Link, will be able to record from 1024 electrodes and is designed to meet these criteria.’
The safety of the technology is an obvious concern, both from a medical standpoint and a cybersecurity one. The technology will need to be subjected to rigorous resting on both fronts before it is even close to ready for any public applications.
One question surrounding BCIs is whether similar results could be achieved with external electrodes, and therefore if implants are actually needed. Neuralink provides a very succinct answer to this question by likening external electrodes to a microphone outside a football stadium and suggesting that, whilst they may give you the overall gist of what is occurring inside, they’ll never grant the full picture.
‘Recordings made at a distance provide some useful, high-level information, but to access fine-scale information, you need to be close to the source. Here, that means recording action potentials, or voltage “spikes,” from individual neurons. Currently, that can only be done by placing electrodes inside the brain.’
Testing of these technologies, especially ones like Neuralink that are intended for consumer and commercial use, will be a challenge and subject to understandably very heavy regulations. Progression on this front may therefore be slow but, given the medical and financial benefits of such technologies, we imagine the progress will continue steadily onwards for a long time to come.
Brain computer interfaces are no longer confined to the pages of science fiction novels and are seeing significant progress. Though advanced consumer-friendly technologies such as Neuralink are not yet available, the leaps being made in their medical use are vast and suggest such things may not be far away.
While this may only be the preliminary stages of this research, the creators of the brain-to-text technology are cautiously optimistic and hope the technology will be made available to the public in the not too distant future.
About the Author: Leo Hynett
Leo Hynett is a contributing Features Writer, with a particular interest in Culture, the Arts and LGBTQ+ Politics.
Recommended for you
As virtual medicine becomes the norm, will we stop differentiating between ‘telemedicine’ and standard healthcare altogether?
Covid-19 restrictions designed to protect the NHS have not protected its staff from burnout.
Inspiring future progress in rare diseases will require an increased focus on real-world data and AI
RwHealth’s Mike Hughes speaks on the value of data-driven innovation within Rare Diseases.
The virtual care platform providing autism and ADHD diagnoses with ongoing family support