Brain-Computer Interface Speaks Up

Chris Palmer

Senior Technology Writer

As reported in July 2021 in The New England Journal of Medicine,a neural implant helped a paralyzed man speak his first comprehensible words in 18 years: ‘‘My family is outside” [1,2]. The achievement—albeit in a single patient, nicknamed Pancho, who as a 20-year-old in 2003 suffered a severe stroke following a horrific car crash—highlights recent progress in engineering braincomputer interfaces (BCIs). The algorithms that control the array of electrodes placed onto the speech areas of Pancho’s brain and translate his neural activity into words were developed by researchers at the University of California, San Francisco (UCSF).The team hopes their work could one day restore the ability to communicate to the thousands of people each year that lose the capacity to speak due to stroke, disease, or injury.

Advances in electronics, materials science, and artificial intelligence(AI)are making these devices smarter,smaller,less invasive,and more compatible with the human body.Analysts now predict a global BCI market worth as much as 5.5 billion USD by 2030 [3],with companies such as Neurolink (San Francisco, CA, USA) and Facebook (Menlo Park, CA, USA) working to develop BCI devices suitable for broad use in patients and by everyday consumers.‘‘There has been so much growth in the field over the past decade,”said David Moses, the postdoctoral researcher who led the UCSF speech neuroprosthesis research. ‘‘One of the clearest signs of this is that industry involvement has skyrocketed.”

At its most basic level,BCI technology allows a human brain and an external device to talk to each other.Surgeons implant invasive BCIs either directly into the brain or, in cases like the one that allowed Pancho to communicate, on the surface of the brain. The devices, often an array of dozens of tiny metal electrodes, target specific sets of neurons, both recording their activity and stimulating them into action.

While implanted devices allow direct access to the brain’s electrical signals, there are notable drawbacks. ‘‘With any surgery there are health risks. And, from the neurons’ point of view, intracortical implants are like being stabbed by a telephone pole,” said Virginia de Sa, a professor of cognitive science at the University of California,San Diego,who uses BCI in her research.‘‘Also,the brain does not like foreign substances in it, so glia tend to grow over intracortical electrodes,which means eventually you can no longer stimulate or record.”

Non-invasive BCIs often use sensors applied to the skull to detect electrical brain activity from up to millions of neurons simultaneously. These can be placed and removed easily without surgery, but their signals may be muffled and imprecise relative to more invasive tools.‘‘It is like trying to make out what two people are saying when standing next to them versus listening from outside the room,” said Kip Ludwig, an associate professor of biomedical engineering at the University of Wisconsin, Madison,who develops neural stimulation devices.

Early BCI development focused on restoring movement in paralyzed individuals, but the technology has found numerous other applications from restoring vision and hearing to treating mental and neurological disorders, as well as allowing paralyzed people to communicate. In work first reported in 2006 [4], a team of scientists from multiple US academic institutions leaned on decades of research in animals to outfit a tetraplegic named Matt Nagle with the first implanted BCI that controlled an artificial hand.Since then, a steady stream of improvements has led to prostheses that allow full three-dimensional control and grasping [5] and provide tactile feedback (Fig. 1) [6].

Fig.1. A soft,elastic,lightweight robotic hand engineering to provide simultaneous myoelectric control and tactile feedback gives amputees greater control than traditional neuroprosthetic hands [6]. Credit: Massachusetts Institute of Technology/Xuanhe Zhao (public domain).

Fig. 2. DBS electrodes are now routinely used to treat Parkinson’s disease. In this simulated image, DBS electrodes (white) have been surgically implanted in the subthalamic nucleus(orange)in both hemispheres of the brain.These brain regions play a role in facilitating desired movement while inhibiting unwanted movements.Credit: Wikimedia Commons (CC BY-SA 4.0).

In addition to improving prosthetic function, BCIs have been used to treat a variety of mental and neurological disorders. The US Food and Drug Administration approved the first deep brain stimulation (DBS) device—essentially a long electrode with multiple electrical contact points controlled by a device implanted under the collarbone—for the treatment of essential tremor in 1997, followed by approvals for the treatment of Parkinson’s disease in 2002 (Fig. 2), dystonia in 2003, and epilepsy in 2018 (see Ref. [7] for a discussion of DBS). ‘‘DBS has been a homerun for Parkinson’s disease and essential tremor,” Ludwig said. ‘‘You can average 60% improvement in motor scores. In many cases, it is the difference between these patients not being able to work at all and being able to get back to their jobs.”

‘‘The technology has,however,struggled to reach all those who might benefit,” Ludwig said. ‘‘Only 5% to 20% of patients that are indicated for DBS for Parkinson’s disease will get it, because if you say, ‘I am going to drill a burr hole and shove a 1.2 mm lead into a deep brain area,’ patients say ‘I will try just about anything else first.’”

The use of DBS for depression has also not met initial expectations. Following early reports of the devices successfully treating 40%-80%of patients in small trials,the world’s largest clinical trial to date was abruptly halted in 2017 by its now defunct medicaldevice-manufacturer sponsor (St. Jude Medical, Little Canada,MN, USA) for reasons that remain unclear [8].

Despite this setback, researchers are still pursuing the technology. The leaders of the aborted trial recently used AI to identify beta rhythm brain activity as a biomarker for DBS against treatment-resistant depression, a finding that they hope can help optimize the approach’s effectiveness [9]. And in November 2021, a UCSF team headed by neurosurgeon Edward Chang, who also led the research on Pancho’s speech neuroprosthesis, announced that they had used a DBS device—NeuroPace Responsive Neurostimulation (RNS) (NeuroPace, Mountain View, CA, USA)—to successfully treat a 36-year-old woman who had suffered from treatment-resistant major depressive disorder since childhood[10].Unlike previous DBS electrodes that simply deliver a constant stream of electrical stimulation, the NeuroPace system detected specific brain activity patterns associated with the woman’s depressive symptoms and delivered electrical impulses to brain circuitry where it provided the greatest benefit. The researchers reported that the woman has experienced symptom relief for more than a year.

Helping people robbed of the ability to speak to better communicate, via typing, handwriting, or synthesized speech, has been another major focus of BCI research. In 2017, a BCI developed by the BrainGate research collaborative, which included researchers from Tufts University (Medford/Somerville, MA, USA) and several other US universities, allowed three paralyzed people to use their minds to select letters using an on-screen cursor,with one of them typing at eight words per minute [11]. In May 2021, the same BrainGate team reported new results with a device that performed handwriting recognition on electrical signals detected in the motor cortex of a quadriplegic subject, enabling her to generate English sentences at a rate of 86 characters per minute (Fig. 3) [12].

Fig. 3. Developed by the BrainGate research collaborative, this BCI comprised of 100 tiny metal electrodes (shown against a US ten cent coin) was used to record signals from portions of the brain’s motor cortices involved in the production of handwriting [12]. The signals were then used to generate corresponding text on a computer in real time. Credit: BrainGate.org (public domain).

Two months later, Moses et al. [1] published their work on Pancho’s speech BCI,building on their previously published results of electrocorticography measurements in patients undergoing surgery for epilepsy(Fig.4)[13,14].In 50 sessions over 81 weeks,the UCSF team connected a rectangular sheet of 128 electrodes implanted on Pancho’s brain to a computer via a cable attached to a port on his head and asked him to try to say words from a limited list of 50 common ones he helped suggest,including‘‘hungry,”‘‘music,” and ‘‘computer.” When Pancho attempted to speak,custom neural network models distinguished subtle patterns in his brain activity to detect speech attempts and identify which words he was trying to say.This training helped create a language model that responded when Pancho attempted to say particular words. The team found the system could decode words from Pancho’s brain activity at an average rate of 15 words per minute and with an accuracy rate of 74%.

‘‘While currently the vocabulary is extremely restricted, it is still impressive,” de Sa said. ‘‘With more time and effort, it will get better.”

But even a limited vocabulary is helpful for people who cannot easily use keyboards and other existing interfaces,Moses said.And their work,he added,showed consistent decoding performance for more than a year. ‘‘This is important because you do not want something that only works for a little while.”

Fig. 4. UCSF researchers used this electrocorticography implant consisting of 256 electrodes in an approximately 6 cm×6 cm square that rests on the surface of the brain without penetrating the tissue to decipher speech in patients with epilepsy undergoing surgical procedures to identify the biological origins of their seizures[14]. Credit: UCSF/Noah Berger (public domain).

While Facebook funded the UCSF speech neuroprosthesis research, the company has since stated its intention to step away from invasive devices; it will focus instead on wrist-worn devices to achieve a goal it announced in 2017:a neural interface system that will enable general consumers to type 100 words per minute [15].

Another company receiving a good deal of press on its efforts to build a BCI is Neuralink (Fremont, CA, USA), a startup launched in 2016 by serial entrepreneur Elon Musk.While Facebook is now targeting general consumers, Neuralink continues with its initial focus on a device for patients. The latest iteration of its wireless,coin-sized neural interface consists of 1024 threadlike, flexible electrodes that penetrate the cerebral cortex to both record and manipulate neural activity. While it remains to be tested in humans, the company’s device enabled a monkey to play simple Pong-like video games in a 2020 demonstration [16].

Experts say that what Neuralink has demonstrated to date is not technologically ahead of other BCI efforts, but that that could change given the company’s vast resources, including a staff of nearly 100 devoted to the project[17].‘‘In terms of getting wireless passive recordings to do predictive work,what Neuralink has done is a really good first step,” Ludwig said. ‘‘The next step, which is showing you can get it working for years in ten of ten animals before going to humans,is a large one.And honestly,the next step is even bigger. They are going to have to redesign the whole thing for regulatory approval and fault tolerance as well as supply chain and price-point considerations.”

Ludwig said he expects market considerations will play a key role in the ultimate success of BCIs.‘‘We are starting to realize very quickly that really expensive technology for a small market does not work out so well,” he said.