Sci-Fi lovers rejoice, soon we may be able to communicate with only our brains. In what is considered to be a significant breakthrough, scientists have developed an advanced artificial intelligence (AI) system capable of translating brain signals into comprehensible speech. This development demonstrates how close technology is to brain-controlled communication devices.
Neuroengineers faced substantial challenges when researching potential brain-computer interfaces, most notably how to translate the wide array of signals produced by the brain into images and words that result in communication. Acceleration in machine learning technology has opened the door to computing massive amounts of abstract data. Last year, a team of Canadian researchers created an algorithm with the potential to utilize electroencephalography (EEG) data to digitally recreate faces a test subject was exposed to.
Deciphering brain waves and turning them into words is another major challenge the researchers face in their quest, but the help from developed machine learning algorithms have definitely moved the project forward. The latest development comes from a team of American neuroengineers that have developed a computer algorithm that they say is capable of “decoding signals recorded from the human auditory cortex and translate them into intelligible speech.”
Scientists gathered their data by studying five patients undergoing neurosurgery for epilepsy. Patients had numerous electrodes implanted into their brain, which allowed the researchers to record comprehensive electrocorticography measurements as the patients listened to short stories. Only 30 minutes of recordings could be done at a time, due to the invasiveness of the brain surgery.
Nima Mesgarani, senior author of the study, says, “Working with Dr. Mehta [the neurosurgeon performing the procedure], we asked epilepsy patients already undergoing brain surgery to listen to sentences spoken by different people, while we measured patterns of brain activity. These neural patterns trained the vocoder.”
To effectively test the algorithm, the system was tasked with decoding voices counting from zero to nine, which were not included in the original training data. Once the speaker went through each digit, the brain signals of the patient were recorded and then promptly run through a vocoder. A neural network was then set to analyze and clean up the output produced by the vocoder.
“We found that people could understand and repeat the sounds about 75 percent of the time, which is well above and beyond any previous attempts,” says Mesgarani. “The sensitive vocoder and powerful neural networks represented the sounds the patients had originally listened to with surprising accuracy.”
Mesgrani is the first to admit that it may well be more than a decade before the technology becomes readily available. Moving forward, the team of scientists plans to refine the algorithms for better understanding of complex words and further decoding from the auditory neural data.
“This would be a game changer. It would give anyone who has lost their ability to speak, whether through injury or disease, the renewed chance to connect to the world around them.” Mesgarani says.
