The scientists, who have published the discovery in ‘Nature Neuroscience’, managed to improve this device, known as a brain-computer interface (BCI), with artificial intelligence (AI) algorithms that decoded the phrases as the woman thought them and then spoke them out loud using a synthetic voice.
Unlike previous initiatives that only produced sounds when the user finished speaking a phrase, the current method can detect words simultaneously and convert them into voice in less than three seconds.
The participant in the study, Ann, lost the ability to speak after suffering a stroke in the brainstem in 2005. Eighteen years later, she underwent a surgical procedure to implant a thin rectangle with 253 electrodes on the surface of the cerebral cortex. The implant can record the combined activity of thousands of neurons at the same time.
The researchers customized the synthetic voice to sound like Ann’s voice before her injury, training artificial intelligence algorithms with recordings of her wedding video.
«We adopted continuous flow transducer techniques, similar to those used by popular ASR methods like Siri or Alexa, and repurposed them for personalized brain-voice synthesis,» said Kaylo Littlejohn, co-lead author of the study.
«This approach led to significant improvements in the decoding speed of the brain-voice neuroprosthesis compared to previous approaches with longer delays,» Littlejohn highlighted.
In the study, the scientists explain that natural oral communication is instantaneous, and delays in speech of more than a few seconds can disrupt the natural flow of conversation.
«This makes it difficult for people with paralysis to engage in meaningful dialogue, which can lead to feelings of isolation and frustration,» they point out.
Therefore, they designed and used recurrent deep learning neural network transducer models to achieve a personalized, intelligible, wide-vocabulary speech synthesis online according to the participant’s voice.
«Our findings introduce a speech-neuroprosthesis paradigm to restore naturalistic spoken communication in people with paralysis,» the scientists emphasize.
### TRAINING OF ANN
The researchers have designed a speech synthesis neuroprosthesis that allows Ann to synthesize the desired speech from neural signals acquired from a 253-channel ECoG matrix implanted on the surface of her sensorimotor cortex and a small portion of the temporal lobe.
To train the system, they recorded neural data while Ann attempted to pronounce individual phrases. She was presented with text on a monitor and asked to silently start speaking once a visual ‘GO’ appeared.
Additionally, the synthesized speech was transmitted through a nearby analog speaker, and the decoded text was displayed on the monitor. The neural decoders of the system were bimodal, as they were trained not only to synthesize speech but also to decode text simultaneously.
They also evaluated the system using a set of 50 reduced-vocabulary phrases and a set of extensive vocabulary phrases with 1,024 general words. The set of 50 phrases was designed as a set of predefined phrases to express the primary needs of caregivers.
In contrast, the set of 1,024 general words was designed as a set of high-vocabulary phrases containing 12,379 unique phrases composed of 1,024 unique words extracted from the X social network and movie transcripts.
Para ofrecer las mejores experiencias, utilizamos tecnologías como las cookies para almacenar y/o acceder a la información del dispositivo. El consentimiento de estas tecnologías nos permitirá procesar datos como el comportamiento de navegación o las identificaciones únicas en este sitio. No consentir o retirar el consentimiento, puede afectar negativamente a ciertas características y funciones.
Funcional
Siempre activo
El almacenamiento o acceso técnico es estrictamente necesario para el propósito legítimo de permitir el uso de un servicio específico explícitamente solicitado por el abonado o usuario, o con el único propósito de llevar a cabo la transmisión de una comunicación a través de una red de comunicaciones electrónicas.
Preferencias
El almacenamiento o acceso técnico es necesario para la finalidad legítima de almacenar preferencias no solicitadas por el abonado o usuario.
Estadísticas
El almacenamiento o acceso técnico que es utilizado exclusivamente con fines estadísticos.El almacenamiento o acceso técnico que se utiliza exclusivamente con fines estadísticos anónimos. Sin un requerimiento, el cumplimiento voluntario por parte de tu proveedor de servicios de Internet, o los registros adicionales de un tercero, la información almacenada o recuperada sólo para este propósito no se puede utilizar para identificarte.
Marketing
El almacenamiento o acceso técnico es necesario para crear perfiles de usuario para enviar publicidad, o para rastrear al usuario en una web o en varias web con fines de marketing similares.
