ΑΙhub.org
 

Decoding brain activity into speech

by
01 May 2019



share this:


A recent paper in Nature reports on a new technology created by UC San Francisco neuroscientists that translates neural activity into speech. Although the technology was trialled on participants with intact speech, the hope is that it could be transformative in the future for people who are unable to communicate as a result of neurological impairments.

The researchers asked five volunteers being treated at the UCSF Epilepsy Center, with electrodes temporarily implanted in their brains, to read several hundred sentences aloud while their brain activity was recorded.

Based on the audio recordings of participants’ voices, the researchers used linguistic principles to reverse engineer the vocal tract movements needed to produce those sounds: pressing the lips together, tightening vocal cords, shifting the tip of the tongue to the roof of the mouth, then relaxing it, and so on.

This detailed mapping of sound to anatomy allowed the scientists to create a realistic virtual vocal tract for each participant that could be controlled by their brain activity. This included two neural networks: a decoder that transforms brain activity patterns produced during speech into movements of the virtual vocal tract, and a synthesizer that converts these vocal tract movements into a synthetic approximation of the participant’s voice.

A video of the resulting brain-to-speech synthesis can be found below.

You can read the UC San Francisco press release on which this news highlight is based here.

Reference
Anumanchipalli, G. K., Chartier, J., & Chang, E. F. (2019). Speech synthesis from neural decoding of spoken sentences. Nature, 568(7753), 493.




AIhub Editor is dedicated to free high-quality information about AI.
AIhub Editor is dedicated to free high-quality information about AI.




            AIhub is supported by:


Related posts :



Keeping learning-based control safe by regulating distributional shift

We propose a new framework to reason about the safety of a learning-based controller with respect to its training distribution.
30 September 2022, by

Bipedal robot achieves Guinness World Record in 100 metres

Cassie the robot, developed at Oregon State University, records the fastest 100 metres by a bipedal robot.
29 September 2022, by

#IJCAI2022 distinguished paper – Plurality veto: A simple voting rule achieving optimal metric distortion

How can we create a voting system that best represents the preferences of the voters?
28 September 2022, by

AIhub monthly digest: September 2022 – environmental conservation, retrosynthesis, and RoboCup

Welcome to our monthly digest, where you can catch up with AI research, events and news from the month past.
27 September 2022, by

The Machine Ethics Podcast: Rights, trust and ethical choice with Ricardo Baeza-Yates

Host Ben Byford chats to Ricardo Baeza-Yates about responsible AI, the importance of AI governance, questioning people's intent to create AGI, and more.
26 September 2022, by

Recurrent model-free RL can be a strong baseline for many POMDPs

Considering an approach for dealing with realistic problems with noise and incomplete information.
23 September 2022, by





©2021 - Association for the Understanding of Artificial Intelligence


 












©2021 - ROBOTS Association