Angrick, M. | bioRxiv Science

bioRxiv2026

Toward A Whole-Brain Interface Enabled by High-Resolution Surface Electrophysiology

The goal of building a whole-brain interface - capable of generating a real-time digital representation of the state of the entire central nervous system - has long been framed as a hardware challenge of recording simultaneously from ever-larger numbers of microelectrodes. Advances in high-resolution neural interfaces now enable electrophysiologic interaction with the human nervous system at spatial and temporal resolutions previously unattainable without disrupting tissue architecture or physiology. These capabilities create new opportunities for mechanistic neuroscience, neuroprosthetics, and precision-guided neurosurgery. With the advent of modern microelectrode array technology, however, the whole-brain interface has also become a data challenge. Rather than pursuing large-scale human brain recording purely through a single device placed in a single patient, we demonstrate here that systematic multi-subject aggregation, across brain regions and behavioral paradigms, in a common anatomical framework, can achieve something functionally similar to a whole-brain interface. Here we present an approach to a functional atlas of the human central nervous system derived from high-resolution surface electrophysiology recordings obtained in the first 70 patients implanted with an anatomically conformal, thin-film neural interface positioned in more than 100 locations. The atlas comprises >100,000 electrode deployments and catalogs placement across a broad range of neuroanatomic targets spanning the motor, sensory, language, auditory, and associative domains, as well as brainstem and spinal cord structures that have historically been difficult to access at high electrophysiologic resolution. Across participants, aggregated recordings reveal somatotopic organization, including digit representations that follow a lateral-to-medial ordering consistent with classical homuncular maps, with transitions between adjacent representations occurring over spatial scales of approximately 0.8 - 2.0 mm; canonical frequency-band signatures, such as movement-related beta-band suppression and focal high-gamma (70 - 200 Hz) increases during motor and speech behaviors; and task-modulated population dynamics in both evoked and spontaneous activity across cortical, brainstem, and spinal cord regions. These findings yield insight into patterns of mesoscale electrophysiological dynamics conserved across the entire human central nervous system, suggesting that fine-scale functional organization is preserved across individuals when aligned to common anatomical landmarks. We further demonstrate several applications of these high-resolution recordings, including real-time neural decoding from upper extremity motor cortex to enable real-time neuroprosthetic control, and from ventrolateral cortex to decode cortical correlates of spoken language, facial movements, and complex facial expressions of emotion, supporting initial feasibility of combined verbal and nonverbal communication interfaces. Together, these results establish a paradigm for approaching large-scale interfaces with the human brain, as well as a starting point for a scalable, extensible atlas of human electrophysiologic function designed to accelerate discovery across basic neuroscience, translational research, and clinical neurotechnology.

neuroscience↗

bioRxiv2020

Real-time Synthesis of Imagined Speech Processes from Minimally Invasive Recordings of Neural Activity

Speech neuroprosthetics aim to provide a natural communication channel to individuals who are unable to speak due to physical or neurological impairments. Real-time synthesis of acoustic speech directly from measured neural activity could enable natural conversations and significantly improve quality of life, particularly for individuals who have severely limited means of communication. Recent advances in decoding approaches have led to high quality reconstructions of acoustic speech from invasively measured neural activity. However, most prior research utilizes data collected during open-loop experiments of articulated speech, which neglects the critical human-in-the-loop aspect of a practical speech neuroprosthetic. Here we present an approach that synthesizes audible speech in real-time for both imagined and whispered speech conditions. Using a participant implanted with stereotactic depth electrodes, we were able to reliably generate audible speech in real-time. The decoding models rely predominately on frontal activity suggesting that speech processes have similar representations when vocalized, whispered, or imagined. Our real-time synthesis approach represents an essential step towards investigating how patients will learn to operate a closed-loop speech neuroprosthesis, as well as the development of techniques that incorporate co-adaptation of the user and system for optimized performance.

neuroscience↗

bioRxiv2018

Speech Synthesis from ECoG using Densely Connected 3D Convolutional Neural Networks

ObjectiveDirect synthesis of speech from neural signals could provide a fast and natural way of communication to people with neurological diseases. Invasively-measured brain activity (electrocorticography; ECoG) supplies the necessary temporal and spatial resolution to decode fast and complex processes such as speech production. A number of impressive advances in speech decoding using neural signals have been achieved in recent years, but the complex dynamics are still not fully understood. However, it is unlikely that simple linear models can capture the relation between neural activity and continuous spoken speech.\n\nApproachHere we show that deep neural networks can be used to map ECoG from speech production areas onto an intermediate representation of speech (logMel spectrogram). The proposed method uses a densely connected convolutional neural network topology which is well-suited to work with the small amount of data available from each participant.\n\nMain resultsIn a study with six participants, we achieved correlations up to r = 0.69 between the reconstructed and original logMel spectrograms. We transfered our prediction back into an audible waveform by applying a Wavenet vocoder. The vocoder was conditioned on logMel features that harnessed a much larger, pre-existing data corpus to provide the most natural acoustic output.\n\nSignificanceTo the best of our knowledge, this is the first time that high-quality speech has been reconstructed from neural recordings during speech production using deep neural networks.

neuroscience↗