Biology subjects

Stavisky, S. D.

Publications and source records attributed to Stavisky, S. D..

At least 19 recordsLinked to original sources

bioRxiv2026

Human-calibrated automated intelligibility evaluation of speech neuroprostheses

Objective: Brain-to-voice neuroprostheses can synthesize speech from the neural activity of people who have lost the ability to speak, enabling more natural and expressive communication. Assessing the intelligibility of synthesized speech, however, remains challenging because there is no ground-truth acoustic signal. The field instead relies on human transcription, which is costly and difficult to scale or compare across studies. Automatic speech recognition (ASR) offers a potential alternative, but state-of-the-art ASR models are optimized for transcription accuracy on typical fluent speech and may not reflect how humans perceive atypical synthesized speech. Approach: We collected human transcriptions for ~14.5 hours of synthesized speech from intracortical brain-to-voice decoders for one participant (yielding ~72 hours of transcribed speech). We used the human-transcribed synthesized speech to fine-tune Whisper-large-v3 with low-rank adaptation (LoRA) so that its word error rate (WER) approximates human WER. We then evaluated the model on held-out sentences. Main results: Fine-tuning increased the Pearson correlation between model WER and human WER from 0.62 to 0.76. The fine-tuned model's WER closely tracked human WER across evaluation sets, including speech synthesized by unseen brain-to-voice decoders. Close agreement with human WER was achieved with only 1.5 hours of synthesized speech. Significance: These results demonstrate that human-calibrated ASR can provide a scalable measure of brain-to-voice intelligibility that reflects human transcription more closely than zero-shot ASR does.

neuroscience↗

bioRxiv2026

A generalizable speech neuroprosthesis

Intracortical brain-computer interfaces (BCIs) can restore communication to people with vocal tract paralysis by decoding cortical activity during attempted speech into text. State-of-the-art systems pairing neural-to-phoneme decoders with phoneme-to-word language models have achieved word error rates (WERs) as low as 1%, but only after collecting thousands of sentences of training data. Shortening the data collection process would facilitate scaling this new technology by reducing the time from device implant to high-accuracy communication. Here we introduce a transformer-based decoder model trained jointly across six intracortical speech BCI participants. For every participant -- regardless of sex, disease etiology, or attempted speaking strategy -- a multi-user model decoded speech more accurately (over 50% lower relative WER on average) than models trained on individual users data. Notably, the multi-user model could be finetuned on fewer than 200 sentences from a held-out user to achieve a WER below 7%. These results reveal how to pool intracortical data across people to yield more accurate, generalizable, and rapidly-deployable decoding models.

bioengineering↗

bioRxiv2026

Brain2voice 2.0: High-performance voice synthesis brain-computer interface

Brain-computer interfaces (BCIs) offer a promising solution to speech loss due to neurological injury by decoding intended speech directly from brain activity. While recent BCIs have restored high-accuracy text-based communication, they fail to provide instantaneous voice output essential for the natural flow of conversation. Brain-to-voice BCIs address this gap by decoding voice directly from neural signals. However, even the state-of-the-art (SOTA) BCI-synthesized voice is not yet intelligible enough for real-world adoption. We introduce brain2voice 2.0, a new multimodal Transformer-based BCI decoder architecture capable of synthesizing highly intelligible voice from intracortical neural signals in real-time. Brain2voice 2.0 is trained on continuous and custom-tokenized acoustic targets and phoneme targets, leveraging their complementary speech information. We use self-supervised and adversarial training objectives that enhance acoustic feature quality and improve synthesis intelligibility. At each 10 ms timestep, the model causally outputs continuous and tokenized acoustic features for real-time voice synthesis as well as time-aligned phoneme predictions (raw phoneme error rate: 7%, comparable to the latest brain-to-text models). We evaluated this new approach on our prior intracortical brain-to-voice benchmark dataset (Wairagkar et al. 2025). Naive human listeners transcribed brain2voice 2.0 synthesized voice with a word error rate of 5.24%--an 8x improvement in intelligibility over previous SOTA results (43.75%). Brain2voice 2.0 demonstrates that highly intelligible real-time voice synthesis from neural signals is achievable, for the first time crossing the intelligibility threshold necessary for clinically viable brain-to-voice BCIs for people with paralysis.

neuroscience↗

bioRxiv2026

Neural decoding of speech using deep neural ensembles

Speech brain-computer interfaces (BCIs) can restore rapid communication to people with paralysis, but decoding errors still limit performance. In recent brain-to-text decoding competitions, deep ensemble methods, which combine predictions from multiple independently trained decoders, have delivered striking accuracy improvements and account for the largest gains over baseline approaches. However, these methods have not previously been tested in real-time, require substantial computational resources, and their performance under various clinically relevant constraints remains poorly understood. Here, we present the first closed-loop test of deep ensembles in a participant with bilateral intracortical microelectrode arrays, demonstrating a reduction in word error rate from 33.7% to 26.0% on a large-vocabulary task. Using additional data from three participants, we then assess how these gains depend on baseline error rate, training dataset size, and ensemble size, including the resource-accuracy tradeoffs most relevant for real-world deployment. Finally, we introduce a computationally efficient pseudoensembling approach based on test-time augmentation that improves decoding accuracy while requiring only a single base decoder, greatly reducing the computational burden of ensembling. Together, these results show that the benefits of deep ensembling can be realized in real time and under practical resource constraints, bringing speech BCIs closer to broader clinical adoption.

neuroscience↗

bioRxiv2026

Premotor cortex uses a compositional neural geometry to plan words

Speech requires precise serial ordering of words and phonemes into novel combinations. To accomplish this, the brain is believed to flexibly prepare utterances before producing them, even allowing pronunciation of never-before spoken words. To discover how neural populations achieve this, intracortical activity from premotor cortex was recorded while two speech neuroprosthesis pilot clinical trial participants attempted to speak factorially-balanced phoneme sequences. During preparation, activity encoded not only the next-phoneme, but multiple upcoming phoneme positions spanning whole words. We found that word-level plans were formed by compositionally combining phoneme representations, a mechanism that may enable efficient planning of novel sequences. When utterances contained more than one word, premotor cortex activity was largely limited to the first word, suggesting that articulatory planning is segmented by higher-order features. Together, these results reveal a compositional, hierarchically-segemented planning geometry, potentially a universal neural strategy for sequence organization across higher levels of language.

neuroscience↗

bioRxiv2026

Cross-brain transfer of high-performance intracortical speech and handwriting BCIs

Intracortical brain-computer interfaces (BCIs) that decode complex movements, such as handwriting and speech, can require substantial training data to achieve high performance. We investigated whether leveraging the neural activity recordings of previous users could reduce this initial data collection burden for new BCI users (an approach we call "cross-brain transfer"). Using intracortical recordings from five BrainGate2 clinical trial participants, we tested cross-brain transfer for both speech and handwriting neural decoders trained and evaluated on general, unconstrained corpora of spoken and written English. We found that cross-brain transfer improved decoding performance when training data from the target user was limited (< 200 sentences), and that dataset-specific input layers to the decoder were critical for combining data across users. Without trainable input layers, transfer failed and performed worse than training from scratch on target user data only. Finally, we measured the effectiveness of cross-brain transfer relative to training with (1) more data from the same user and (2) more electrode-permuted data from the same user, which simulates sampling from another brain with identical neural latent structure. In some cases (T16 speech, T12 handwriting), cross-brain transfer appeared as effective as additional permuted data from the same user, while in others (T12 speech, T15 speech) electrode-permuted data was more beneficial. Our results successfully demonstrate and characterize cross-brain transfer learning between multiple intracortical BCI users, for both speech and handwriting, using a general open-ended dataset not restricted to small sets of words or phrases. This work highlights a promising path towards addressing a key barrier to the clinical translation of BCIs, while clarifying when cross-brain transfer may be most beneficial and the decoder design choices needed to realize those gains.

neuroscience↗

bioRxiv2025

Decoding intended speech with an intracortical brain-computer interface in a person with longstanding anarthria and locked-in syndrome

Intracortical brain-computer interfaces (iBCIs) for decoding intended speech have provided individuals with ALS and severe dysarthria an intuitive method for high-throughput communication. These advances have been demonstrated in individuals who are still able to vocalize and move speech articulators. Here, we decoded intended speech from an individual with longstanding anarthria, locked-in syndrome, and ventilator dependence due to advanced symptoms of ALS. We found that phonemes, words, and higher-order language units could be decoded well above chance. While sentence decoding accuracy was below that of demonstrations in participants with dysarthria, we are able to attain an extensive characterization of the neural signals underlying speech in a person with locked-in syndrome and through our results identify several directions for future improvement. These include closed-loop speech imagery training and decoding linguistic (rather than phonemic) units from neural signals in middle precentral gyrus. Overall, these results demonstrate that speech decoding from motor cortex may be feasible in people with anarthria and ventilator dependence. For individuals with longstanding anarthria, a purely phoneme-based decoding approach may lack the accuracy necessary to support independent use as a primary means of communication; however, additional linguistic information embedded within neural signals may provide a route to augment the performance of speech decoders.

neuroscience↗

bioRxiv2025

Long-term independent use of an intracortical brain-computer interface for speech and cursor control

Brain-computer interfaces (BCIs) can provide naturalistic communication and digital access to people with severe paralysis by decoding neural activity associated with attempted speech and movement. Recent work has demonstrated highly accurate intracortical BCIs for speech and cursor control, but two critical capabilities needed for practical viability were unmet: independent at-home operation without researcher assistance, and reliable long-term performance supporting accurate speech and cursor decoding. Here, we demonstrate the independent and near-daily use of a multimodal BCI with novel brain-to-text speech and computer cursor decoders by a man with paralysis and severe dysarthria due to amyotrophic lateral sclerosis (ALS). Over nearly two years, the participant used the BCI for more than 3,800 cumulative hours to maintain rich interpersonal communication with his family and friends, independently control his personal computer, and sustain full-time employment - despite being paralyzed. He communicated 183,060 sentences - totaling 1,960,163 words - at an average rate of 56.1 words per minute. He labeled 92.3% of sentences as being decoded at least mostly correctly. In formal quantifications of performance where he was asked to say words presented on a screen, attempted speech was consistently decoded with over 99% word accuracy (125,000 word vocabulary). The participant also used the speech BCI as keyboard input and the cursor BCI as mouse input to control his personal computer, enabling him to send text messages, emails, and to browse the internet. These results demonstrate that intracortical BCIs have the potential to support independent use in the home, marking a critical step toward practical assistive technology for people with severe motor impairment.

neuroscience↗

bioRxiv2025

Error encoding in human speech motor cortex

Humans monitor their actions, including detecting errors during speech production. This self-monitoring capability also enables speech neuroprosthesis users to recognize mistakes in decoded output upon receiving visual or auditory feedback. However, it remains unknown whether neural activity related to error detection is present in the speech motor cortex. In this study, we demonstrate the existence of neural error signals in speech motor cortex firing rates during intracortical brain-to-text speech neuroprosthesis use. This activity could be decoded to enable the neuroprosthesis to identify its own errors with up to 86% accuracy. Additionally, we observed distinct neural patterns associated with specific types of mistakes, such as phonemic or semantic differences between the persons intended and displayed words. These findings reveal how feedback errors are represented within the speech motor cortex, and suggest strategies for leveraging these additional cognitive signals to improve neuroprostheses.

neuroscience↗

bioRxiv2025

Encoding of speech modes and loudness in ventral precentral gyrus

The ability to vary the mode and loudness of speech is an important part of the expressive range of human vocal communication. However, the encoding of these behaviors in the ventral precentral gyrus (vPCG) has not been studied at the resolution of neuronal firing rates. We investigated this in two participants who had intracortical microelectrode arrays implanted in their vPCG as part of a speech neuroprosthesis clinical trial. Neuronal firing rates modulated strongly in vPCG as a function of attempted mimed, whispered, normal or loud speech. At the neural ensemble level, mode/loudness and phonemic content were encoded in distinct neural subspaces. Attempted mode/loudness could be decoded from vPCG with an accuracy of 94% and 89% for two participants respectively, and corresponding neural preparatory activity could be detected hundreds of milliseconds before speech onset. We then developed a closed-loop loudness decoder that achieved 94% online accuracy in modulating a brain-to-text speech neuroprosthesis output based on attempted loudness. These findings demonstrate the feasibility of decoding mode and loudness from vPCG, paving the way for speech neuroprostheses capable of synthesizing more expressive speech.

neuroscience↗

bioRxiv2024

Representation of Verbal Thought in Motor Cortex and Implications for Speech Neuroprostheses

Speech brain-computer interfaces show great promise in restoring communication for people who can no longer speak1-3, but have also raised privacy concerns regarding their potential to decode private verbal thought4-6. Using multi-unit recordings in three participants with dysarthria, we studied the representation of inner speech in the motor cortex. We found a robust neural encoding of inner speech, such that individual words and continuously imagined sentences could be decoded in real-time This neural representation was highly correlated with overt and perceived speech. We investigated the possibility of "eavesdropping" on private verbal thought, and demonstrated that verbal memory can be decoded during a non-speech task. Nevertheless, we found a neural "overtness" dimension that can help to avoid any unintentional decoding. Together, these results demonstrate the strong representation of verbal thought in the motor cortex, and highlight important design considerations and risks that must be addressed as speech neuroprostheses become more widespread.

neuroscience↗

bioRxiv2024

A mosaic of whole-body representations in human motor cortex

Understanding how the body is represented in motor cortex is key to understanding how the brain controls movement. The precentral gyrus (PCG) has long been thought to contain largely distinct regions for the arm, leg and face (represented by the "motor homunculus"). However, mounting evidence has begun to reveal a more intermixed, interrelated and broadly tuned motor map. Here, we revisit the motor homunculus using microelectrode array recordings from 20 arrays that broadly sample PCG across 8 individuals, creating a comprehensive map of human motor cortex at single neuron resolution. We found whole-body representations throughout all sampled points of PCG, contradicting traditional leg/arm/face boundaries. We also found two speech-preferential areas with a broadly tuned, orofacial-dominant area in between them, previously unaccounted for by the homunculus. Throughout PCG, movement representations of the four limbs were interlinked, with homologous movements of different limbs (e.g., toe curl and hand close) having correlated representations. Our findings indicate that, while the classic homunculus aligns with each areas preferred body region at a coarse level, at a finer scale, PCG may be better described as a mosaic of functional zones, each with its own whole-body representation.

neuroscience↗

bioRxiv2024

An instantaneous voice synthesis neuroprosthesis

Brain computer interfaces (BCIs) have the potential to restore communication to people who have lost the ability to speak due to neurological disease or injury. BCIs have been used to translate the neural correlates of attempted speech into text1-3. However, text communication fails to capture the nuances of human speech such as prosody, intonation and immediately hearing ones own voice. Here, we demonstrate a "brain-to-voice" neuroprosthesis that instantaneously synthesizes voice with closed-loop audio feedback by decoding neural activity from 256 microelectrodes implanted into the ventral precentral gyrus of a man with amyotrophic lateral sclerosis and severe dysarthria. We overcame the challenge of lacking ground-truth speech for training the neural decoder and were able to accurately synthesize his voice. Along with phonemic content, we were also able to decode paralinguistic features from intracortical activity, enabling the participant to modulate his BCI-synthesized voice in real-time to change intonation, emphasize words, and sing short melodies. These results demonstrate the feasibility of enabling people with paralysis to speak intelligibly and expressively through a BCI.

neuroscience↗

bioRxiv2023

BRAND: A platform for closed-loop experiments with deep network models

Artificial neural networks (ANNs) are state-of-the-art tools for modeling and decoding neural activity, but deploying them in closed-loop experiments with tight timing constraints is challenging due to their limited support in existing real-time frameworks. Researchers need a platform that fully supports high-level languages for running ANNs (e.g., Python and Julia) while maintaining support for languages that are critical for low-latency data acquisition and processing (e.g., C and C++). To address these needs, we introduce the Backend for Realtime Asynchronous Neural Decoding (BRAND). BRAND comprises Linux processes, termed nodes, which communicate with each other in a graph via streams of data. Its asynchronous design allows for acquisition, control, and analysis to be executed in parallel on streams of data that may operate at different timescales. BRAND uses Redis to send data between nodes, which enables fast inter-process communication and supports 54 different programming languages. Thus, developers can easily deploy existing ANN models in BRAND with minimal implementation changes. In our tests, BRAND achieved <600 microsecond latency between processes when sending large quantities of data (1024 channels of 30 kHz neural data in 1-millisecond chunks). BRAND runs a brain-computer interface with a recurrent neural network (RNN) decoder with less than 8 milliseconds of latency from neural data input to decoder prediction. In a real-world demonstration of the system, participant T11 in the BrainGate2 clinical trial performed a standard cursor control task, in which 30 kHz signal processing, RNN decoding, task control, and graphics were all executed in BRAND. This system also supports real-time inference with complex latent variable models like Latent Factor Analysis via Dynamical Systems. By providing a framework that is fast, modular, and language-agnostic, BRAND lowers the barriers to integrating the latest tools in neuroscience and machine learning into closed-loop experiments.

neuroscience↗

bioRxiv2021

Large-scale neural recordings with single-cell resolution in human cortex using high-density Neuropixels probes

Recent advances in multi-electrode array technology have made it possible to monitor large neuronal ensembles at cellular resolution. In humans, however, current approaches either restrict recordings to only a few neurons per penetrating electrode or combine the signals of thousands of neurons in local field potential (LFP) recordings. Here, we describe a new probe variant and set of techniques which enable simultaneous recording from over 200 well-isolated cortical single units in human participants during intraoperative neurosurgical procedures using silicon Neuropixels probes. We characterized a diversity of extracellular waveforms with eight separable single unit classes, with differing firing rates, positions along the length of the linear electrode array, spatial spread of the waveform, and modulation by LFP events such as inter-ictal discharges and burst suppression. While some additional challenges remain in creating a turn-key recording system, high-density silicon arrays provide a path for studying human-specific cognitive processes and their dysfunction at unprecedented spatiotemporal resolution.

neuroscience↗

bioRxiv2020

Decoding spoken English phonemes from intracortical electrode arrays in dorsal precentral gyrus

ObjectiveTo evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of classifiers trained to discriminate a comprehensive basis set for speech: 39 English phonemes. We classified neural correlates of spoken-out-loud words in the "hand knob" area of precentral gyrus, which we view as a step towards the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak. ApproachNeural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrodes binned action potential counts or high-frequency local field potential power. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes onset times. Main resultsA linear decoder achieved up to 29.3% classification accuracy (chance = 6%) across 39 phonemes, while a recurrent neural network classifier achieved 33.9% accuracy. Parameter sweeps indicated that performance did not saturate when adding more electrodes or more training data, and that accuracy improved when utilizing time-varying structure in the data. Microphonic contamination and phoneme onset differences modestly increased decoding accuracy, but could be mitigated by acoustic artifact subtraction and using a neural speech onset marker, respectively. SignificanceThe ability to decode a comprehensive set of phonemes using intracortical electrode array signals from a nontraditional speech area suggests that placing electrode arrays in ventral speech areas is a promising direction for speech BCIs.

neuroscience↗

bioRxiv2020

The neural representation of force across grasp types in motor cortex of humans with tetraplegia

Intracortical brain-computer interfaces (iBCIs) have the potential to restore hand grasping and object interaction to individuals with tetraplegia. Optimal grasping and object interaction require simultaneous production of both force and grasp outputs. However, since overlapping neural populations are modulated by both parameters, grasp type could affect how well forces are decoded from motor cortex in a closed-loop force iBCI. Therefore, this work quantified the neural representation and offline decoding performance of discrete hand grasps and force levels in two participants with tetraplegia. Participants attempted to produce three discrete forces (light, medium, hard) using up to five hand grasp configurations. A two-way Welch ANOVA was implemented on multiunit neural features to assess their modulation to force and grasp. Demixed principal component analysis was used to assess for population-level tuning to force and grasp and to predict these parameters from neural activity. Three major findings emerged from this work: 1) Force information was neurally represented and could be decoded across multiple hand grasps (and, in one participant, across attempted elbow extension as well); 2) Grasp type affected force representation within multi-unit neural features and offline force classification accuracy; and 3) Grasp was classified more accurately and had greater population-level representation than force. These findings suggest that force and grasp have both independent and interacting representations within cortex, and that incorporating force control into real-time iBCI systems is feasible across multiple hand grasps if the decoder also accounts for grasp type. Significance StatementIntracortical brain-computer interfaces (iBCIs) have emerged as a promising technology to potentially restore hand grasping and object interaction in people with tetraplegia. This study is among the first to quantify the degree to which hand grasp affects force-related - or kinetic - neural activity and decoding performance in individuals with tetraplegia. The study results enhance our overall understanding of how the brain encodes kinetic parameters across varying kinematic behaviors -- and in particular, the degree to which these parameters have independent versus interacting neural representations. Such investigations are a critical first step to incorporating force control into human-operated iBCI systems, which would move the technology towards restoring more functional and naturalistic tasks.

neuroscience↗

bioRxiv2018

Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis

Speaking is a sensorimotor behavior whose neural basis is difficult to study at the resolution of single neurons due to the scarcity of human intracortical measurements and the lack of animal models. We recorded from electrode arrays in the hand knob area of motor cortex in people with tetraplegia. Neurons in this area, which have not previously been implicated in speech, modulated during speaking and during non-speaking movement of the tongue, lips, and jaw. This challenges whether the conventional model of a motor homunculus division by major body regions extends to the single-neuron scale. Spoken words and syllables could be decoded from single trials, demonstrating the potential utility of intracortical recordings for brain-computer interfaces (BCIs) to restore speech. Two neural population dynamics features previously reported for arm movements were also present during speaking: a large initial condition-invariant signal, followed by rotatory dynamics. This suggests that common neural dynamical motifs may underlie movement of arm and speech articulators.

neuroscience↗