Biology subjects

Beauchamp, M. S.

Publications and source records attributed to Beauchamp, M. S..

18 recordsLinked to original sources

bioRxiv2026

Size Scaling of the Electrochemical Performance of Ti3C2Tx MXene Microelectrode Arrays for Electrophysiological Recording and Stimulation

Miniaturized neural interfaces for research, diagnostics, and neuromodulation therapies require electrode materials that maintain low impedance and high charge injection capacity as device dimensions shrink to ensure high-quality recordings and safe stimulation. Conventional interfaces rely on metals like platinum (Pt), which are limited by intrinsically high impedance and low charge transfer capacity, reducing their performance in sub-100 {micro}m applications. Ti3C2Tx MXene has emerged as a promising alternative for high-density recording and stimulation interfaces, though the fundamental charge transfer mechanisms governing its performance remain poorly understood. This study evaluates Ti3C2Tx MXene microelectrodes across a range of diameters (25 - 500 {micro}m) and systematically elucidates the mechanisms governing their recording and stimulation capabilities. Electrochemical impedance spectroscopy, cyclic voltammetry, and voltage transient measurements - supported by equivalent-circuit modeling - revealed enhanced recording and stimulation capabilities of the MXene microelectrodes over size-matched Pt microelectrodes, attributed to reduced charge-transfer resistance and increased double-layer capacitance. Finally, varying the volume and concentration of the spray-coated Ti3C2Tx films showed that increased MXene concentration and volume enhanced performance by creating thicker, rougher interfaces. Together, these results establish Ti3C2Tx MXene as a promising electrode material with exceptional performance at the microscale.

bioengineering↗

bioRxiv2025

A Deep Neural Network Trained on Congruent Audiovisual Speech Reports the McGurk Effect

In the McGurk effect, perception of an auditory syllable changes dramatically when it is paired with an incongruent visual syllable, countering our intuition that speech perception is solely an auditory process. The dominant modeling framework for the study of audiovisual speech perception is that of Bayesian causal inference, but current Bayesian models are unable to predict the wide range of percepts evoked by McGurk syllables. We explored whether a deep neural network (DNN) known as AVHuBERT could provide an alternative modeling framework. AVHuBERT model variants were presented with McGurk syllables consisting of auditory "ba" paired with visual "ga" recorded from 8 different talkers. AVHuBERT identified McGurk syllables as something other than "ba" at a rate of 59%, demonstrating a robust McGurk effect. The rate of the McGurk effect was similar to that observed in humans: 100 participants presented with the same McGurk syllables reported non-"ba" percepts on 56% of trials. AVHubert variants and humans produced a wide variety of responses to McGurk syllables, including the canonical McGurk fusion percept of "da", responses without any initial consonant such as "ah" and responses with other initial consonants such as "fa". The ability to predict percepts experienced by humans but not predicted by current Bayesian models suggest that DNNs and Bayesian models may provide complementary windows into the perceptual mechanisms underlying human audiovisual speech perception.

neuroscience↗

bioRxiv2025

Dissociating stimulus encoding and task demands in ECoG responses from human visual cortex

Brain responses to sensory stimuli depend on the specific task performed by the observer. Previously, fMRI measurements in ventral temporal cortex (VTC) showed that BOLD responses to visual input scale with task demands. This scaling of BOLD responses is thought to be driven by the engagement of cognitive functions required for the task. To better understand how these cognitive task demands influence neural activity in VTC, we measured electrocorticography (ECoG) responses to images of varying contrast in two human participants during either a fixation or image categorization task. ECoG high frequency broadband activity (>70 Hz) showed that local neuronal responses increased with image contrast as expected for sensory encoding, and were scaled by task demands approximately 200 ms after stimulus onset. In contrast, ECoG low frequency activity in the alpha/beta range (8-28 Hz) was insensitive to image contrast and showed larger decreases with increasing task demands. These results indicate that high frequencies in VTC reflect both the encoding of sensory inputs and task demands, similar to prior BOLD response measurements, whereas low frequency oscillations represent task demands, not sensory inputs per se. In line with the interpretation that low frequency oscillations reflect pulsed inhibition, we speculate that a power decrease in low frequency oscillations amplifies neural activity in VTC in support of visual task demands.

neuroscience↗

bioRxiv2024

Integrating Data Across Oscillatory Power Bands Predicts the Epileptogenic Zone: the Frequency Range Explorer Epileptogenic Zone (FREEZ) Identification Algorithm

Accurate identification of the seizure onset zone (SOZ) using intracranial electroencephalography (iEEG) remains challenging. Although diverse methods have leveraged spectral features to classify patient outcomes, few approaches focus on identifying individual electrodes within the SOZ or integrate a broad spectrum of frequency ranges within a single model. We developed an interpretable machine learning model that integrates power across delta, theta, alpha, beta, gamma, and high-gamma frequencies over time to identify the SOZ. For 1,511 electrodes implanted across 21 patients, we computed the mean spectral power in each frequency band for the first 20 seconds after seizure onset and analyzed the differences in power between SOZ and non-SOZ electrodes. In patients who were seizure-free after surgery (n = 14), electrodes within the SOZ showed significantly higher area under the curve (AUC) for mean power over time in the first 20 seconds after seizure onset compared to electrodes outside the SOZ in the alpha (p = 0.0272), beta (p = 0.0263), gamma (p = 0.0013), and high gamma (p = 0.0086) ranges. Additionally, electrodes within the SOZ in patients that became seizure-free after surgery had significantly higher AUC compared to equivalent electrodes in patients who did not become seizure-free after surgery (n = 7) in the gamma (p = 0.0145) and high gamma (p = 0.0024) power ranges. We trained a stacked random forest ensemble model using these features over time to label electrodes within the SOZ. Leave-one-out patient cross validation of the machine learning model yielded a 96.6% positive predictive value and 99.9% specificity for identifying electrodes within the SOZ. Our dataset included a diverse array of seizure onset patterns, which were all classified accurately by the model. A second model was trained to predict post-operative seizure freedom, yielding 95.2% accuracy for predicting seizure outcome based on a planned resection. This two-model design mirrors clinical workflow, first localizing SOZ electrodes to support surgical planning, then predicting outcome based on a surgical plan. An advantage of our interpretable machine learning approach is the ability to interrogate our models to understand how predictions are made. For electrode classification, the model weighed beta (0.66 {+/-} 0.07), high gamma (0.54 {+/-} 0.06), and delta (0.51 {+/-} 0.06) power bands most heavily. Viewing the models frequency band weights over time reveals that the model identified a pattern resembling the "fingerprint of the epileptogenic zone", reinforcing the importance of this dominant fundamental neurophysiologic signature of seizure onset.

neuroscience↗

bioRxiv2024

The Noisy Encoding of Disparity Model Predicts Perception of the McGurk Effect in Native Japanese Speakers

The McGurk effect is an illusion that demonstrates the influence of information from the face of the talker on the perception of auditory speech. The diversity of human languages has prompted many intercultural studies of the effect, including in native Japanese speakers. Studies of large samples of native English speakers have shown that the McGurk effect is characterized by high variability, both in the susceptibility of different individuals to the illusion and in the frequency with which different experimental stimuli induce the illusion. The noisy encoding of disparity (NED) model of the McGurk effect uses Bayesian principles to account for this variability by separately estimating the susceptibility and sensory noise for each individual and the strength of each stimulus. To test whether the NED model could account for McGurk perception in a non-Western culture, we applied it to data collected from 80 native Japanese-speaking participants. Fifteen different McGurk stimuli were presented, along with audiovisual congruent stimuli. The McGurk effect was highly variable across stimuli and participants, with the percentage of illusory fusion responses ranging from 3% to 78% across stimuli and from 0% to 91% across participants. Despite this variability, the NED model accurately predicted perception, predicting fusion rates for individual stimuli with 2.1% error and for individual participants with 2.4% error. Stimuli containing the unvoiced pa/ka pairing evoked more fusion responses than the voiced ba/ga pairing. Model estimates of sensory noise was correlated with participant age, with greater sensory noise in older participants. The NED model of the McGurk effect offers a principled way to account for individual and stimulus differences when examining the McGurk effect within and across cultures.

neuroscience↗

bioRxiv2024

The Effect on Speech-in-Noise Perception of Real Faces and Synthetic Faces Generated with either Deep Neural Networks or the Facial Action Coding System

The prevalence of synthetic talking faces in both commercial and academic environments is increasing as the technology to generate them grows more powerful and available. While it has long been known that seeing the face of the talker improves human perception of speech-in-noise, recent studies have shown that synthetic talking faces generated by deep neural networks (DNNs) are also able to improve human perception of speech-in-noise. However, in previous studies the benefit provided by DNN synthetic faces was only about half that of real human talkers. We sought to determine whether synthetic talking faces generated by an alternative method would provide a greater perceptual benefit. The facial action coding system (FACS) is a comprehensive system for measuring visually discernible facial movements. Because the action units that comprise FACS are linked to specific muscle groups, synthetic talking faces generated by FACS might have greater verisimilitude than DNN synthetic faces which do not reference an explicit model of the facial musculature. We tested the ability of human observers to identity speech-in-noise accompanied by a blank screen; the real face of the talker; and synthetic talking face generated either by DNN or FACS. We replicated previous findings of a large benefit for seeing the face of a real talker for speech-in-noise perception and a smaller benefit for DNN synthetic faces. FACS faces also improved perception, but only to the same degree as DNN faces. Analysis at the phoneme level showed that the performance of DNN and FACS faces was particularly poor for phonemes that involve interactions between the teeth and lips, such as /f/, /v/, and /th/. Inspection of single video frames revealed that the characteristic visual features for these phonemes were weak or absent in synthetic faces. Modeling the real vs. synthetic difference showed that increasing the realism of a few phonemes could substantially increase the overall perceptual benefit of synthetic faces, providing a roadmap for improving communication in this rapidly developing domain.

neuroscience↗

bioRxiv2023

YAEL: Your Advanced Electrode Localizer

Intracranial electroencephalography (iEEG) provides a unique opportunity to measure human brain function with implanted electrodes. A key step in neuroscience inference from iEEG is localizing the electrodes relative to individual subject anatomy and identified regions in brain atlases. While there are number of workflows for electrode localization, most suffer from one or more limitations. The first limitation is a lack of integration: scientists must install and use different software packages for each localization step. Second, they are inefficient: while most iEEG analysis steps can be automated, electrode localization is still largely a manual process. Third, most current tools are limited to the localization process itself, leaving users without the ability to create high-quality visualizations for clinical and research purposes. We developed YAEL (Your Advanced Electrode Localizer) to overcome these limitations. First, YAEL is completely integrated: a single easy-to-use graphical user interface (GUI) controls every step of the localization process. Second, YAEL uses a flexible 3D viewer and automation tools to make accurate localization of electrodes quick and easy. Third, after localization is complete, YAEL leverages the same viewer to create high-quality visualizations of electrode data including identified brain areas from atlases; the response to experimental tasks measured with iEEG; and clinical measures such as epileptiform activity or the results of electrical stimulation mapping. YAEL contains more than 30,000 lines of code, is free and open source, and can be installed in minutes on Mac, Windows and Linux platforms from https://yael.wiki. User interactions with YAEL occur through a web browser ensuring a familiar user experience and consistent operation across platforms and whether YAEL is used locally or deployed in the cloud.

neuroscience↗

bioRxiv2021

Orientation and color tuning of the human visual gamma rhythm

Narrowband gamma oscillations (NBG: [~]20-60Hz) in visual cortex reflect rhythmic fluctuations in population activity generated by underlying circuits tuned for stimulus location, orientation, and color. Consequently, the amplitude and frequency of induced NBG activity is highly sensitive to these stimulus features. For example, in the non-human primate, NBG displays biases in orientation and color tuning at the population level. Such biases may relate to recent reports describing the large-scale organization of single-cell orientation and color tuning in visual cortex, thus providing a potential bridge between measurements made at different scales. Similar biases in NBG population tuning have been predicted to exist in the human visual cortex, but this has yet to be fully examined. Using intracranial recordings from human visual cortex, we investigated the tuning of NBG to orientation and color, both independently and in conjunction. NBG was shown to display a cardinal orientation bias (horizontal) and also an end- and mid-spectral color bias (red/blue and green). When jointly probed, the cardinal bias for orientation was attenuated and an end-spectral preference for red and blue predominated. These data both elaborate on the close, yet complex, link between the population dynamics driving NBG oscillations and known feature selectivity biases in visual cortex, adding to a growing set of stimulus dependencies associated with the genesis of NBG. Together, these two factors may provide a fruitful testing ground for examining multi-scale models of brain activity, and impose new constraints on the functional significance of the visual gamma rhythm. Significance StatementOscillations in electrophysiological activity occur in visual cortex in response to stimuli that strongly drive the orientation or color selectivity of visual neurons. The significance of this induced gamma rhythm to brain function remains unclear. Answering this question requires understanding how and why some stimuli can reliably generate gamma activity while others do not. We examined how different orientations and colors independently and jointly modulate gamma oscillations in the human brain. Our data show gamma oscillations are greatest for certain orientations and colors that reflect known biases in visual cortex. Such findings complicate the functional significance of gamma activity, but open new avenues for linking circuits to population dynamics in visual cortex. ClassificationNeuroscience

neuroscience↗

bioRxiv2020

RAVE: comprehensive open-source software for reproducible analysis and visualization of intracranial EEG data

Direct recording of neural activity from the human brain using implanted electrodes (iEEG, intracranial electroencephalography) is a fast-growing technique in human neuroscience. While the ability to record from the human brain with high spatial and temporal resolution has advanced our understanding, it generates staggering amounts of data: a single patient can be implanted with hundreds of electrodes, each sampled thousands of times a second for hours or days. The difficulty of exploring these vast datasets is the rate-limiting step in discovery. To overcome this obstacle, we created RAVE ("R Analysis and Visualization of iEEG"). All components of RAVE, including the underlying "R" language, are free and open source. User interactions occur through a web browser, making it transparent to the user whether the back-end data storage and computation is occurring on a local machine, a lab server, or in the cloud. Without writing a single line of computer code, users can create custom analyses, apply them to data from hundreds of iEEG electrodes, and instantly visualize the results on cortical surface models. Multiple types of plots are used to display analysis results, each of which can be downloaded as publication-ready graphics with a single click. RAVE consists of nearly 50,000 lines of code designed to prioritize an interactive user experience, reliability and reproducibility.

neuroscience↗

bioRxiv2020

Causal inference explains the stimulus-level relationship between the McGurk Effect and auditory speech perception

The McGurk effect is widely used as a measure of multisensory integration during speech perception. Two observations have raised questions about the relationship between the effect and everyday speech perception. First, there is high variability in the strength of the McGurk effect across different stimuli and observers. Second, there is low correlation across observers between perception of the McGurk effect and measures of everyday speech perception, such as the ability to understand noisy audiovisual speech. Using the framework of the causal inference of multisensory speech (CIMS) model, we explored the relationship between the McGurk effect, syllable perception, and sentence perception in seven experiments with a total of 296 different participants. Perceptual reports revealed a relationship between the efficacy of different McGurk stimuli created from the same talker and perception of the auditory component of the McGurk stimuli presented in isolation, either with or without added noise. The CIMS model explained this high stimulus-level correlation using the principles of noisy sensory encoding followed by optimal cue combination within a representational space that was identical for McGurk and everyday speech. In other experiments, CIMS successfully modeled low observer-level correlation between McGurk and everyday speech. Variability in noisy speech perception was modeled using individual differences in noisy sensory encoding, while variability in McGurk perception involved additional differences in causal inference. Participants with all combinations of high and low sensory encoding noise and high and low causal inference disparity thresholds were identified. Perception of the McGurk effect and everyday speech can be explained by a common theoretical framework that includes causal inference.

neuroscience↗

bioRxiv2020

Responses to Visual Speech in Human Posterior Superior Temporal Gyrus Examined with iEEG Deconvolution

Experimentalists studying multisensory integration compare neural responses to multisensory stimuli with responses to the component modalities presented in isolation. This procedure is problematic for multisensory speech perception since audiovisual speech and auditory-only speech are easily intelligible but visual-only speech is not. To overcome this confound, we developed intracranial encephalography (iEEG) deconvolution. Individual stimuli always contained both auditory and visual speech but jittering the onset asynchrony between modalities allowed for the time course of the unisensory responses and the interaction between them to be independently estimated. We applied this procedure to electrodes implanted in human epilepsy patients (both male and female) over the posterior superior temporal gyrus (pSTG), a brain area known to be important for speech perception. iEEG deconvolution revealed sustained, positive responses to visual-only speech and larger, phasic responses to auditory-only speech. Confirming results from scalp EEG, responses to audiovisual speech were weaker than responses to auditory- only speech, demonstrating a subadditive multisensory neural computation. Leveraging the spatial resolution of iEEG, we extended these results to show that subadditivity is most pronounced in more posterior aspects of the pSTG. Across electrodes, subadditivity correlated with visual responsiveness, supporting a model in visual speech enhances the efficiency of auditory speech processing in pSTG. The ability to separate neural processes may make iEEG deconvolution useful for studying a variety of complex cognitive and perceptual tasks. Significance statementUnderstanding speech is one of the most important human abilities. Speech perception uses information from both the auditory and visual modalities. It has been difficult to study neural responses to visual speech because visual-only speech is difficult or impossible to comprehend, unlike auditory-only and audiovisual speech. We used intracranial encephalography (iEEG) deconvolution to overcome this obstacle. We found that visual speech evokes a positive response in the human posterior superior temporal gyrus, enhancing the efficiency of auditory speech processing.

neuroscience↗

bioRxiv2020

Linking Activity in Human Superior Temporal Cortex to Perception of Noisy Audiovisual Speech

Regions of the human posterior superior temporal gyrus and sulcus (pSTG/S) respond to the visual mouth movements that constitute visual speech and the auditory vocalizations that constitute auditory speech. We hypothesized that these multisensory responses in pSTG/S underlie the observation that comprehension of noisy auditory speech is improved when it is accompanied by visual speech. To test this idea, we presented audiovisual sentences that contained either a clear auditory component or a noisy auditory component while measuring brain activity using BOLD fMRI. Participants reported the intelligibility of the speech on each trial with a button press. Perceptually, adding visual speech to noisy auditory sentences rendered them much more intelligible. Post-hoc trial sorting was used to examine brain activations during noisy sentences that were more or less intelligible, focusing on multisensory speech regions in the pSTG/S identified with an independent visual speech localizer. Univariate analysis showed that less intelligible noisy audiovisual sentences evoked a weaker BOLD response, while more intelligible sentences evoked a stronger BOLD response that was indistinguishable from clear sentences. To better understand these differences, we conducted a multivariate representational similarity analysis. The pattern of response for intelligible noisy audiovisual sentences was more similar to the pattern for clear sentences, while the response pattern for unintelligible noisy sentences was less similar. These results show that for both univariate and multivariate analyses, successful integration of visual and noisy auditory speech normalizes responses in pSTG/S, providing evidence that multisensory subregions of pSTG/S are responsible for the perceptual benefit of visual speech. Significance StatementEnabling social interactions, including the production and perception of speech, is a key function of the human brain. Speech perception is a complex computational problem that the brain solves using both visual information from the talkers facial movements and auditory information from the talkers voice. Visual speech information is particularly important under noisy listening conditions when auditory speech is difficult or impossible to understand alone Regions of the human cortex in posterior superior temporal lobe respond to the visual mouth movements that constitute visual speech and the auditory vocalizations that constitute auditory speech. We show that the pattern of activity in cortex reflects the successful multisensory integration of auditory and visual speech information in the service of perception.

neuroscience↗

bioRxiv2019

Cross-modal suppression of auditory association cortex by visual speech as a mechanism for audiovisual speech perception

Vision provides a perceptual head start for speech perception because most speech is \"mouth-leading\": visual information from the talkers mouth is available before auditory information from the voice. However, some speech is \"voice-leading\" (auditory before visual). Consistent with a model in which vision modulates subsequent auditory processing, there was a larger perceptual benefit of visual speech for mouth-leading vs. voice-leading words (28% vs. 4%). The neural substrates of this difference were examined by recording broadband high-frequency activity from electrodes implanted over auditory association cortex in the posterior superior temporal gyrus (pSTG) of epileptic patients. Responses were smaller for audiovisual vs. auditory-only mouth-leading words (34% difference) while there was little difference (5%) for voice-leading words. Evidence for cross-modal suppression of auditory cortex complements our previous work showing enhancement of visual cortex (Ozker et al., 2018b) and confirms that multisensory interactions are a powerful modulator of activity throughout the speech perception network.\n\nImpact StatementHuman perception and brain responses differ between words in which mouth movements are visible before the voice is heard and words for which the reverse is true.

physiology↗

bioRxiv2019

Distinct Narrow and Broadband Gamma Responses in Human Visual Cortex

High frequency activity (> 30 Hz) in the neocortical local field potential, typically referred to as the gamma range, is thought to have a critical role in visual perception and cognition more broadly. Historically, animal studies recording from visual cortex documented clear narrowband gamma oscillations (NBG; [~]20-60 Hz) in response to visual stimuli. However, invasive measurements from human neocortex have highlighted a different broadband or high gamma response (BBG; [~]70-150+ Hz). Growing evidence suggests these two forms of gamma response are distinct, but often conceptually or analytically conflated as the same gamma response. Furthermore, recent debate has highlighted that both the occurrence and spectral properties of gamma band activity in visual cortex appears to be dependent on the attributes and class of presented visual stimuli. Using high-density intracranial recordings from human visual cortex, we integrate and extend these findings, dissociating the spectral, temporal and functional properties of NBG and BBG activity. We report results from two experiments, manipulating visual stimulus attributes (contrast-varying gratings) and class (object categories) dissecting the differential properties of NBG and BBG responses. NBG oscillations were only reliably recorded for grating stimuli, while their peak frequency varied with contrast level. Whereas BBG activity was observed in response to all stimulus classes tested, with no systematic change in its spectral features. Temporally, induced NBG was sustained throughout stimulus presentation, in opposition to a more transient response for the BBG. These findings challenge the ubiquity of gamma activity in visual cortex, by clearly dissociating oscillatory and broadband effects.\n\nSignificance StatementNeocortical narrowband gamma oscillations ([~]20-60 Hz) have been implicated in vision and cognition as a mechanism for synchronizing brain regions. Efforts to study this phenomenon have revealed an additional high-gamma range response ([~]70-150+ Hz), which is broadband and non-oscillatory. These different gamma range activities are often conflated in support of the same functional role. Using invasive recordings from human visual cortex, we show that narrow and broadband gamma can be dissociated by spectral, temporal and functional response properties. While broadband gamma responses were more transient to the presentation of all stimuli, narrowband gamma responses were sustained and only occurred reliably to grating stimuli. These differences have important implications for the study, analysis and interpretation of neocortical gamma range activity.

neuroscience↗

bioRxiv2018

Dynamic Electrical Stimulation of Sites in Visual Cortex Produces Form Vision in Sighted and Blind Humans

Visual cortical prosthetics (VCPs) offer the promise of restoring sight to blind patients. Electrical stimulation of a single site in visual cortex can reliably produce a percept of a spot of light in a fixed visual field location, known as a phosphene. Researchers developing VCPs have assumed that multiple phosphenes produced by concurrent stimulation of multiple sites in visual cortex can combine to form a coherent form, like pixels in a visual display. However, existing data do not support this assumption. Therefore, we developed a novel stimulation paradigm for VCPs termed dynamic current steering in which the visual form to be conveyed is traced on the surface of visual cortex by electrically stimulating electrodes in a dynamic sequence. When tested in sighted and blind subjects, this method of stimulating visual cortex allowed for the immediate recognition of a variety of letter shapes without training and with high accuracy.\n\nOne Sentence SummaryStimulating human visual cortex using dynamic patterns of activity allows both blind and sighted patients to perceive visual percepts of useful forms.

neuroscience↗

bioRxiv2018

Phase resetting in human auditory cortex to visual speech

Natural conversation is multisensory: when we can see the speakers face, visual speech cues influence our perception of what is being said. The neuronal basis of this phenomenon remains unclear, though there is indication that phase modulation of neuronal oscillations--ongoing excitability fluctuations of neuronal populations in the brain--provides a mechanistic contribution. Investigating this question using naturalistic audiovisual speech with intracranial recordings in humans, we show that neuronal populations in auditory cortex track the temporal dynamics of unisensory visual speech using the phase of their slow oscillations and phase-related modulations in high-frequency activity. Auditory cortex thus builds a representation of the speech streams envelope based on visual speech alone, at least in part by resetting the phase of its ongoing oscillations. Phase reset could amplify the representation of the speech stream and organize the information contained in neuronal activity patterns. SIGNIFICANCE STATEMENTWatching the speaker can facilitate our understanding of what is being said. The mechanisms responsible for this influence of visual cues on the processing of speech remain incompletely understood. We studied those mechanisms by recording the human brains electrical activity through electrodes implanted surgically inside the skull. We found that some regions of cerebral cortex that process auditory speech also respond to visual speech even when it is shown as a silent movie without a soundtrack. This response can occur through a reset of the phase of ongoing oscillations, which helps augment the response of auditory cortex to audiovisual speech. Our results contribute to discover the mechanisms by which the brain merges auditory and visual speech into a unitary perception.

neuroscience↗

bioRxiv2018

Face Viewing Behavior Predicts Multisensory Gain During Speech Perception

During face viewing, some individuals prefer to fixate the mouth while others fixate the eyes. Individuals who have a history of mouth fixation might have stronger associations between visual and auditory speech, resulting in improved comprehension. First, we measured eye movements during face-viewing and observed high interindividual variability in mouth fixation time. Next, we measured eye movements and comprehension during perception of noisy auditory speech with or without visual speech. When visual speech was present, participants primarily fixated the mouth, but derived substantial benefit compared to noisy auditory speech with high interindividual variability. The benefit of visual speech was predicted by the eye movements made during the initial face-viewing task, but not by eye movements during the noisy speech task. These findings suggest a link between eye movements during face viewing and audiovisual speech perception and suggest that individual histories of visual exposure shape abilities across cognitive domains.

neuroscience↗

bioRxiv2018

Rules Governing Perception of Multiple Phosphenes by Human Observers

Stimulation of single sites in primary visual cortex results in the perception of a small flash of light known as a phosphene. Little is known about how phosphenes from multiple electrodes can be combined to form perception of coherent patterns. Here we examine the percepts reported by human observers as various spatial configurations of 2 to 5 electrodes in visual cortex were stimulated simultaneously. When two electrodes were stimulated simultaneously, subjects reliably perceived either one or two phosphenes depending on the physical distance separating the electrodes. In cases where two phosphenes were perceived, they were located in the same visual field location as when the two electrodes were stimulated separately. Adding a third electrode produced similar results. In several subjects, we obtained combination of 4 to 5 electrodes that generated individual phosphenes when stimulated concurrently. Subjects were able to reliably discriminate between different multiple electrode stimulation patterns that were presented in random order. These results demonstrate that simple pattern information can be conveyed to subjects with surface electrodes spaced at millimeters apart on the cortex.

neuroscience↗