Biology subjects

Noppeney, U.

Publications and source records attributed to Noppeney, U..

At least 19 recordsLinked to original sources

bioRxiv2026

Learned Visual Relevance Shapes Perceptual Inference: Evidence from Eyes to Hands

Perceptual inference requires the brain to decide which signals come from common causes and should hence be integrated or else be segregated. Leading accounts posit that sensory signals are weighted near-optimally by their relative precision consistent with Bayesian principles. It is unknown whether learnt behavioural relevance influences how the brain weighs signals in sensory integration. This study investigated whether visual priority acquired via instrumental learning can reweight sensory inputs in a subsequent unrelated spatial localisation task. In the visual instrumental learning phase, participants responded to one of two differently coloured visual flashes presented in a cluttered environment while withholding their response to the other one. In the audiovisual phase, they located brief sounds presented synchronously midway between brief flashes of the previously relevant target and the competing non-target. Observers' perceived sound location shifted toward the previously relevant target as audiovisual disparity increased. Crucially, this audiovisual spatial bias arose dynamically across the entire cascade of perceptual inference and decision making: from rapid saccades and sustained gaze orienting towards the target location, both occurring after the target's disappearance to initial deviations of manual response trajectories and on a fraction of trials, a change of mind with a return towards the sound as accumulating evidence suggested a separate cause structure. Perceptual inference is thus governed not only by sensory precision but also by visual priorities acquired through instrumental learning. What we learn to prioritise in vision changes where we perceive sounds even in novel and unrelated contexts.

neuroscience↗

bioRxiv2026

Watching Yourself Talk: Motor Experience Sharpens Sensitivity to Gesture-Speech Asynchrony

Language is inherently multisensory, with speech often accompanied by iconic gestures that convey semantic meaning related to actions, objects, or spatial relationships. Although the temporal coordination between speech and gesture is variable, the brain integrates these signals seamlessly. Yet, much of the cognitive mechanisms behind this integration remain unclear. This study investigates whether sensorimotor experience, specifically with ones own speech and gestures, enhances temporal sensitivity through internal forward models that guide audiovisual prediction. Participants first produced sentences with corresponding iconic gestures, which were audiovisually recorded. These recordings were later temporally manipulated and presented in a simultaneity judgment task, where participants evaluated both their own and others recordings. Results revealed narrower temporal binding windows (TBWs), indicating heightened sensitivity to audiovisual asynchrony, when participants judged their own speech-gesture recordings compared to those of others. To further explore the role of motor experience, we analysed individual variability in gesture-speech timing during production and found no reliable relationship between production variability and perceptual sensitivity, suggesting that perceptual precision is not simply a reflection of motor consistency. These findings demonstrate that sensorimotor experience with self-generated movements sharpens multisensory temporal integration, likely via predictive internal models, and underscore the functional role of predictive motor mechanisms in supporting temporal integration across perceptual and action systems.

neuroscience↗

bioRxiv2026

Invisible flashes bias spatial coding in the auditory system

Information integration is widely considered fundamental for human consciousness. Combining spatial ventriloquism and dynamic continuous flash suppression, this psychophysics-EEG study investigated how invisible flashes influence spatial perception of sounds. We show that invisible flashes, even when mislocalized by observers, bias observers perceived sound location. EEG multivariate pattern analyses unravelled that the locations of invisible flashes are encoded in early (150-250ms) neural activity. Crucially, even though this visuospatial information rapidly dissipates for invisible mislocalized flashes, being no longer decodable from neural activity after 300ms, it progressively biases neural encoding of sound location (300-600ms), thereby ultimately inducing a perceptual ventriloquist illusion. Our results demonstrate that subjective awareness is not a prerequisite for integrating information across the senses. Spatial information that dissipates from the visual system and evades awareness biases neural encoding and perception of sound location. These results pose challenges for leading theories of consciousness as a unified all-or-none phenomenon.

neuroscience↗

bioRxiv2026

Feeling the music: Audiotactile encoding of temporal structure in the human brain

In everyday situations, like a rock party or an organ concert, we feel music vibrating through our bodies. How do these vibrotactile signals influence music processing? How do they aid auditory scene analysis? Combining psychophysics, fMRI and time-resolved EEG decoding, this work reveals how the brain encodes the temporal structure of music (beat and envelope) across audition and touch and uses this information to guide multisensory integration and segregation in simple and more complex perceptual scenes. Participants experienced monophonic and polyphonic piano pieces through auditory, vibrotactile and audiotactile stimulation. Vibrotactile signals improved the detection of a brief target embedded in music, establishing the functional relevance of audiotactile integration in naturalistic settings. fMRI and EEG multivariate decoding revealed that auditory and tactile beat information converged in planum temporale and parietal operculum, albeit through distinct neural dynamics and representations. Superior temporal cortices reliably encoded envelope information from audition, but only weakly from touch. Nevertheless, vibrotactile signals significantly enhanced neural encoding of auditory beat as early as 100 ms, and envelope representations from 250 ms onward. These encoding benefits were associated with superadditive interactions in primary auditory cortex, where tactile signals sharpen and amplify auditory envelope representations. In complex polyphonic music, touch further amplified the segregation and encoding of temporally coherent auditory streams. Our findings highlight the important, yet largely unexplored influence of touch on auditory processing, enriching music perception and supporting auditory scene analysis in real-world environments.

neuroscience↗

bioRxiv2023

Perceptual uncertainty explains activation differences between audiovisual congruent speech and McGurk stimuli

Face-to-face communication relies on the integration of acoustic speech signals with the corresponding facial articulations. While the McGurk illusion is widely used as an index of audiovisual speech integration, critics argue that it arises from perceptual processes that differ categorically from natural speech recognition. Conversely, Bayesian theoretical frameworks suggest that both the illusory McGurk and the veridical audiovisual congruent speech percepts result from probabilistic inference based on noisy sensory signals. According to these models, the inter-sensory conflict in McGurk stimuli may only increase observers perceptual uncertainty. This functional magnetic resonance imaging (fMRI) study presented participants (20 male and 24 female) with audiovisual congruent, incongruent, and McGurk stimuli along with their unisensory counterparts in a syllable categorization task. Behaviorally, observers response entropy was greater for McGurk compared to congruent audiovisual stimuli. At the neural level, McGurk stimuli increased activations in a widespread neural system, extending from the inferior frontal sulci (IFS) to the pre-supplementary motor area (pre-SMA) and insulae, typically involved in cognitive control processes. Crucially, in line with Bayesian theories these activation increases were fully accounted for by observers perceptual uncertainty as measured by their response entropy. Our findings suggest that McGurk and congruent speech processing rely on shared neural mechanisms, thereby supporting the McGurk illusion as a valid measure of natural audiovisual speech perception. Significance StatementEffective face-to-face communication relies on integrating acoustic speech signals with the corresponding facial articulations. While McGurk illusion is extensively used to study audiovisual speech perception, recent critiques argue that it may be categorically different from typical speech recognition because of the conflict between the audiovisual inputs. This study demonstrates that McGurk stimuli increase activations in a network of regions typically involved in cognitive control. Crucially, the activation differences between McGurk and normal speech stimuli could be fully accounted for by the variation in observers perceptual uncertainties. Our results suggest that McGurk and congruent audiovisual speech stimuli rely on shared neural mechanisms - thereby supporting the validity of the McGurk illusion as a tool for studying natural audiovisual speech perception.

neuroscience↗

bioRxiv2023

rTMS to right parietal cortex decreases the precision of spatial priors in perceptual decision making

Throughout life human observers make perceptual decisions under uncertainty guided by prior knowledge about the worlds causal structure and properties. According to Bayesian probability theory, optimal decision making relies on integrating prior knowledge with current sensory inputs, weighted by their relative precisions (i.e., inverse of uncertainty). Thus, representing uncertainty is essential for optimal decisions. Although extensive research suggests that human perceptual decision making follows Bayesian principles, the neural underpinnings of priors and their uncertainties remain elusive. In this five-day study, we employed psychophysics, Bayesian causal inference models, and offline low-frequency (i.e., 1Hz) transcranial magnetic stimulation (TMS) to probe the role of right intraparietal sulcus (IPS), a key region for spatial processing, in the integration of prior knowledge with auditory/visual inputs for spatial decisions. Model-free and Bayesian modelling analyses consistently showed a reduction in the precision of observers long-term spatial prior and in the influence of their previous spatial choices on their current spatial decisions for right IPS-TMS compared to sham-TMS. In contrast, the causal prior and the auditory/visual uncertainties remained unaffected. The results show that offline IPS-TMS can selectively reduce the precision or influence of observers long-term spatial prior and their short-term spatial expectations on perceptual decisions, without affecting their causal prior or sensory uncertainties (i.e., likelihood). Our findings provide causal evidence for the role of parietal cortex, situated at the top of the audiovisual spatial processing hierarchy, in encoding the uncertainty of spatial - but not causal - priors during perceptual decision-making. Significance statementPerceptual decisions under uncertainty are pervasive in everyday life. Substantial evidence suggests that humans perform perceptual decisions near-optimally. They combine sensory inputs with prior knowledge about the signals causal structure and locations weighted by their uncertainties. Yet, the neural underpinnings remain elusive. Combining psychophysics, Bayesian models, and offline low-frequency inhibitory transcranial magnetic stimulation (TMS), we provide causal evidence that the parietal cortex is crucial for encoding the uncertainty of spatial - but not causal - priors during perceptual decision-making. Intriguingly, observers relied less on their long-term and short-term prior spatial expectations after parietal-TMS, as compared to sham-TMS. These results provide important insights into the neural substrates of priors and their uncertainties as key ingredients for near-optimal decisions consistent with normative Bayesian principles.

neuroscience↗

bioRxiv2023

Computations and neural dynamics of audiovisual causal and perceptual inference in schizophrenia

Hallucinations and perceptual abnormalities in psychosis are thought to arise from imbalanced integration of prior information and sensory inputs during perceptual inference. In this study, we combined psychophysics, Bayesian modelling and electroencephalography (EEG) to investigate potential changes in perceptual and causal inference in medicated individuals with schizophrenia when exposed to audiovisual sequences with varying numbers of flashes and beeps from either common or independent sources. Our findings reveal that individuals with schizophrenia, like their healthy controls, balance sensory integration and segregation in line with Bayesian causal inference rather than resorting to simpler heuristics. Both groups showed comparable weighting of prior information regarding the signals causal structure, with the schizophrenia group slightly overweighting prior information about the number of flashes or beeps. At the neural level, both groups computed Bayesian causal inference through dynamic encoding of perceptual estimates that segregate and flexibly combine audiovisual inputs. In conclusion, our results demonstrate that the computational and neural mechanisms of multisensory perceptual and causal inference remain remarkably intact in medicated individuals with schizophrenia during flash-beep scenarios.

neuroscience↗

bioRxiv2023

Metacognition in the audiovisual McGurk illusion: perceptual and causal confidence

Almost all decisions in everyday life rely on multiple sensory inputs that can come from common or independent causes. These situations invoke perceptual uncertainty about environmental properties and the signals causal structure. Using the audiovisual McGurk illusion this study investigated how observers formed perceptual and causal confidence judgments in multisensory integration tasks under causal uncertainty. Observers were presented with spoken syllables, their corresponding articulatory lip movements, or their congruent and McGurk combinations (e.g. auditory B/P with visual G/K). Observers reported their perceived auditory syllable, the causal structure and confidence for each judgement. Observers were more accurate and confident on congruent than unisensory trials. Their perceptual and causal confidence were tightly related over trials as predicted by the interactive nature of perceptual and causal inference. Further, observers assigned comparable perceptual and causal confidence to veridical G/K percepts on audiovisual congruent trials and their causal and perceptual metamers on McGurk trials (i.e. illusory G/K percepts). Thus, observers metacognitively evaluate the integrated audiovisual percept with limited access to the conflicting unisensory stimulus components on McGurk trials. Collectively, our results suggest that observers form meaningful perceptual and causal confidence judgments about multisensory scenes that are qualitatively consistent with principles of Bayesian causal inference.

neuroscience↗

bioRxiv2023

Audiovisual integration is preserved in older adults across the cortical hierarchy

Effective interactions with the environment rely on integration of multisensory signals: our brains must efficiently combine signals that share a common source, and segregate those that do not. Healthy ageing can change or impair this process. This functional magnetic resonance imaging study assessed the neural mechanisms underlying age differences in the integration of auditory and visual spatial cues. Participants were presented with synchronous audiovisual signals at various degrees of spatial disparity and indicated their perceived sound location. Behaviourally, older adults were able to maintain localisation accuracy, albeit with longer response times. At the neural level, they integrated auditory and visual cues into spatial representations along dorsal auditory and visual processing pathways similarly to their younger counterparts, but showed greater activations in a widespread system of frontal, temporal and parietal areas. According to multivariate Bayesian decoding, these areas encoded critical stimulus information beyond that which was encoded in the brain areas commonly activated by both groups. Surprisingly, however, the boost in information provided by these areas with age-related activation increases was comparable across the two age groups. This dissociation--between comparable response accuracy and information encoded in brain activity patterns across the two age groups, but age-related increases in response times and regional activations--suggests that older participants accumulate noisier sensory evidence for longer, to maintain reliable neural encoding of stimulus-relevant information and thus preserve localisation accuracy.

neuroscience↗

bioRxiv2021

Audiovisual adaptation is expressed in spatial and decisional codes

The brain adapts dynamically to the changing sensory statistics of its environment. The neural circuitries and representations that support this cross-sensory plasticity remain unknown. We combined psychophysics and model-based representational fMRI and EEG to characterize how the adult human brain adapts to misaligned audiovisual signals. We show that audiovisual adaptation moulds regional BOLD-responses and fine-scale activity patterns in a widespread network from Heschls gyrus to dorsolateral prefrontal cortices. Crucially, audiovisual recalibration relies on distinct spatial and decisional codes that are expressed with opposite gradients and timecourses across the auditory processing hierarchy. Early activity patterns in auditory cortices encode sounds in a continuous space that flexibly adapts to misaligned visual inputs. Later activity patterns in frontoparietal cortices code decisional uncertainty consistent with these spatial transformations. Our findings demonstrate that regions throughout the auditory processing hierarchy multiplex spatial and decisional codes to adapt flexibly to the changing sensory statistics in the environment.

neuroscience↗

bioRxiv2020

The influence of auditory attention on rhythmic speech entrainment

Language comprehension relies on integrating words into progressively more complex structures, like phrases and sentences. This hierarchical structure building is reflected in rhythmic neural activity across multiple timescales in E/MEG (Ding et al., 2016, 2017). How does selective attention across levels of the hierarchy influence the expression of these rhythms? We investigated these questions in an EEG study of 72 healthy human volunteers listening to streams of monosyllabic isochronous English words that were either unrelated (scrambled condition) or composed of four-word-sequences building meaningful sentences (sentential condition). Importantly, there were no physical cues between four-word-sentences but boundaries were marked by syntactic structure and thematic role assignment. Participants were divided into three attention groups: from passive listening (passive group) to attending to individual words (word group) or sentences (sentence group). The passive and word group were naive to the sentential structure of the stimulus material, while the sentence group were not. We found significant entrainment at word- and sentence rate across all three groups, with sentence entrainment linked to left middle temporal gyrus and right superior temporal gyrus. Goal-directed attention to words did not enhance word-rate-entrainment suggesting that word entrainment relies on largely automatic mechanisms. Importantly, goal-directed attention to sentences relative to words significantly increased sentence-rate-entrainment over left inferior frontal gyrus. This attentional modulation of rhythmic EEG activity at the sentential level highlights the role of attention in integrating individual words into complex linguistic structures. SIGNIFICANCE STATEMENTNeural activity is known to entrain to physical characteristics of auditory stimuli. However, entrainment also occurs with structures lacking physical cues but rather require comprehension of the stimulus meaning - for example, entrainment to sentences in speech even without acoustic gaps separating these higher linguistic structures. We investigated how goal-directed attention to low-level (words) and high-level (sentences) linguistic structures influences entrainment strength. Whilst sentence entrainment occurred independently of selective attention, it increased with goal-directed attention towards sentences. Conversely, no such attentional effect was found for word entrainment. While goal-directed attention towards sentences strengthens entrainment, it is no prerequisite for it to occur, suggesting that low attentional effort is required for sentence comprehension, potentially reflecting the importance of speech in humans.

neuroscience↗

bioRxiv2019

Spatial attention and expectations rely on modality-specific and multisensory mechanisms

In our natural environment, the brain needs to combine signals from multiple sensory modalities into a coherent percept. While spatial attention guides perceptual decisions by prioritizing processing of signals that are task-relevant, spatial expectations encode the probability of signals over space. Previous studies have shown that behavioral effects of spatial attention generalize across sensory modalities. However, because they manipulated spatial attention as signal probability over space, these studies could not dissociate attention and expectation or assess their interaction. In two experiments, we orthogonally manipulated spatial attention (i.e., task-relevance) and expectation (i.e., signal probability) selectively in one sensory modality (i.e., primary modality) (experiment 1: audition, experiment 2: vision) and assessed their effects on primary and secondary sensory modalities in which attention and expectation were held constant. Our results show behavioral effects of spatial attention that are comparable for audition and vision as primary modalities; yet, signal probabilities were learnt more slowly in audition, so that spatial expectations were formed later in audition than vision. Critically, when these differences in learning between audition and vision were accounted for, both spatial attention and expectation affected responses more strongly in the primary modality in which they were manipulated, and generalized to the secondary modality only in an attenuated fashion. Collectively, our results suggest that both spatial attention and expectation rely on modality-specific and multisensory mechanisms.

neuroscience↗

bioRxiv2019

Using the past to estimate sensory uncertainty

To form the most reliable percept of the environment, the brain needs to represent sensory uncertainty. Current theories of perceptual inference assume that the brain computes sensory uncertainty instantaneously and independently for each stimulus.\n\nIn a series of psychophysics experiments human observers localized auditory signals that were presented in synchrony with spatially disparate visual signals. Critically, the visual noise changed dynamically over time with or without intermittent jumps. Our results show that observers integrate audiovisual inputs weighted by sensory reliability estimates that combine information from past and current signals as predicted by an optimal Bayesian learner or approximate strategies of exponential discounting\n\nOur results challenge classical models of perceptual inference where sensory uncertainty estimates depend only on the current stimulus. They demonstrate that the brain capitalizes on the temporal dynamics of the external world and estimates sensory uncertainty by combining past experiences with new incoming sensory signals.

neuroscience↗

bioRxiv2019

Resolving multisensory and attentional influences across cortical depth in sensory cortices

In our environment our senses are bombarded with a myriad of signals, only a subset of which is relevant for our goals. Using sub-millimeter-resolution fMRI at 7T we resolved BOLD-response and activation patterns across cortical depth in early sensory cortices to auditory, visual and audiovisual stimuli under auditory or visual attention. In visual cortices, auditory stimulation induced widespread inhibition irrespective of attention, whereas auditory relative to visual attention suppressed mainly central visual field representations. In auditory cortices, visual stimulation suppressed activations, but amplified responses to concurrent auditory stimuli, in a patchy topography. Critically, multisensory interactions in auditory cortices were stronger in deeper laminae, while attentional influences were greatest at the surface. These distinct depth-dependent profiles suggest that multisensory and attentional mechanisms regulate sensory processing via partly distinct circuitries. Our findings are crucial for understanding how the brain regulates information flow across senses to interact with our complex multisensory world.

neuroscience↗

bioRxiv2018

To integrate or not to integrate: Temporal dynamics of Bayesian Causal Inference

To form a percept of the environment, the brain needs to solve the binding problem - inferring whether signals come from a common cause and be integrated, or come from independent causes and be segregated. Behaviourally, humans solve this problem near-optimally as predicted by Bayesian Causal Inference; but, the neural mechanisms remain unclear. Combining Bayesian modelling, electroencephalography (EEG), and multivariate decoding in an audiovisual spatial localization task, we show that the brain accomplishes Bayesian Causal Inference by dynamically encoding multiple spatial estimates. Initially, auditory and visual signal locations are estimated independently; next, an estimate is formed that combines information from vision and audition. Yet, it is only from 200 ms onwards that the brain integrates audiovisual signals weighted by their bottom-up sensory reliabilities and top-down task-relevance into spatial priority maps that guide behavioural responses. Critically, as predicted by Bayesian Causal Inference, these spatial priority maps take into account the brains uncertainty about the worlds causal structure and flexibly arbitrate between sensory integration and segregation. The dynamic evolution of perceptual estimates thus reflects the hierarchical nature of Bayesian Causal Inference, a statistical computation, crucial for effective interactions with the environment.

neuroscience↗

bioRxiv2018

The neural dynamics of hierarchical Bayesian inference in multisensory perception

Transforming the barrage of sensory signals into a coherent multisensory percept relies on solving the binding problem - deciding whether signals come from a common cause and should be integrated, or instead be segregated. Human observers typically arbitrate between integration and segregation consistent with Bayesian Causal Inference, but the neural mechanisms remain poorly understood. We presented observers with audiovisual sequences that varied in the number of flashes and beeps. Combining Bayesian modelling and EEG representational similarity analyses, we show that the brain initially represents the number of flashes and beeps and their numeric disparity mainly independently. Later, it computes them by averaging the forced-fusion and segregation estimates weighted by the probabilities of common and independent cause models (i.e. model averaging). Crucially, prestimulus oscillatory alpha power and phase correlate with observers prior beliefs about the worlds causal structure that guide their arbitration between sensory integration and segregation.

neuroscience↗

bioRxiv2018

Older adults sacrifice response speed to preserve multisensory integration performance

Ageing has been shown to impact multisensory perception, but the underlying computational mechanisms are unclear. For effective interactions with the environment, observers should integrate signals that share a common source, weighted by their reliabilities, and segregate those from separate sources. Observers are thought to accumulate evidence about the worlds causal structure over time until a decisional threshold is reached.\n\nCombining psychophysics and Bayesian modelling, we investigated how ageing affects audiovisual perception of spatial signals. Older and younger adults were comparable in their final localisation and common-source judgement responses under both speeded and unspeeded conditions, but were disproportionately slower for audiovisually incongruent trials.\n\nBayesian modelling showed that ageing did not affect the ability to arbitrate between integration and segregation under either unspeeded or speeded conditions. However, modelling the within-trial dynamics of evidence accumulation under speeded conditions revealed that older observers accumulate noisier auditory representations for longer, set higher decisional thresholds, and have impaired motor speed. Older observers preserve audiovisual localisation performance, despite noisier sensory representations, by sacrificing response speed.

neuroscience↗

bioRxiv2018

Distinct neural mechanisms and temporal constraints govern a cascade of audiotactile interactions

Asynchrony is a critical cue informing the brain whether sensory signals are caused by a common source and should be integrated or segregated. It is unclear how the brain binds audiotactile signals into behavioural benefits depending on their asynchrony. Participants actively responded (psychophysics) or passively attended (electroencephalogrpahy) to noise bursts, taps-to-the-face, and their audiotactile (AT) combinations at seven audiotactile asynchronies: 0, {+/-}20, {+/-}70, and {+/-}500ms. Observers were faster at detecting AT than unisensory stimuli, maximally for synchronous stimulation and declining within a [≤]70ms temporal integration window. We observed AT interactions for (1) near-synchronous stimuli within a [≤]20ms temporal integration window for evoked response potentials (ERPs) at 110ms and [~]400ms, (2) specifically {+/-}70ms asynchronies, across the P200 ERP and theta-band inter-trial coherence (ITC) and power at [~]200ms, with a frontocentral topography, and (3) beta-band power across several asynchronies. Our results suggest that early AT interactions for ERP and theta-band ITC and power mediate behavioural response facilitation within a [≤]70ms temporal integration window, but beta-band power reflects AT interactions that are less relevant for behaviour. This diversity of temporal profiles and constraints demonstrates how audiotactile integration unfolds in a cascade of interactions to generate behavioural benefits.

neuroscience↗