Biology subjects

Kayser, C.

Publications and source records attributed to Kayser, C..

At least 37 records · Page 2Linked to original sources

bioRxiv2022

The history of multisensory discrepancies affects the ventriloquism effect and the immediate aftereffect differentially.

Multisensory integration and recalibration are two processes by which perception deals with discrepant signals. Both are often studied in the spatial ventriloquism paradigm. There, integration is probed by the presentation of discrepant audio-visual stimuli, while recalibration manifests as an aftereffect in subsequent unisensory judgements. Both biases are typically quantified against the degree of audio-visual discrepancy, reflecting the possibility that both may arise from common underlying multisensory principles. We tested a specific prediction of this: that both processes should also scale similarly with the history of multisensory discrepancies experienced in previous trials. Analysing data from ten experiments we confirmed the expected dependency of each bias on the immediately presented discrepancy. And in line with the aftereffect being a cumulative process, this scaled with the discrepancies presented in multiple preceding audio-visual trials. However, the ventriloquism bias did not depend on the history of multisensory discrepancies and also did not depend on the aftereffect biases in previous trials - making these two multisensory processes experimentally dissociable. These findings support the notion that the ventriloquism bias and the aftereffect reflect distinct functions, with integration maintaining a stable percept by reducing immediate sensory discrepancies and recalibration maintaining an accurate percept by accounting for consistent discrepancies.

neuroscience↗

bioRxiv2022

Visual and auditory cortices represent acoustic speech-related information during silent lip reading

Speech is an intrinsically multisensory signal and seeing the speakers lips forms a cornerstone of communication in acoustically impoverished environments. Still, it remains unclear how the brain exploits visual speech for comprehension and previous work debated whether lip signals are mainly processed along the auditory pathways or whether the visual system directly implements speech-related processes. To probe this question, we systematically characterized dynamic representations of multiple acoustic and visual speech-derived features in source localized MEG recordings that were obtained while participants listened to speech or viewed silent speech. Using a mutual-information framework we provide a comprehensive assessment of how well temporal and occipital cortices reflect the physically presented signals and speech-related features that were physically absent but may still be critical for comprehension. Our results demonstrate that both cortices are capable of a functionally specific form of multisensory restoration: during lip reading both reflect unheard acoustic features, with occipital regions emphasizing spectral information and temporal regions emphasizing the speech envelope. Importantly, the degree of envelope restoration was predictive of lip reading performance. These findings suggest that when seeing the speakers lips the brain engages both visual and auditory pathways to support comprehension by exploiting multisensory correspondences between lip movements and spectro-temporal acoustic cues. HighlightsO_LIVisual and auditory cortex represent unheard acoustic information during lip reading C_LIO_LIAuditory cortex emphasizes the acoustic envelope C_LIO_LIVisual cortex emphasizes a pitch signature C_LIO_LITracking of unheard features in auditory cortex is associated with behavior C_LI

neuroscience↗

bioRxiv2021

The context of experienced sensory discrepancies shapes multisensory integration and recalibration differently

Whether two sensory cues interact during perceptual judgments depends on their immediate properties, but as suggested by Bayesian models, also on the observers a priori belief that these originate from a common source. While in many experiments this a priori belief is considered fixed, in real life it must adapt to the momentary context or environment. To understand the adaptive nature of human multisensory perception we investigated the context-sensitivity of spatial judgements in a ventriloquism paradigm. We exposed observers to audio-visual stimuli whose discrepancy either varied over a wider ({+/-} 46{degrees}) or a narrower range ({+/-} 26{degrees}) and hypothesized that exposure to a wider range of discrepancies would facilitate multisensory binding by increasing participants a priori belief about a common source for a given discrepancy. Our data support this hypothesis by revealing an enhanced integration (ventriloquism) bias in the wider context, which was echoed in Bayesian causal inference models fit to participants data, which assigned a stronger a priori integration tendency during the wider context. Interestingly, the immediate ventriloquism aftereffect, a multisensory response bias obtained following a multisensory test trial, was not affected by the contextual manipulation, although participants confidence in their spatial judgments differed between contexts for both integration and recalibration trials. These results highlight the context-sensitivity of multisensory binding and suggest that the immediate ventriloquism aftereffect is not a purely sensory-level consequence of the multisensory integration process.

neuroscience↗

bioRxiv2021

Visuo-proprioceptive integration and recalibration with multiple visual stimuli

To organize the plethora of sensory signals from our environment into a coherent percept, our brain relies on the processes of multisensory integration and sensory recalibration. We here asked how visuo-proprioceptive integration and recalibration are shaped by the presence of more than one potentially relevant visual stimulus, hence paving the way to studying multisensory perception under more naturalistic settings with multiple signals per sensory modality. By manipulating the spatio-temporal correspondence between the hand position and two visual stimuli during a cursor-control task, we contrasted two alternative accounts: that only the temporally more proximal signal enters integration and recalibration similar to a winner-takes-all process, or that the influences of both visual signals superpose. Our results show that integration - and likely also recalibration - are shaped by the superposed contributions of multiple stimuli rather than by only individual ones.

neuroscience↗

bioRxiv2021

The influence of the respiratory cycle on reaction times in sensory-cognitive paradigms

Behavioural and electrophysiological studies point to widespread influence of the state of respiration on brain activity and cognitive performance. Still, the prevalence and relevance of such respiratory-behavioural relations in typical sensory-cognitive tasks remain unclear. We here used a battery of six tasks probing sensory detection, discrimination and short-term memory to address the questions of whether and by how much behaviour covaries with the respiratory cycle. Our results show that participants tended to align their respiratory cycle to the experimental paradigm. Furthermore, their reaction times, but not so much their response accuracy, consistently and significantly covaried with the respiratory cycle, and this effect was strongest when analyzed contingent on the respiratory state at participants responses. The respective effect sizes where comparable to those seen in many typical neurocognitive experimental manipulations. These results support a prominent relation between respiration and sensory-cognitive function and suggest that sensation is intricately linked to rhythmic bodily or interoceptive functions.

neuroscience↗

bioRxiv2021

Task switching costs contribute to the apparent speeding of multisensory reaction times

Faster reaction times for the detection of multisensory compared to unisensory stimuli are considered a hallmark of multisensory integration. While this multisensory redundant signals effect (RSE) has been reproduced many times, it has also been repeatedly criticized as confounding multisensory integration and general task switching effects. When unisensory and multisensory conditions are presented in random order, some trials repeat the same sensory-motor association (e.g. an auditory followed by an auditory trial), while others switch this association (e.g. an auditory followed by a visual trial). This switch may slow down unisensory reaction times and inflate the observed RSE. Following this line of ideas, we used an audio-visual detection task and quantified the RSE for trials derived from pure unisensory blocks and trials from mixed blocks involving a repeat or switch of modalities. The RSE was largest for switch trials and smallest for unisensory trials. In fact, during unisensory blocks the multisensory reaction times did not differ from predictions by the race model, speaking against a genuine multisensory benefit. These results confirm that the observed multisensory RSE can be easily confounded by task switching costs, and suggest that the true benefit of multisensory stimuli for reaction speed may often be overestimated.

neuroscience↗

bioRxiv2021

Postdiction: when temporal regularity drives space perception through pre-stimulus alpha oscillations

During postdiction, the last stimulus of a sequence changes the perception of the preceding ones: in the rabbit illusion, a three-flash series presented regularly in time, but not in space, is - illusory - perceived as spatially regular. Such a reorganization of (spatial) perception could be driven by internal priors, e.g. favoring slow motion for the rabbit illusion. Although postdiction is a ubiquitous phenomenon, its neural underpinnings remain poorly understood. Here, we focused on the role of priors during postdiction and hypothesized that these could be reflected by alpha oscillations (8-12Hz), previously observed to correlate with idiosyncratic biases. We presented human participants with ambiguous visual stimuli that elicited the rabbit illusion on about half the trials, allowing us to contrast MEG-EEG brain responses to the same physical events causing distinct percepts. Given that a strong prior will increase the overall probability of perceiving the illusion, we used the percentage of perceived illusion as a proxy for an individuals prior. We found that high fronto-parietal alpha power was associated with perceiving the sequence according to individual biases: participants with high susceptibility to the illusion would report the illusion, while participants with low susceptibility would report the veridical sequence. Additionally, we found that pre-stimulus alpha phase in occipital and frontal areas dissociated illusory from non-illusory trials. These results point to a dissociated relation of the power and timing of alpha band activity to illusory perception, with power reflecting prior expectations and phase influencing behavioral performance, potentially due to the modulation of sensory uncertainty. Significance StatementLate events may sometimes influence how earlier events are perceived, as if the arrow of time was reversed in the brain. This surprising phenomenon, called postdiction, is observed in the rabbit illusion, and highlights a predominant mechanism for perceptual processes. Perception builds up from the combination of prior expectations with incoming sensory evidence, which takes time. We showed that pre-stimulus neural activity, and more specifically alpha oscillations (8-12Hz), play a double role in postdiction. Fronto-parietal alpha power reflects individual prior expectation, while occipital and frontal alpha phase predicts illusory perception. Postdiction might actually be a means of compensating for the neural delays inherent in perceptual processes, so that the arrow of perceptual time matches the arrow of physical time.

neuroscience↗

bioRxiv2021

Robust neurophysiological correlates of the rubber hand illusion emerge at early latencies but are unrelated to changes in skin conductance accompanying the illusion

The neurophysiological processes reflecting body illusions such as the rubber hand remain debated. In particular, previous studies investigating neural responses evoked by the illusion-inducing stimuli provide diverging reports as to when these responses reflect the illusory state. To resolve these controversies in previous work we applied multivariate (cross-) classification to EEG responses obtained during the rubber hand illusion and multiple control conditions in human participants. These controls were designed to test for markers of the illusory state that generalize across the spatial arrangements of limbs or the specific nature of the control condition (rubber hand or participants real hand) - hence which are independent of the precise experimental conditions used as contrast for the illusion. This revealed a parcellation of evoked responses into a temporal sequence of events that each differentiate the illusion and control conditions along distinct dimensions. Importantly, around 130-150 ms following stimulus onset the neurophysiological signals reliably differentiated the illusory-state from all non-illusion epochs. This neurophysiological signature was not correlated with changes in skin conductance accompanying the illusion, suggesting that neurophysiological and bodily signals reflect distinct illusion-related processes.

neuroscience↗

bioRxiv2020

Robust spatial ventriloquism effect and aftereffect under memory interference

Our brain adapts to discrepancies in the sensory inputs. One example is provided by the ventriloquism effect, experienced when the sight and sound of an object are displaced. Here the discrepant multisensory stimuli not only result in a biased localization of the sound, but also recalibrate the perception of subsequent unisensory acoustic information in the so-called ventriloquism aftereffect. This aftereffect has been linked to memory-related processes based on its parallels to general sequential effects in perceptual decision making experiments and insights obtained in neuroimaging studies. For example, we have recently implied memory-related medial parietal regions in the trial-by-trial ventriloquism aftereffect. Here, we tested the hypothesis that the ventriloquism aftereffect is indeed susceptible to manipulations interfering with working memory. Across three experiments we systematically manipulated the temporal delays between stimuli and response for either the ventriloquism or the aftereffect trials, or added a sensory-motor masking trial in between. Our data reveal no significant impact of either of these manipulations on the aftereffect, suggesting that the recalibration reflected by the ventriloquism aftereffect is surprisingly resilient to manipulations interfering with memory-related processes.

neuroscience↗

bioRxiv2020

Delta/theta band EEG differentially tracks low and high frequency speech envelopes

The representation of speech in the brain is often examined by measuring the alignment of rhythmic brain activity to the speech envelope. To conveniently quantify this alignment (termed speech tracking) many studies consider the overall speech envelope, which combines acoustic fluctuations across the spectral range. Using EEG recordings, we show that using this overall envelope can provide a distorted picture on speech encoding. We systematically investigated the encoding of spectrally-limited speech-derived envelopes presented by individual and multiple noise carriers in the human brain. Tracking in the 1 to 6 Hz EEG bands differentially reflected low (0.2 - 0.83 kHz) and high (2.66 - 8 kHz) frequency speech-derived envelopes. This was independent of the specific carrier frequency but sensitive to attentional manipulations, and reflects the context-dependent emphasis of information from distinct spectral ranges of the speech envelope in low frequency brain activity. As low and high frequency speech envelopes relate to distinct phonemic features, our results suggest that functionally distinct processes contribute to speech tracking in the same EEG bands, and are easily confounded when considering the overall speech envelope. HighlightsO_LIDelta/theta band EEG tracks band-limited speech-derived envelopes similar to real speech C_LIO_LILow and high frequency speech-derived envelopes are represented differentially C_LIO_LIHigh-frequency derived envelopes are more susceptible to attentional and contextual manipulations C_LIO_LIDelta band tracking shifts towards low frequency derived envelopes with more acoustic detail C_LI

neuroscience↗

bioRxiv2020

Delta/Theta band EEG activity shapes the rhythmic perceptual sampling of auditory scenes

Many studies speak in favor of a rhythmic mode of listening, by which the encoding of acoustic information is structured by rhythmic neural processes at the time scale of about 1 to 4 Hz. Indeed, psychophysical data suggest that humans sample acoustic information in extended soundscapes not uniformly, but weigh the evidence at different moments for their perceptual decision at the time scale of about 2 Hz. We here test the critical prediction that such rhythmic perceptual sampling is directly related to the state of ongoing brain activity prior to the stimulus. Human participants judged the direction of frequency sweeps in 1.2 s long soundscapes while their EEG was recorded. Computing the perceptual weights attributed to different epochs within these soundscapes contingent on the phase or power of pre-stimulus oscillatory EEG activity revealed a direct link between the 4Hz EEG phase and power prior to the stimulus and the phase of the rhythmic component of these perceptual weights. Hence, the temporal pattern by which the acoustic information is sampled over time for behavior is directly related to pre-stimulus brain activity in the delta/theta band. These results close a gap in the mechanistic picture linking ongoing delta band activity with their role in shaping the segmentation and perceptual influence of subsequent acoustic information.

neuroscience↗

bioRxiv2020

The neurophysiological basis of short- and long-term ventriloquism aftereffects

Our senses often receive conflicting multisensory information, which our brain reconciles by adaptive recalibration. A classic example is the ventriloquist aftereffect, which emerges following both long-term and trial-wise exposure to spatially discrepant multisensory stimuli. Still, it remains debated whether the behavioral biases observed following short- and long-term exposure arise from largely the same or rather distinct neural origins, and hence reflect the same or distinct mechanisms. We address this question by probing EEG recordings for physiological processes predictive of the single-trial ventriloquism biases following the exposure to spatially offset audio-visual stimuli. Our results support the hypothesis that both short- and long-term aftereffects are mediated by common neurophysiological correlates, which likely arise from sensory and parietal regions involved in multisensory inference and memory, while prolonged exposure to consistent discrepancies additionally recruits prefrontal regions. These results posit a central role of parietal regions in mediating multisensory spatial recalibration and suggest that frontal regions contribute to increasing the behavioral bias when the perceived sensory discrepancy is consistent and persistent over time.

neuroscience↗

bioRxiv2020

Diversification of perceptual mechanisms underlying preserved multisensory behaviour in healthy aging

The manner in which humans exploit multisensory information for subsequent decisions changes with age. Multiple causes for such age-effects are being discussed, including a reduced precision in peripheral sensory representations, changes in cognitive inference about causal relations between sensory cues, and a decline in memory contributing to altered sequential patterns of multisensory behaviour. To dissociate these putative contributions, we investigated how healthy young and older adults integrate audio-visual spatial information within trials (the ventriloquism effect) and between trials (the ventriloquism aftereffect). With both a model-free and (Bayesian) model-based analyses we found that both biases differed between groups. Our results attribute the age-change in the ventriloquism bias to a decline in spatial hearing rather than a change in cognitive processes. This decline in peripheral function, combined with a more prominent influence from preceding responses rather than preceding stimuli in the elderly, can also explain the observed age-effect in the ventriloquism aftereffect. Our results suggest a transition from a sensory- to a behavior-driven influence of past multisensory experience on perceptual decisions with age, due to reduced sensory precision and change in memory capacity.

neuroscience↗

bioRxiv2019

Alpha Activity Reflects the Effort to Compensate an Individual Bias in Human Perception

Biases in sensory perception can arise from both experimental manipulations and personal trait-like features. These idiosyncratic biases and their neural underpinnings are often overlooked in studies on the physiology underlying perception. A potential candidate mechanism reflecting such idiosyncratic biases could be spontaneous alpha band activity, a prominent brain rhythm known to influence perceptual reports in general. Using a temporal order judgement task, we here tested the hypothesis that alpha power reflects the overcoming of an idiosyncratic bias. Importantly, to understand the interplay between idiosyncratic biases and contextual (temporary) biases induced by experimental manipulations, we quantified this relation before and after temporal recalibration. Using EEG recordings in human participants (male and female), we find that pre-stimulus frontal alpha power correlates with the tendency to respond relative to an own idiosyncratic bias, with stronger alpha leading to responses matching the bias. In contrast, alpha power does not predict response correctness. These results also hold after temporal recalibration and are specific to the alpha band, suggesting that alpha band activity reflects, directly or indirectly, processes that help to overcome an individuals momentary bias in perception. We propose that combined with established roles of parietal alpha in the encoding of sensory information frontal alpha reflects complementary mechanisms influencing perceptual decisions. Significance statementThe brain is a biased organ, frequently generating systematically distorted percepts of the world, leading each of us to evolve in our own subjective reality. However, such biases are often overlooked or considered noise when studying the neural mechanisms underlying perception. We show that spontaneous alpha band activity predicts the degree of biasedness of human choices in a time perception task, suggesting that alpha activity indexes processes needed to overcome an individuals idiosyncratic bias. This result provides a window onto the neural underpinnings of subjective perception, and offers the possibility to quantify or manipulate such priors in future studies.

neuroscience↗

bioRxiv2019

Largely distinct networks mediate perceptually-relevant auditory and visual speech representations

Visual speech is an integral part of communication, but it remains unclear whether information carried by lip movements is represented in the same brain regions that mediate acoustic speech comprehension. Our ability to understand acoustic speech seems independent from that to understand visual speech, yet neuroimaging studies suggest that the neural representations largely overlap. Addressing this discrepancy, we tested where the brain represents acoustically and visually conveyed word identities in a full-brain MEG study. Our analyses dissociate cerebral representations that merely reflect the physical stimulus from those that also predict comprehension, and suggest that these overlap only in specific temporal and frontal regions. Moreover, representations predictive of auditory and visual comprehension converge only in angular and inferior frontal regions. These results provide a neural explanation for the behavioural dissociation of acoustic and visual speech comprehension and suggest that cerebral representations encoding word identities may be more modality-specific than often upheld.

neuroscience↗

bioRxiv2019

Evidence for the rhythmic perceptual sampling of auditory scenes

Converging results suggest that perception is controlled by rhythmic processes in the brain. In the auditory domain, neuroimaging studies show that the perception of brief sounds is shaped by rhythmic activity prior to the stimulus and electrophysiological recordings have linked delta band (1-2 Hz) activity to the functioning of individual neurons. These results have promoted theories of rhythmic modes of listening and generally suggest that the perceptually relevant encoding of acoustic information is structured by rhythmic processes along auditory pathways. A prediction from this perspective - which so far has not been tested - is that such rhythmic processes also shape how acoustic information is combined over time to judge extended soundscapes. The present study was designed to directly test this prediction. Human participants judged the overall change in perceived frequency content in temporally extended (1.2 to 1.8 s) soundscapes, while the perceptual use of the available sensory evidence was quantified using psychophysical reverse correlation. Model-based analysis of individual participants perceptual weights revealed a rich temporal structure, including linear trends, a U-shaped profile tied to the overall stimulus duration, and importantly, rhythmic components at the time scale of 1 to 2Hz. The collective evidence found here across four versions of the experiment supports the notion that rhythmic processes operating on the delta band time scale structure how perception samples temporally extended acoustic scenes.

neuroscience↗

bioRxiv2019

Shared neural underpinnings of multisensory integration and trial-by-trial perceptual recalibration

Multisensory stimuli create behavioral flexibility, e.g. by allowing us to derive a weighted combination of the information received by different senses. They also allow perception to adapt to discrepancies in the sensory world, e.g. by biasing the judgement of unisensory cues based on preceding multisensory evidence. While both facets of multisensory perception are central for behavior, it remains unknown whether they arise from a common neural substrate. In fact, very little is known about the neural mechanisms underlying multisensory perceptual recalibration. To reveal these, we measured whole-brain activity using MEG while human participants performed an audio-visual ventriloquist paradigm designed to reveal multisensory integration within a trial, and the (trial-by-trial) recalibration of subsequent unisensory judgements. Using single trial classification and behavioral modelling, we localized the encoding of sensory information within and between trials, and determined the behavioral relevance of candidate neural representations. While we found neural signatures of perceptual integration within temporal and parietal regions, of these, only medial superior parietal activity retained multisensory information between trials and combined this with current evidence to mediate perceptual recalibration. These results suggest a common neural substrate of sensory integration and trial-by-trial perceptual recalibration, and expose the medial superior parietal cortex as a flexible hub that links present and previous evidence within and between senses to guide behavior.

neuroscience↗

bioRxiv2018

Causal inference in the multisensory brain

When combining information across different senses humans need to flexibly select cues of a common origin whilst avoiding distraction from irrelevant inputs. The brain could solve this challenge using a hierarchical principle, by deriving rapidly a fused sensory estimate for computational expediency and, later and if required, filtering out irrelevant signals based on the inferred sensory cause(s). Analysing time- and source-resolved human magnetoencephalographic data we unveil a systematic spatio-temporal cascade of the relevant computations, starting with early segregated unisensory representations, continuing with sensory fusion in parietal-temporal regions and culminating as causal inference in the frontal lobe. Our results reconcile previous computational accounts of multisensory perception by showing that prefrontal cortex guides flexible integrative behaviour based on candidate representations established in sensory and association cortices, thereby framing multisensory integration in the generalised context of adaptive behaviour.

neuroscience↗