Biology subjects

Cappelloni, M. S.

Publications and source records attributed to Cappelloni, M. S..

6 recordsLinked to original sources

bioRxiv2022

Music and Speech Elicit Similar Subcortical Responses in Human Listeners

Music and speech are two sounds that are unique to human beings and encountered in daily life. Both are transformed by the auditory pathway from an initial acoustical encoding to higher level cognition. Most studies of speech and music processing are focused on the cortex, and the subcortical response to natural, polyphonic music is essentially unstudied. This study was aimed to compare the subcortical encoding of music and speech using the auditory brainstem response (ABR). While several methods have recently been developed to derive the ABR to continuous speech, they are either not applicable to music or give poor results. In this study, we explored deriving the ABR through deconvolution using three regressors: 1) the half-wave rectified stimulus waveform, 2) the modeled inner hair cell potential, and 3) the auditory nerve model firing rate (ANM), where the latter two were generated from a computational auditory periphery model. We found the ANM regressor yields robust and interpretable ABR waveforms to diverse genres of music and multiple types of speech. We then used the ANM-derived ABRs to compare the subcortical responses to music and speech and found that they are highly similar in morphology. We further investigated cortical responses using the same deconvolution method, and found the responses there were also quite similar, which was unexpected based on previous studies. We conclude that when using our proposed deconvolution regressor that accounts for acoustical differences nonlinear effects on peripheral encoding, the derived brainstem and cortical responses to music and speech are highly correlated.

neuroscience↗

bioRxiv2022

Humans rely more on talker identity than temporal coherence in an audiovisual selective attention task using speech-like stimuli

Audiovisual integration of speech can benefit the listener by not only improving comprehension of what a talker is saying but also helping a listener pick a particular talkers voice out of a mix of sounds. Binding, an early integration of auditory and visual streams that helps an observer allocate attention to a combined audiovisual object, is likely involved in audiovisual speech processing. Although temporal coherence of stimulus features across sensory modalities has been implicated as an important cue for non-speech stimuli (Maddox et al., 2015), the specific cues that drive binding in speech are not fully understood due to the challenges of studying binding in natural stimuli. Here we used speech-like artificial stimuli that allowed us to isolate three potential contributors to binding: temporal coherence (are the face and the voice changing synchronously?), articulatory correspondence (do visual faces represent the correct phones?), and talker congruence (do the face and voice come from the same person?). In a trio of experiments, we examined the relative contributions of each of these cues. Normal hearing listeners performed a dual detection task in which they were instructed to respond to events in a target auditory stream and a visual stream while ignoring events in a distractor auditory stream. We found that viewing the face of a talker who matched the attended voice (i.e., talker congruence) offered a performance benefit. Importantly, we found no effect of temporal coherence on performance in this task, a result that prompts an important recontextualization of previous findings.

neuroscience↗

bioRxiv2022

Inferring sources of suboptimality in perceptual decision making using a causal inference task

Perceptual decision-making has been extensively modeled using the ideal observer framework. However, a range of deviations from optimality demand an extension of this framework to characterize the different sources of suboptimality. Prior work has mostly formalized these sources by adding biases and variability in the context of specific process models but are hard to generalize to more complex tasks. Here, we formalize suboptimalities as part of the brains probabilistic model of the task. Data from a traditional binary discrimination task cannot separate between different kinds of biases, or between sensory noise and approximate computations. We showed that this was possible using a recently developed causal inference task in which observers discriminated auditory cues in the presence of choice-uninformative visual cues. An extension of the task with different stimulus durations provided evidence for an increase in the precision of the computations with stimulus duration, separate from a decrease in observation noise.

animal behavior and cognition↗

bioRxiv2020

Independent mechanisms of temporal and linguistic cue correspondence benefiting audiovisual speech processing

When listening is difficult, seeing the face of the talker aids speech comprehension. Faces carry both temporal (low-level physical correspondence of mouth movement and auditory speech) and linguistic (learned physical correspondences of mouth shape (viseme) and speech sound (phoneme)) cues. Listeners participated in two experiments investigating how these cues may be used to process sentences when maskers are present. In Experiment I, faces were rotated to disrupt linguistic but not temporal cue correspondence. Listeners suffered a deficit in speech comprehension when the faces were rotated, indicating that visemes are processed in a rotation-dependent manner, and that linguistic cues aid comprehension. In Experiment II, listeners were asked to detect pitch modulation in the target speech with upright and inverted faces that either matched the target or masker speech such that performance differences could be explained by binding, an early multisensory integration mechanism distinct from traditional late integration. Performance in this task replicated previous findings that temporal integration induces binding, but there was no behavioral evidence for a role of linguistic cues in binding. Together these experiments point to temporal cues providing a speech processing benefit through binding and linguistic cues providing a benefit through late integration.

animal behavior and cognition↗

bioRxiv2020

Effects of auditory reliability and ambiguous visual stimuli on auditory spatial discrimination

ABSTRACTThe brain combines information from multiple sensory modalities to interpret the environment. Multisensory integration is often modeled by ideal Bayesian causal inference, a model proposing that perceptual decisions arise from a statistical weighting of information from each sensory modality based on its reliability and relevance to the observer’s task. However, ideal Bayesian causal inference fails to describe human behavior in a simultaneous auditory spatial discrimination task in which spatially aligned visual stimuli improve performance despite providing no information about the correct response. This work tests the hypothesis that humans weight auditory and visual information in this task based on their relative reliabilities, even though the visual stimuli are task-uninformative, carrying no information about the correct response, and should be given zero weight. Listeners perform an auditory spatial discrimination task with relative reliabilities modulated by the stimulus durations. By comparing conditions in which task-uninformative visual stimuli are spatially aligned with auditory stimuli or centrally located (control condition), listeners are shown to have a larger multisensory effect when their auditory thresholds are worse. Even in cases in which visual stimuli are not task-informative, the brain combines sensory information that is scene-relevant, especially when the task is difficult due to unreliable auditory information.Competing Interest StatementThe authors have declared no competing interest.View Full Text

animal behavior and cognition↗

bioRxiv2019

Improvement in auditory spatial discrimination from ambiguous visual stimuli is not explained by ideal observer causal inference

In order to survive and function in the world, we must understand the content of our environment. This requires us to gather and parse complex, sometimes conflicting, information. Yet, the brain is capable of translating sensory stimuli from disparate modalities into a cohesive and accurate percept with little conscious effort. Previous studies of multisensory integration have suggested that the brains integration of cues is well-approximated by an ideal observer implementing Bayesian causal inference. However, behavioral data from tasks that include only one stimulus in each modality fail to capture what is in nature a complex process. Here we employed an auditory spatial discrimination task in which listeners were asked to determine on which side they heard one of two concurrently presented sounds. We compared two visual conditions in which task-uninformative shapes were presented in the center of the screen, or spatially aligned with the auditory stimuli. We found that performance on the auditory task improved when the visual stimuli were spatially aligned with the auditory stimuli--even though the shapes provided no information about which side the auditory target was on. We also demonstrate that a model of a Bayesian ideal observer performing causal inference cannot explain this improvement, demonstrating that humans deviate systematically from the ideal observer model.

animal behavior and cognition↗