Biology subjects

Riesenhuber, M.

Publications and source records attributed to Riesenhuber, M..

8 recordsLinked to original sources

bioRxiv2025

Evidence for hierarchical representations of written and spoken words from an open-science human neuroimaging dataset

Reading and speech recognition rely on multi-level processing that builds from basic visual or sound features to complete word representations, yet details of these processing hierarchies (in particular those for spoken words) are still poorly understood. We re-analyzed the functional magnetic resonance imaging (fMRI) data provided in the Mother Of all Unification Studies (MOUS) open-science dataset by using parametric regressions of word frequency and sublexical unit (bigram or syllable) frequency during reading and speech listening tasks in order to elucidate lexical processing hierarchies in the visual and auditory modalities. We first validated our approach in the written word domain, where the technique identified significant correlations for word frequency in the left mid-fusiform cortex (at the location of the Visual Word Form Area) with a left occipital region tracking bigram frequency, compatible with prior reports. During listening, low-frequency spoken words elicited greater responses in a left mid-superior temporal region consistent with the recently-described Auditory Word Form Area (AWFA), while a more posterior region of the superior temporal gyrus was sensitive to syllable frequency. Activation in the left inferior frontal gyrus correlated with both written and spoken word frequency. These findings demonstrate parallel hierarchical organizations in the anteroventral visual and auditory streams, with modality-specific lexica and upstream sublexical representations that converge in higher-order language areas.

neuroscience↗

bioRxiv2025

Tool-use brain representations are independent of the acting body part and motor experience

The sensorimotor system is broadly organized somatotopically. However, an action-type organization has also been found: a division based on action-type independent of acting body parts has been shown for reaching and grasping actions. Does this generalization extend to non-ethological actions? Here, we examined fMRI responses for tool-use actions that participants performed with their hands or feet. We additionally tested individuals born without hands to control for hand motor imagery when performing foot actions. We show that the primary sensorimotor cortices have hand and foot selectivity, consistent with a somatotopic organization. In contrast, higher-level motor areas within the tool-use network, such as the premotor cortex, supplementary motor area, and superior parietal cortices, showed a shared preference for tool-use independent of the executing body part and sensorimotor experience. Multivariate decoding of action-type in these areas generalized between controls hand and foot and was successful in individuals born without hands. Finally, the temporal dynamics pattern in primary and association areas carried effector-specific and action-type information, respectively. Altogether, we show that the tool-use network in motor association areas represents higher-order action information beyond concrete motor parameters associated with specific effectors, and regardless of hand motor experience. This suggests that an action-type, effector-independent organization extends beyond ethological actions, supporting a hierarchical organization in the action domain. Further, it shows that functional organization in congenital handlessness is based on the hierarchical organization of the intact cortex.

neuroscience↗

bioRxiv2022

Evidence for a Spoken Word Lexicon in the Auditory Ventral Stream

The existence of a neural representation for whole words (i.e., a lexicon) is a common feature of many models of speech processing. Prior studies have provided evidence for a visual lexicon containing representations of whole written words in an area of the ventral visual stream known as the "Visual Word Form Area" (VWFA). Similar experimental support for an auditory lexicon containing representations of spoken words has yet to be shown. Using fMRI rapid adaptation techniques, we provide evidence for an auditory lexicon in the "Auditory Word Form Area" (AWFA) in the human left anterior superior temporal gyrus that contains representations highly selective for individual spoken words. Furthermore, we show that familiarization with novel auditory words sharpens the selectivity of their representations in the AWFA. These findings reveal strong parallels in how the brain represents written and spoken words, showing convergent processing strategies across modalities in the visual and auditory ventral streams. HighlightsO_LIIndividual auditory word form areas (AWFA) were defined via an auditory localizer C_LIO_LIThe AWFA shows tuning for individual real words but not untrained pseudowords C_LIO_LIThe AWFA develops tuning for individual pseudowords after training C_LI

neuroscience↗

bioRxiv2021

Neural basis of learning to perceive speech through touch using an acoustic-to-vibrotactile speech sensory substitution

The goal of sensory substitution is to convey the information transduced by one sensory system through a novel sensory modality. One example is vibrotactile (VT) speech, for which acoustic speech is transformed into vibrotactile patterns. Despite an almost century-long history of studying vibrotactile speech, there has been no study of the neural bases of VT speech learning. We here trained hearing adult participants to recognize VT speech syllables. Using fMRI, we showed that both somatosensory (left post-central gyrus) and auditory (right temporal lobe) regions acquire selectivity for VT speech stimuli following training. The right planum temporale in particular was selective for both VT and auditory speech. EEG source-estimated activity revealed temporal dynamics consistent with direct, low-latency engagement of right temporal lobe following activation of the left post-central gyrus. Our results suggest that VT speech learning achieves integration with the auditory speech system by "piggybacking" onto corresponding auditory speech representations. Significance statementIn sensory substitution, the information conveyed by one sensory system is used to replace the function of another. Blind individuals, for example, can learn to use a visual-to-acoustic sensory substitution device to navigate the world. We tested the hypothesis that sensory substitution can more generally occur in typical individuals, by exploiting the existence of multi-sensory convergence areas in the brain. We trained hearing participants to recognize speech syllables presented as vibrotactile stimulation patterns. Using fMRI and EEG, we show that vibrotactile speech learning integrates with the auditory speech system by "piggybacking" onto corresponding auditory speech representations. These results suggest that the human brain can bootstrap new senses by leveraging one sensory system to process information normally processed by another.

neuroscience↗

bioRxiv2021

Metamodal Coupling of Vibrotactile and Auditory Speech Processing Systems Through Matched Stimulus Representations

It has been postulated that the brain is organized by "metamodal", sensory-independent cortical modules implementing particular computations, leading to the intriguing hypothesis that brain areas can perform tasks (such as word recognition) not just in "standard" sensory modalities but also in novel sensory modalities. Yet, evidence for this theory, especially in neurotypical subjects, has been variable. We hypothesized that effective metamodal engagement of a brain area requires congruence between the novel and standard sensory modalities not only at the task level (e.g., "word recognition") but critically also a match at the algorithmic level (in Marrs terminology), i.e., at the level of neural representation of the information of interest. To test this hypothesis, we trained participants to recognize vibrotactile versions of auditory words using two encoding schemes. The vocoded approach preserved the dynamics and representational similarities of auditory speech while the token-based approach used an abstract phoneme-based code. Although both groups learned the vibrotactile word recognition task, only in the vocoded group did trained vibrotactile stimuli recruit the auditory speech network and lead to increased coupling between somatosensory and auditory speech areas. In contrast, the token-based encoding appeared to rely on paired-associate learning. Thus, matching neural input representations is a critical factor for assessing and leveraging the metamodal potential of cortical modules.

neuroscience↗

bioRxiv2020

Microsaccades during high speed continuous visual search

Here, we provide an analysis of the microsaccades that occurred during continuous visual search and targeting of small faces that we pasted either into cluttered background photos or into a simple gray background. Subjects continuously used their eyes to target singular 3-degree upright or inverted faces in changing scenes. As soon as the participants gaze reached the target face, a new face was displayed in a different and random location. Regardless of the experimental context (e.g. background scene, no background scene), or target eccentricity (from 4 to 20 degrees of visual angle), we found that the microsaccade rate dropped to near zero levels within only 12 milliseconds after trial onset. There were almost never any microsaccades before the saccade to the face. One subject completed 118 consecutive trials without a single microsaccade. However, in about 20% of the trials, there was a single corrective microsaccade that occurred almost immediately after the preceding saccades offset. These corrective microsaccades were task oriented because their facial landmark targeting distributions matched those of saccades within both the upright and inverted face conditions. Our findings show that a single feedforward pass through the visual hierarchy for each stimulus is likely all that is needed to effectuate prolonged continuous visual search. In addition, we provide evidence that microsaccades can serve perceptual functions like correcting saccades or effectuating task-oriented goals during continuous visual search.

neuroscience↗

bioRxiv2020

Evidence for face selectivity in early vision

The commonly accepted "simple-to-complex" model of visual processing in the brain posits that visual tasks on complex objects such as faces are based on representations in high-level visual areas. Yet, recent experimental data showing the visual systems ability to localize faces in natural images within 100ms (Crouzet et al., 2010) challenge the prevalent hierarchical description of the visual system, and instead suggest the hypothesis of face-selectivity in early visual areas. In the present study, we tested this hypothesis with human participants in two eye tracking experiments, an fMRI experiment and an EEG experiment. We found converging evidence for neural representations selective for upright faces in V1/V2, with latencies starting around 40 ms post-stimulus onset. Our findings suggest a revision of the standard "simple-to-complex" model of hierarchical visual processing. Significance statementVisual processing in the brain is classically described as a series of stages with increasingly complex object representations: early visual areas encode simple visual features (such as oriented bars), and high-level visual areas encode representations for complex objects (such as faces). In the present study, we provide behavioral, fMRI, and EEG evidence for representations of complex objects - namely faces - in early visual areas. Our results challenge the standard "simple-to-complex" model of visual processing, suggesting that it needs to be revised to include neural representations for faces at the lowest levels of the visual hierarchy. Such early object representations would permit the rapid and precise localization of complex objects, as has previously been reported for the object class of faces.

neuroscience↗

bioRxiv2020

Leveraging prior concept learning improves ability to generalize from few examples in computational models of human object recognition

Humans quickly learn new visual concepts from sparse data, sometimes just a single example. Decades of prior work have established the hierarchical organization of the ventral visual stream as key to this ability. Computational work has shown that networks which hierarchically pool afferents across scales and positions can achieve human-like object recognition performance and predict human neural activity. Prior computational work has also reused previously acquired features to efficiently learn novel recognition tasks. These approaches, however, require magnitudes of order more examples than human learners and only reuse intermediate features at the object level or below. None has attempted to reuse extremely high-level visual features capturing entire visual concepts. We used a benchmark deep learning model of object recognition to show that leveraging prior learning at the concept level leads to vastly improved abilities to learn from few examples. These results suggest computational techniques for learning even more efficiently as well as neuroscientific experiments to better understand how the brain learns from sparse data. Most importantly, however, the model architecture provides a biologically plausible way to learn new visual concepts from a small number of examples, and makes several novel predictions regarding the neural bases of concept representations in the brain. Author summaryWe are motivated by the observation that people regularly learn new visual concepts from as little as one or two examples, far better than, e.g., current machine vision architectures. To understand the human visual systems superior visual concept learning abilities, we used an approach inspired by computational models of object recognition which: 1) use deep neural networks to achieve human-like performance and predict human brain activity; and 2) reuse previous learning to efficiently master new visual concepts. These models, however, require many times more examples than human learners and, critically, reuse only low-level and intermediate information. None has attempted to reuse extremely high-level visual features (i.e., entire visual concepts). We used a neural network model of object recognition to show that reusing concept-level features leads to vastly improved abilities to learn from few examples. Our findings suggest techniques for future software models that could learn even more efficiently, as well as neuroscience experiments to better understand how people learn so quickly. Most importantly, however, our model provides a biologically plausible way to learn new visual concepts from a small number of examples.

neuroscience↗