Biology subjects

Feather, J.

Publications and source records attributed to Feather, J..

6 recordsLinked to original sources

bioRxiv2026

Neural prediction decorrelation reveals that adversarial robustness substantially improves DNN prediction accuracy across the entire human auditory cortex

Sensory neuroscientists seek to model the neural computations that encode complex stimuli. Distinct encoding models often make similar predictions for natural stimuli such as speech, posing a challenge for model comparison. We developed a method to synthesize stimuli that decorrelate model predictions across a neural population, termed neural prediction decorrelation (NPD). Using fMRI responses to NPD sounds, we compared standard and adversarially robust deep neural network models of human auditory cortex. Prediction accuracy for NPD sounds was substantially better for the adversarially robust model in every region tested, an effect completely masked with natural sounds. Population responses to natural and synthesized NPD sounds shared an interpretable low-dimensional organization that was reproduced by the robust encoding model. NPD provides a general approach for comparing encoding models and reveals that adversarial robustness expands the predictive power of DNNs beyond natural stimuli, which is likely critical for targeting population activity through stimulus synthesis.

neuroscience↗

bioRxiv2026

Investigating Invariances in Auditory Event Categorization with Model Metamers

Real-world acoustic inputs contain rich sensory information that we parse into discrete auditory objects and categories. Although deep neural networks (DNNs) are increasingly used to model auditory perception, the field lacks rigorous behavioral benchmarks, particularly for auditory event categorization. Here, we developed a 25-way categorization paradigm for broad classes of natural sounds to test whether categorical invariances of DNNs align with those of human observers. We first confirmed that humans could reliably categorize the natural sounds, demonstrating that our paradigm is well-suited for testing invariances in auditory categories. To probe model invariances, we evaluated human recognition of model metamers (synthetic stimuli matched to the models internal activations for each natural stimulus) for a wide range of architectures trained on speech or auditory event recognition. Evaluating widely-used public models, we found that human recognition of auditory event model metamers was generally influenced by the training task and data distribution; speech models trained on standard, curated datasets produced less recognizable metamers than auditory event recognition models. We additionally analyzed a controlled set of models to directly investigate the influence of training task and adversarial training, revealing that improved metamer recognition induced by adversarial training is task-dependent. However, even in the best-performing models, we observed a sharp decline in human recognition at the final classification layer compared to the penultimate representation layer. Overall, our results suggest that while invariances in modern architectures better align with human observers for auditory event categorization, there is still a large discrepancy between the categorical invariances of auditory neural networks and the invariances of human observers. Code and models are available at https://github.com/Feather-Lab/env-sound-metamers

neuroscience↗

bioRxiv2026

Model-optimized stimulus distortions for adaptive estimation of individual sensory representations

Representations of the same physical stimulus vary between individuals. Characterizing individual differences has practical implications, but is challenging because these representations are not directly observable. Given a model of how representations vary within a population, we propose a Bayesian adaptive procedure for estimating an individual observers representation from a series of targeted perceptual discrimination judgments. A key component of our approach is using Fisher information to identify stimulus distortions that efficiently differentiate observers in the population. As a proof of concept, we focus on individual differences in color perception and simulate observers with cone fundamentals drawn from an individual colorimetric observer model. We demonstrate that our approach can recover key aspects of a sampled observers cone fundamentals using simulated three-alternative forced-choice oddity judgments with approximately 500 trials, corresponding to an experimental duration of approximately one hour. Our Bayesian adaptive framework provides a promising and generalizable approach to efficiently link behavioral measurements to individual differences in sensory representations.

neuroscience↗

bioRxiv2022

Many but not all deep neural network audio models capture brain responses and exhibit hierarchical region correspondence

Models that predict brain responses to stimuli provide one measure of understanding of a sensory system, and have many potential applications in science and engineering. Deep artificial neural networks have emerged as the leading such predictive models of the visual system, but are less explored in audition. Prior work provided examples of audio-trained neural networks that produced good predictions of auditory cortical fMRI responses and exhibited correspondence between model stages and brain regions, but left it unclear whether these results generalize to other neural network models, and thus how to further improve models in this domain. We evaluated model-brain correspondence for publicly available audio neural network models along with in-house models trained on four different tasks. Most tested models out-predicted previous filter-bank models of auditory cortex, and exhibited systematic model-brain correspondence: middle stages best predicted primary auditory cortex while deep stages best predicted non-primary cortex. However, some state-of-the-art models produced substantially worse brain predictions. Models trained to recognize speech in background noise produced better brain predictions than models trained to recognize speech in quiet, potentially because hearing in noise imposes constraints on biological auditory representations. The training task influenced the prediction quality for specific cortical tuning properties, with best overall predictions resulting from models trained on multiple tasks. The results generally support the promise of deep neural networks as models of audition, though they also indicate that current models do not explain auditory cortical responses in their entirety.

neuroscience↗

bioRxiv2022

Model metamers illuminate divergences between biological and artificial neural networks

Deep neural network models of sensory systems are often proposed to learn representational transformations with invariances like those in the brain. To reveal these invariances we generated "model metamers" - stimuli whose activations within a model stage are matched to those of a natural stimulus. Metamers for state-of-the-art supervised and unsupervised neural network models of vision and audition were often completely unrecognizable to humans when generated from deep model stages, suggesting differences between model and human invariances. Targeted model changes improved human-recognizability of model metamers, but did not eliminate the overall human-model discrepancy. The human-recognizability of a models metamers was well predicted by their recognizability by other models, suggesting that models learn idiosyncratic invariances in addition to those required by the task. Metamer recognition dissociated from both traditional brain-based benchmarks and adversarial vulnerability, revealing a distinct failure mode of existing sensory models and providing a complementary benchmark for model assessment.

neuroscience↗

bioRxiv2019

Intracranial recordings from human auditory cortex reveal a neural population selective for musical song

How are neural representations of music organized in the human brain? While neuroimaging has suggested some segregation between responses to music and other sounds, it remains unclear whether finer-grained organization exists within the domain of music. To address this question, we measured cortical responses to natural sounds using intracranial recordings from human patients and inferred canonical response components using a data-driven decomposition algorithm. The inferred components replicated many prior findings including distinct neural selectivity for speech and music. Our key novel finding is that one component responded nearly exclusively to music with singing. Song selectivity was not explainable by standard acoustic features and was co-located with speech- and music-selective responses in the middle and anterior superior temporal gyrus. These results suggest that neural representations of music are fractionated into subpopulations selective for different types of music, at least one of which is specialized for the analysis of song.

neuroscience↗