Biology subjects

Serences, J.

Publications and source records attributed to Serences, J..

12 recordsLinked to original sources

bioRxiv2026

Changes in perceptual sampling contribute to representational drift

Gradual changes in neural response patterns to the same stimulus over time, termed representational drift, have been widely observed across cortical areas. Drift is typically attributed to intrinsic neural dynamics driven by synaptic plasticity or turnover. Here we test a complementary hypothesis that drift arises due to systematic changes in behavior such as shifts in attention or gaze. We conducted a longitudinal eye-tracking experiment in which fourteen adults freely viewed naturalistic images across 6 experimental sessions spanning 2-4 weeks, with a subset of images repeated within and across sessions. For each participant, we quantified the similarity of fixation density maps between session pairs using the Wasserstein distance. Fixation patterns became increasingly dissimilar with greater temporal separation between sessions, indicating systematic and directional drift in gaze behavior. To assess whether these behavioral changes could plausibly induce changes in neural representations, we passed fixation-masked images through CORnet-S, a hierarchical deep neural network model of the primate ventral visual stream. Representational distances, quantified as the Frobenius norm of pairwise activation differences, increased with the number of sessions that separated the fixation maps across all four model layers (V1, V2, V4, IT). A kernel-based maximum mean discrepancy test further confirmed that the empirical distribution of representational distances differed significantly from shuffled controls. These findings suggest that small but systematic shifts in the sampling of a visual scene over time are sufficient to cause representational drift in visual cortex. More generally, these results suggest that subtle changes in behavior over time are inevitable, even in simple tasks, and that these changes may be sufficient to drive representational drift in the absence of intrinsic reconfigurations of neural codes. Author summaryIn studies of representational drift in visual cortex, shifts in attention and gaze across repeated stimulus exposures can produce drift-like neural patterns. While the current focus is on contributions of changes in visual sampling to drift, the results raise the possibility that other subtle changes in behavior, many that are hard to quantify, could drive drift in other domains.

neuroscience↗

bioRxiv2025

Memory recall errors reflect interacting sensory and mnemonic representations

Visual working memory (WM) enables the maintenance of information that is no longer present in the environment. Some accounts propose that WM is supported by abstract representations so that new sensory inputs do not interfere with existing memories. Others posit that early sensory representations are recruited to maintain memory precision, potentially at the cost of interference caused by new inputs. Here we tested these accounts using an orientation recall task to determine whether memory errors reflect interacting representations of sensory and mnemonic information. We found that adding noise to the memoranda and presenting feature-neutral distractors independently increased recall errors. However, distractors that shared a feature with remembered stimuli led to systematic attractive biases such that memory errors were pulled toward the orientation of the distractor. The magnitude of this bias was modulated by both stimulus noise and whether the distractor was behaviorally relevant. Our results demonstrate that while working memory can utilize abstract representations, it remains susceptible to feature-specific sensory interference, suggesting partial reliance on sensory-like codes.

neuroscience↗

bioRxiv2024

Manipulating attentional priority creates a trade-off between memory and sensory representations in human visual cortex

People often remember visual information over brief delays while actively engaging with ongoing inputs from the surrounding visual environment. Depending on the situation, one might prioritize mnemonic contents (i.e., remembering details of a past event), or preferentially attend sensory inputs (i.e., minding traffic while crossing a street). Previous fMRI work has shown that early visual areas can simultaneously represent both mnemonic and passively viewed visual information. Here we test the limits of such simultaneity by manipulating attention towards visual distractors during a working memory task performed by human subjects undergoing fMRI scanning. Participants remembered the orientation of a target grating while a distractor grating was shown throughout most of the memory delay. Critically, there were several subtle changes in the contrast and the orientation of the distractor, and participants were cued to either ignore the distractor, detect a change in contrast, or detect a change in orientation. Despite visual stimulation being matched in all three conditions, the fidelity of memory representations in early visual cortex was highest when the distractor was ignored, intermediate when participants attended distractor contrast, and lowest when participants attended the orientation of the distractor during the delay. In contrast, the fidelity of distractor representations was lowest when ignoring the distractor, intermediate when attending distractor-contrast, and highest when attending distractor-orientation. These data suggest a trade-off in early sensory representations when engaging top-down feedback to attend both seen and remembered features, and may partially explain memory failures that occur when subjects are distracted by external events.

neuroscience↗

bioRxiv2020

Steady-state visually evoked potentials and feature-based attention: Pre-registered null results and a focused review of methodological considerations.

Feature-based attention is the ability to selectively attend to a particular feature (e.g., attend to red but not green items while looking for the ketchup bottle in your refrigerator), and steady-state visually evoked potentials (SSVEPs) measured from the human electroencephalogram (EEG) signal have been used to track the neural deployment of feature-based attention. Although many published studies suggest that we can use trial-by-trial cues to enhance relevant feature information (i.e., greater SSVEP response to the cued color), there is ongoing debate about whether participants may likewise use trial-by-trial cues to voluntarily ignore a particular feature. Here, we report the results of a pre-registered study in which participants either were cued to attend or to ignore a color. Counter to prior work, we found no attention-related modulation of the SSVEP response in either cue condition. However, positive control analyses revealed that participants paid some degree of attention to the cued color (i.e., we observed a greater P300 component to targets in the attended versus the unattended color). In light of these unexpected null results, we conducted a focused review of methodological considerations for studies of feature-based attention using SSVEPs. In the review, we quantify potentially important stimulus parameters that have been used in the past (e.g., stimulation frequency; trial counts) and we discuss the potential importance of these and other task factors (e.g., feature-based priming) for SSVEP studies.

neuroscience↗

bioRxiv2020

Biased orientation representations can be explained by experience with non-uniform training set statistics

Visual acuity is better for vertical and horizontal compared to other orientations. This cross-species phenomenon is often explained by "efficient coding", whereby more neurons show sharper tuning for the orientations most common in natural vision. However, it is unclear if experience alone can account for such biases. Here, we measured orientation representations in a convolutional neural network, VGG-16, trained on modified versions of ImageNet (rotated by 0{degrees}, 22.5{degrees}, or 45{degrees} counter-clockwise of upright). Discriminability for each model was highest near the orientations that were most common in the networks training set. Furthermore, there was an over-representation of narrowly tuned units selective for the most common orientations. These effects emerged in middle layers and increased with depth in the network. Biases emerged early in training, consistent with the possibility that non-uniform representations may play a functional role in the networks task performance. Together, our results suggest that biased orientation representations can emerge through experience with a non-uniform distribution of orientations, supporting the efficient coding hypothesis.

neuroscience↗

bioRxiv2019

Inverted encoding models estimate sensible channel responses for sensible models

In a commentary published in eNeuro, Gardner & Liu (2019) discuss the role of model specification in interpreting the output of complex models of neural data. As a case study, they suggest that one variant of such analyses, the inverted encoding model (IEM) analysis framework, should not be used to assay properties of \"stimulus representations\" because the ability to apply linear transformations at various stages of the analysis procedure renders results arbitrary. As we discuss, the specification of all models is arbitrary to the extent that an experimenter makes choices based on current knowledge of the model system. However, the results derived from any given model, such as the reconstructed channel response profiles obtained from an IEM analysis, are uniquely defined and are arbitrary only in the sense that changes in the model can predictably change results. Moreover, with knowledge of the model used for IEM analyses, the results remain informative as comparisons between reconstructed channel response profiles across task conditions using a fixed encoding model - the most common use of the IEM technique - can generally capture changes in population-level representation magnitude across linear transformations. Thus, changes in the magnitude of the response profiles across conditions are preserved, even across unprincipled linear transforms. IEM-based channel response profiles should therefore not be considered arbitrary when the model is clearly specified and guided by our best understanding of neural population representations in the brain regions being analyzed. Intuitions derived from this case study are important to consider when interpreting results from all model-based analyses, which are similarly contingent upon the specification of the models used.

neuroscience↗

bioRxiv2019

Value-driven attentional capture enhances distractor representations in early visual cortex

When a behaviorally relevant stimulus has been previously associated with reward, behavioral responses are faster and more accurate compared to equally relevant but less valuable stimuli. Conversely, task irrelevant stimuli that were previously associated with a high reward can capture attention and distract processing away from relevant stimuli (e.g. the chocolate bar in the pantry when you are looking for a nice healthy apple). While increasing the value of task-relevant stimuli systematically up-regulates neural responses in early visual cortex to facilitate information processing, it is not clear if the value of task-irrelevant distractors influences behavior via competition in early visual cortex or via competition at later stages of decision-making and response selection. Here, we measured fMRI in human visual cortex while subjects performed a value-based learning task, and applied a multivariate inverted encoding model to assess the fidelity of distractor representations in early visual cortex. We found that the fidelity of neural representations related to task-irrelevant distractors increased when the distractors were previously associated with a high reward. Moreover, this value-based modulation of distractor representations only occurred when the distractors were previously selected as targets on preceding trials. Together, these findings suggest that value-driven attentional capture begins with sensory modulations of distractor representations in early areas of visual cortex.

neuroscience↗

bioRxiv2018

The speed-accuracy tradeoff reveals flexible access to accumulating sensory evidence during human decision making

Decisions made about identical perceptual stimuli can be radically different under changing task demands. For example, the need to make a fast decision undermines the accuracy of that decision, a well-documented effect termed the speed-accuracy tradeoff (SAT). Models of the SAT are generally based on theories of decision making in which responses are triggered only after sensory evidence accumulation terminates at a set threshold. Within this accumulate-to-bound framework, speed pressure operates by lowering the response threshold, allowing for faster responses at the expense of accumulated sensory evidence. To empirically examine the mechanisms necessary for adaptively adjusting the speed and accuracy of decisions, we used an event-related potential that indexes sensory evidence accumulation in the human brain. Instead of lowering response thresholds, we found that speed pressure adaptively shifts responses to occur close to where the rate of evidence accumulation peaks. Moreover, responses are not triggered automatically by the termination of the accumulation process, as sensory evidence continues to build after speeded decisions. Together these results suggest that response processes adaptively access accumulating sensory evidence depending on task demands and support parallel over serial models of decision making.

neuroscience↗

bioRxiv2018

Reconciling fMRI and EEG indices of attentional modulations in human visual cortex

Functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) are the two most popular non-invasive methods used to study the neural mechanisms underlying human cognition. These approaches are considered complementary: fMRI has higher spatial resolution but sluggish temporal resolution, whereas EEG has millisecond temporal resolution, but only at a broad spatial scale. Beyond the obvious fact that fMRI measures properties of blood and EEG measures changes in electric fields, many foundational studies assume that, aside from differences in spatial and temporal precision, these two methods index the same underlying neural modulations. We tested this assumption by using EEG and fMRI to measure attentional modulations of neural responses to stimuli of different visual contrasts. We found that equivalent experiments performed using fMRI and EEG on the same participants revealed remarkably different patterns of attentional modulations: event-related fMRI responses provided evidence for an additive increase in responses across all contrasts equally, whereas early stimulus-evoked event-related potentials (ERPs) showed larger modulations with increasing stimulus contrast and only a later negative-going ERP and low-frequency oscillatory EEG signals showed effects similar to fMRI. These results demonstrate that there is not a one-to-one correspondence between the physiological mechanisms that give rise to modulations of fMRI responses and the most commonly used ERP markers, and that the typical approach of employing fMRI and EEG to gain complementary information about localization and temporal dynamics is over-simplified. Instead, fMRI and EEG index different physiological modulations and their joint application affords synergistic insights into the neural mechanisms supporting human cognition.

neuroscience↗

bioRxiv2018

Human frontoparietal cortex represents behaviorally-relevant target status during invariant object recognition

Searching for items that are useful given current goals, or \"target\" recognition, requires an observer to generalize across identity-preserving transformations such as viewpoint changes, as well as to incorporate contextual information. While past work has found target recognition signals in areas of ventral visual cortex, it is not clear whether these signals support performance on demanding tasks that require invariant, flexible search. Here, we used a task that required subjects to match novel object stimuli based on invariant features (identity and viewpoint). Based on multivariate fMRI analyses, the data suggest that the multiple-demand (MD) network, including sub-regions of parietal and frontal cortex, encodes invariant representations of an objects status as a target. Furthermore, target information in MD regions, but not early or ventral visual cortex, was higher on correct compared to incorrect trials, suggesting a strong link between MD target signals and behavior.

neuroscience↗

bioRxiv2017

When conflict cannot be avoided: executive control dominates early selective sensory modulations during cognitive conflict

When different sources of sensory information suggest competing behavioral responses, the efficiency of decision-making is impaired. Prior work suggests that at least two mechanisms may play a role in mitigating this interference: using early selective attention to extract the most relevant sensory inputs to avoid conflict or increasing the efficiency of the executive control network to resolve conflict during post-perceptual processing. To test these alternatives, we combined a stimulus-frequency tagging technique with a classic color-word Stroop paradigm, where color-bar targets and letter-string distractors were simultaneously flickered at different frequencies. Using electroencephalography (EEG), we measured the quality of early sensory processing by assessing the amplitude of steady-state visually evoked potentials (SSVEPs) elicited by the targets and distractors. We also measured the engagement of the executive control network by assessing changes in frontal theta (4-7Hz) and posterior alpha oscillations (8-14Hz). Counter to the early selective sensory modulation account, the amplitude of the SSVEP response was not modulated by manipulations of color/word congruency, while the frontal theta activity increased and the posterior alpha activity decreased in response to conflict. Moreover, target-related SSVEP amplitude was not correlated with response times (RTs) and a higher (not lower) distractor-related SSVEP amplitude predicted faster RTs. On the other hand, the amplitude of the frontal theta and alpha activity was highly correlated with RTs, irrespective of conflict levels. Over all, these results highlight the dominant role of the executive control network in conflict resolution during post-perceptual processing.\n\nSignificance StatementConflicting information interferes with decision-making. However, this interference can be mitigated either by extracting the most relevant inputs during early sensory processing or by increasing the efficiency of the executive control processes to resolve conflict. By measuring electroencephalography (EEG) in humans performing a modified color-word Stroop task, we examined early sensory responses evoked by targets and distractors while simultaneously monitoring frontal theta and posterior alpha oscillations to index the activation of the executive control network. We found evidence that the executive control network played a more prominent role in resolving conflict.

neuroscience↗

bioRxiv2016

Spatial tuning shifts increase the discriminability and fidelity of population codes in visual cortex

AcknowledgementsMany thanks to the lab and particularly to Rosanne Rademaker and Edward Vul for comments on analyses and on the manuscript. This work was supported by National Science Foundation Graduate Research Fellowships to V.A.V. and T.C.S., a grant from the National Eye Institute (R01-EY025872) and a Scholar Award from the James S. McDonnell Foundation to J.T.S.\n\nABSTRACTSelective visual attention enables organisms to enhance the representation of behaviorally relevant stimuli by altering the encoding properties of single receptive fields (RFs). Yet we know little about how the attentional modulations of single RFs contribute to the encoding of an entire visual scene. Addressing this issue requires (1) measuring a group of RFs that tile a continuous portion of visual space, (2) constructing a population-level measurement of spatial representations based on these RFs, and (3) linking how different types of RF attentional modulations change the population-level representation. To accomplish these aims, we used fMRI to characterize the responses of thousands of voxels in retinotopically organized human cortex. First, we found that the response modulations of voxel RFs (vRFs) depend on the spatial relationship between the RF center and the visual location of the attended target. Second, we used two analyses to assess the spatial encoding quality of a population of voxels. We found that attention increased fine spatial discriminability and representational fidelity near the attended target. Third, we linked these findings by manipulating the observed vRF attentional modulations and recomputing our population measures. Surprisingly, we discovered that attentional enhancements of population-level representations largely depend on position shifts of vRFs, rather than changes in size or gain. Our data suggest that position shifts of single RFs are a principal mechanism by which attention enhances population-level representations in visual cortex.

neuroscience↗