Biology subjects

Roelfsema, P. R.

Publications and source records attributed to Roelfsema, P. R..

17 recordsLinked to original sources

bioRxiv2025

End-to-end Learning of Safe Stimulation Parameters for Cortical Neuroprosthetic Vision

Direct electrical stimulation of the brain via cortical visual neuroprostheses is a promising approach to restore basic sight for the visually impaired by inducing a percept of localized light called phosphenes. Apart from the challenge of condensing complex sensory information into meaningful stimulation patterns at low temporal and spatial resolution, providing safe stimulation levels to the brain is crucial. We propose an end-to-end framework to learn optimal stimulation parameters (amplitude, pulse width and frequency) within safe biological constraints. The learned stimulation parameters are passed to a biologically plausible phosphene simulator which takes into account the size, brightness, and temporal dynamics of perceived phosphenes. Our experiments on naturalistic navigation videos demonstrate that constraining stimulation parameters to safe levels not only maintains task performance in image reconstruction from phosphenes but consistently results in more meaningful phosphene vision, while providing insights into the optimal range of stimulation parameters. Our study presents a stimulus-generating encoder that learns stimulation parameters (1) satisfying safety constraints, and (2) maximizing the combined objective of image reconstruction and phosphene interpretability with a highly realistic phosphene simulator accounting for temporal dynamics of stimulation. End-to-end learning of stimulation parameters this way enables enforcement of critical biological safety constraints as well as technical limits of the hardware at hand.

bioengineering↗

bioRxiv2024

Deep generative networks reveal the tuning of neurons in IT and predict their influence on visual perception

Finding the tuning of visual neurons has kept neuroscientists busy for decades. One approach to this problem has been to test specific hypotheses on the relevance of a visual property (e.g. orientation or color), build a set of "artificial" stimuli that vary along that property and then record neural responses to those stimuli. Here, we present a complementary, data-driven method to retrieve the tuning properties of visual neurons. Exploiting deep generative networks and electrophysiology in monkeys, we first used a method to reconstruct any stimulus from its evoked neuronal activity in the inferotemporal cortex (IT). Then, by arbitrarily perturbing the response of individual cortical sites in the model, we generated naturalistic and interpretable sequences of images that strongly influence neural activity of that site. This method enables the discovery of previously unknown tuning properties of high-level visual neurons that are easily interpretable, which we tested with carefully controlled stimuli. When we knew which images drove the neurons, we activated the cells with electrical microstimulation and observed a predicable shift of the monkey perception in the direction of the preferred image. By allowing the brain to tell us what it cares about, we are no longer limited by our experimental imagination.

neuroscience↗

bioRxiv2024

Pronouns reactivate conceptual representations in human hippocampal neurons

During discourse comprehension, every new word adds to an evolving representation of meaning that accumulates over consecutive sentences and constrains the next words. To minimize repetition and utterance length, languages use pronouns, like the word she, to refer to nouns and phrases that were previously introduced. It has been suggested that language comprehension requires that pronouns activate the same neuronal representations as the nouns themselves. Here, we test this hypothesis by recording from individual neurons in the human hippocampus during a reading task. We found that cells that are selective to a particular noun are later reactivated by pronouns that refer to the cells preferred noun. These results imply that concept cells contribute to a rapid and dynamic semantic memory network which is recruited during language comprehension. This study uniquely demonstrates, at the single-cell level, how memory and language are linked.

neuroscience↗

bioRxiv2024

Experience-dependent predictions of feedforward and contextual information in mouse visual cortex

To navigate complex environments, the visual system must prioritize unexpected inputs or omissions over predictable sensory backgrounds. This saliency assignment is a hallmark of visual processing and depends on comparing visual feedforward inputs with learned contextual surrounds, a process often conceptualized as a difference computation. However, cortical circuitry poses fundamental challenges: visual contextual feedback is predominantly amplifying, and feedback-driven inhibitory neurons necessary for subtractive computations are relatively sparse. How then can complex contextual feedback be selectively compared with feedforward input at the level of individual neurons? Here, we show that divergent plasticity rules within single pyramidal neurons provide a solution to this problem. Using two-photon calcium imaging, we recorded layer 2/3 pyramidal cell responses in mouse primary visual cortex before and after visual familiarization. We dissociated feedforward and contextual signals by comparing responses to stimuli with and without an occluded sector. Repeated feedforward responses selectively weakened through adaptation and surround suppression, whereas contextual responses strengthened and generalized across scenes. These strengthened contextual inputs amplified non-adapted responses to novel images and drove neuronal activity when occlusion of feedforward input prevented surround suppression. This produced a conserved structure of feedforward and contextual responses across mice, monkeys, and humans. Ultimately, these results reveal a cellular mechanism for saliency assignment that does not require feedback-driven inhibition. Pyramidal neurons autonomously learn the generalized statistical structure of familiar contexts, allowing them to selectively amplify improbable inputs or omissions.

neuroscience↗

bioRxiv2024

Optimal placement of high-channel visual prostheses in human retinotopic visual cortex

AO_SCPLOWBSTRACTC_SCPLOWO_ST_ABSObjectiveC_ST_ABSRecent strides in neurotechnology show potential to restore vision in individuals afflicted with blindness due to early visual pathway damage. As neuroprostheses mature and become available to a larger population, manual placement and evaluation of electrode designs becomes costly and impractical. An automatic method to optimize the implantation process of electrode arrays at large-scale is currently lacking. ApproachHere, we present a comprehensive method to automatically optimize electrode placement for visual prostheses, with the objective of matching pre-defined phosphene distributions. Our approach makes use of retinotopic predictions combined with individual anatomy data to minimize discrepancies between simulated and target phosphene patterns. While demonstrated with a 1000-channel 3D electrode array in V1, our pipeline is versatile, potentially accommodating any electrode design and allowing for design evaluation. Main resultsNotably, our results show that individually optimized placements in 362 brain hemi-spheres outperform average brain solutions, underscoring the significance of anatomical specificity. We further show how virtual implantation of multiple individual brains highlights the challenges of achieving full visual field coverage owing to single electrode constraints, which may be overcome by introducing multiple arrays of electrodes. Including additional surgical considerations, such as intracranial vasculature, in future iterations could refine the optimization process. SignificanceOur open-source software streamlines the refinement of surgical procedures and facilitates simulation studies, offering a realistic exploration of electrode configuration possibilities.

neuroscience↗

bioRxiv2023

Biologically plausible gated recurrent neural networks for working memory and learning-to-learn

The acquisition of knowledge does not occur in isolation; rather, learning experiences in the same or similar domains amalgamate. This process through which learning can accelerate over time is referred to as learning-to-learn or meta-learning. While meta-learning can be implemented in recurrent neural networks, these networks tend to be trained with architectures that are not easily interpretable or mappable to the brain and with learning rules that are biologically implausible. Specifically, these rules employ backpropagation-through-time for learning, which relies on information that is unavailable at synapses that are undergoing plasticity in the brain. While memory models that exclusively use local information for their weight updates have been developed, they have limited capacity to integrate information over long timespans and therefore cannot easily learn-to-learn. Here, we propose a novel gated recurrent network named RECOLLECT, which can flexibly retain or forget information by means of a single memory gate and biologically plausible trial-and-error-learning that requires only local information. We demonstrate that RECOLLECT successfully learns to represent task-relevant information over increasingly long memory delays in a pro-/anti-saccade task, and that it learns to flush its memory at the end of a trial. Moreover, we show that RECOLLECT can learn-to-learn an effective policy on a reversal bandit task. Finally, we show that the solutions acquired by RECOLLECT resemble how animals learn similar tasks.

neuroscience↗

bioRxiv2023

Visuomotor experience induces functional and structural plasticity of chandelier cells

Detailed characterization of interneuron types in primary visual cortex (V1) has greatly contributed to understanding visual perception, yet the role of chandelier cells (ChCs) in visual processing remains poorly characterized. Using viral tracing we found that V1 ChCs predominantly receive monosynaptic input from local layer 5 pyramidal cells and higher-order cortical regions. Two-photon calcium imaging and convolutional neural network modelling revealed that ChCs are visually responsive but weakly selective for stimulus content. In mice running in a virtual tunnel, ChCs respond strongly to events known to elicit arousal, including locomotion and visuomotor mismatch. Repeated exposure of the mice to the virtual tunnel was accompanied by reduced visual responses of ChCs and structural plasticity of ChC boutons and axon initial segment length. Finally, ChCs only weakly inhibited pyramidal cells. These findings suggest that ChCs provide an arousal-related signal to layer 2/3 pyramidal cells that may modulate their activity and/or gate plasticity of their axon initial segments during behaviorally relevant events.

neuroscience↗

bioRxiv2023

Modulatory feedback explain object segmentation by attention

Studies in neuroscience inspired progress in the design of artificial neural networks (ANNs), and, vice versa, ANNs provide new insights into the functioning of brain circuits. So far, the focus has been on how ANNs can help to explain the tuning of neurons at various stages of the visual cortical hierarchy. However, the role of modulatory feedback connections, which play a role in attention and perceptual organization, has not been resolved yet. The present study presents a biologically plausible neural network that performs scene segmentation and can shift attention using modulatory feedback connections from higher to lower brain areas. The model replicates several neurophysiological signatures of recurrent processing. Specifically, figural regions elicit more activity in model units than background regions. The modulation of activity by figure and ground occurs at a delay after the first feedforward response, because it depends on a loop through the higher model areas. Importantly, the figural response enhancement is enhanced by object-based attention, which stays focused on the figural regions and does not spill over to the adjacent background, just as is observed in the visual cortex. Our results indicate how progress in artificial intelligence can be used to garner insight into the recurrent cortical processing for scene segmentation and object-based attention. Author SummaryRecent feedforward networks in artificial intelligence provide unmatched models of tuning of neurons in the visual cortex. However, these feedforward models do not explain the influences of object-based attention and image segmentation on neuronal responses, which rely on feedback interactions between cortical regions that are not included in the feedforward networks. In particular, the role of feedback connections from higher brain regions that modulate neural activity in lower cortical regions has not yet been studied extensively so that we still lack an in silico model of the role of these connections. Here, we present a biologically plausible neural network that successfully performs image segmentation and can shift object-based attention using modulatory feedback connections. The model evolved representations that mirror the properties of neurons in the visual cortex, including orientation tuning, shape-selectivity, surround suppression and a sensitivity to figure-ground organization, while trained only on a segmentation task. The new model provides insight into how the perception of coherent objects can emerge from the interaction between lower and higher visual cortical areas.

neuroscience↗

bioRxiv2022

Feedback brings scene information to the representation of occluded image regions in area V1 of monkeys and humans

Neuronal activity in the primary visual cortex (V1) is driven by feedforward input from within the neurons receptive fields (RFs) and modulated by contextual information in regions surrounding the RF. The effect of contextual information on spiking activity occurs rapidly, and is therefore challenging to dissociate from feedforward input. To address this challenge, we recorded the spiking activity of V1 neurons in monkeys viewing either natural scenes or scenes where the information in the RF was occluded, effectively removing the feedforward input. We found that V1 neurons responded rapidly and selectively to occluded scenes. V1 responses elicited by occluded stimuli could be used to decode scene identity and could be predicted from those elicited by non-occluded images, indicating that there is overlap between visually-driven and contextual responses. We used representational similarity analysis to show that the structure of V1 representations of occluded scenes measured with electrophysiology in monkeys correlates strongly with the representations of the same scenes in humans measured with fMRI. Our results reveal that contextual influences alter the spiking activity of V1 in monkeys across large distances on a rapid time scale, carry information about scene identity and resemble those in human V1.

neuroscience↗

bioRxiv2022

Involvement of superior colliculus in complex figure detection of mice

Object detection is an essential function of the visual system. Although the visual cortex plays an important role in object detection, the superior colliculus can support detection when the visual cortex is ablated or silenced. Moreover, it has been shown that superficial layers of mouse SC (sSC) encode visual features of complex objects, and that this code is not inherited from the primary visual cortex. This suggests that mouse sSC may provide a significant contribution to complex object vision. Here, we use optogenetics to show that mouse sSC is causally involved in figure detection based on differences in figure contrast, orientation and phase. Additionally, our neural recordings show that in mouse sSC, image elements that belong to a figure elicit stronger activity than those same elements when they are part of the background. The discriminability of this neural code is higher for correct trials than incorrect trials. Our results provide new insight into the behavioral relevance of the visual processing that takes place in sSC.

neuroscience↗

bioRxiv2022

The direct and indirect pathways of the basal ganglia antagonistically influence cortical activity and perceptual decisions

The striatum, input nucleus of the basal ganglia, receives topographically organized input from the cortex and gives rise to the direct and indirect pathways with antagonistic effects on the output of the basal ganglia. We optogenetically stimulated the direct and indirect pathways in mice and measured their influence on perceptual decisions and neuronal activity in the cortex. In a task in which mice had to detect a visual stimulus, unilateral direct-pathway stimulation increased the probability of lick responses to the non-stimulated side and increased cortical activity globally. In contrast, indirect-pathway stimulation increased the probability of licks to the stimulated side and decreased activity in visual cortical areas. To probe the possible role of the two pathways in working memory, we trained the mice to report the location of a stimulus with licking one of two spouts, after a memory delay. Direct-pathway stimulation prior to and during the memory delay enhanced both the neural representation of a contralateral visual stimulus and the number of contraversive choices, whereas indirect-pathway stimulation had the opposite effects, in accordance with an antagonistic influence of the direct and indirect pathways on licking direction. Our results demonstrate how these two pathways influence perceptual decisions and working memories, and modify activity in the cerebral cortex. One sentence summaryVisuomotor transformations are influenced antagonistically by the direct and indirect pathways of the basal ganglia during visual detection and working memory tasks

neuroscience↗

bioRxiv2022

Pop-in: the inversion of pop-out for a feature dimension during visual search in area V4 of the monkey cortex

During visual search, it is important to reduce the interference of distracting objects in the scene. The neuronal responses elicited by the search target stimulus are typically enhanced. However, it is equally important to suppress the representations of distracting stimuli, especially if they are salient and capture attention. We trained monkeys to make an eye movement to a unique pop-out shape stimulus among an array of distracting stimuli. One of these distractors had a salient color that varied across trials and differed from the color of the other stimuli, causing it to also pop-out. The monkeys were able to select the pop-out shape target with high accuracy and actively avoided the pop-out color distractor. This behavioral pattern was reflected in the activity of neurons in area V4. Responses to the shape targets were enhanced, while the activity evoked by the pop-out color distractor was only briefly enhanced, directly followed by a sustained period of pronounced suppression. These behavioral and neuronal results demonstrate a cortical selection mechanism that rapidly inverts a pop-out signal to pop-in for an entire feature dimension thereby facilitating goal-directed visual search in the presence of salient distractors. Significance statementGoal-directed behaviors like visual search involve both the selection of behaviorally relevant targets and the suppression of task-irrelevant distractors. This is especially important if distractors are salient and capture attention. Here we demonstrate that non-human primates suppress a salient color distractor while searching for a target that is defined by shape, i.e. another feature dimension. The neuronal activity of V4 neurons revealed the temporal evolution of target selection and distractor suppression. The neuronal responses elicited by the pop-out target stimuli were enhanced whereas responses elicited by salient pop-out color distractors were suppressed, after an initial brief phase of response enhancement. Our results reveal a pop-in mechanism by which the visual cortex inverts an attentional capture signal into suppression to facilitate visual search.

neuroscience↗

bioRxiv2021

The influence of objecthood on the representation of natural images in the visual cortex

Neurons in early visual cortex are not only sensitive to the image elements in their receptive field but also to the context determining whether the elements are part of an object or background. We here assessed the effect of objecthood in natural images on neuronal activity in early visual cortex, with fMRI in humans and electrophysiology in monkeys. We report that boundaries and interiors of objects elicit more activity than the background. Boundary effects occur remarkably early, implying that visual cortical neurons are tuned to features characterizing object boundaries in natural images. When a new image is presented the influence of the object interiors on neuronal activity occurs during a late phase of neuronal response and earlier when eye movements shift the image representation, implying that object representations are remapped across eye-movements. Our results reveal how object perception shapes the representation of natural images in early visual cortex.

neuroscience↗

bioRxiv2019

Flexible working memory through selective gating and attentional tagging

Working memory is essential for intelligent behavior as it serves to guide behavior of humans and nonhuman primates when task-relevant stimuli are no longer present to the senses. Moreover, complex tasks often require that multiple working memory representations can be flexibly and independently maintained, prioritized, and updated according to changing task demands. Thus far, neural network models of working memory have been unable to offer an integrative account of how such control mechanisms are implemented in the brain and how they can be acquired in a biologically plausible manner. Here, we present WorkMATe, a neural network architecture that models cognitive control over working memory content and learns the appropriate control operations needed to solve complex working memory tasks. Key components of the model include a gated memory circuit that is controlled by internal actions, encoding sensory information through untrained connections, and a neural circuit that matches sensory inputs to memory content. The network is trained by means of a biologically plausible reinforcement learning rule that relies on attentional feedback and reward prediction errors to guide synaptic updates. We demonstrate that the model successfully acquires policies to solve classical working memory tasks, such as delayed match-to-sample and delayed pro-saccade/antisaccade tasks. In addition, the model solves much more complex tasks including the hierarchical 12-AX task or the ABAB ordered recognition task, which both demand an agent to independently store and updated multiple items separately in memory. Furthermore, the control strategies that the model acquires for these tasks subsequently generalize to new task contexts with novel stimuli. As such, WorkMATe provides a new solution for the neural implementation of flexible memory control. Author SummaryWorking Memory, the ability to briefly store sensory information and use it to guide behavior, is a cornerstone of intelligent behavior. Existing neural network models of Working Memory typically focus on how information is stored and maintained in the brain, but do not address how memory content is controlled: how the brain can selectively store only stimuli that are relevant for a task, or how different stimuli can be maintained in parallel, and subsequently replaced or updated independently according to task demands. The models that do implement control mechanisms are typically not trained in a biologically plausible manner, and do not explain how the brain learns such control. Here, we present WorkMATe, a neural network architecture that implements flexible cognitive control and learns to apply these control mechanisms using a biologically plausible reinforcement learning method. We demonstrate that the model acquires control policies to solve a range of both simple and more complex tasks. Moreover, the acquired control policies generalize to new situations, as with human cognition. This way, WorkMATe provides new insights into the neural organization of Working Memory beyond mere storage and retrieval.

neuroscience↗

bioRxiv2019

Theta-phase dependent neuronal coding during sequence learning in human single neurons.

The ability to maintain a sequence of items in short-term memory (STM) is a fundamental cognitive function. In the rodent hippocampus, the representation of sequentially organized spatial locations is reflected by the phase of action potentials relative to the theta oscillation (phase precession). We investigated whether the timing of neuronal activity relative to the theta brain oscillation also reflects sequence order in the medial temporal lobe of humans. We used a task in which human subjects learned sequences of pictures and recorded single neuron and local field potential activity with implanted electrodes. We report that spikes for three consecutive items in the sequence (the preferred stimulus for each cell, as well as the stimuli immediately preceding and following it) were phase-locked at distinct phases of the theta oscillation. Consistent with phase precession, spikes were fired at progressively earlier phases as the sequence advanced. These findings generalize previous findings in the rodent hippocampus to the human temporal lobe and suggest that encoding stimulus information at distinct oscillatory phases may play a role in maintaining their sequential order in STM.

neuroscience↗

bioRxiv2018

Feedforward and feedback processing during figure-ground perception in mice

The segregation of figures from the background is an important step in visual perception. In primary visual cortex, figures evoke stronger activity than backgrounds during a delayed phase of the neuronal responses, but it is unknown how this figure-ground modulation (FGM) arises and whether it is necessary for perception. Here we show, using optogenetic silencing in mice, that the delayed V1 response phase is necessary for figure-ground segregation. Neurons in higher visual areas also exhibit FGM and optogenetic silencing of higher areas reduced FGM in V1. In V1, figures elicited higher activity of vasoactive intestinal peptide-expressing (VIP) interneurons than the background, whereas figures suppressed somatostatin-positive interneurons, resulting in an increased activation of pyramidal cells. Optogenetic silencing of VIP neurons reduced FGM in V1, indicating that disinhibitory circuits contribute to FGM. Our results provide new insight in how lower and higher areas of the visual cortex interact to shape visual perception.

neuroscience↗

bioRxiv2017

Electrochemical Measurement Of Acetylcholine In The Dorsolateral Prefrontal Cortex: A Technical Report

Ever since the discovery of acetylcholine in 1913, its role as neuromodulator has been extensively studied in a variety of model systems. These previous studies revealed that acetylcholine is of critical importance for several cognitive functions including attention, learning and memory. In spite of these previous findings, it has proven difficult to determine the amount of acetylcholine that is released during cognitive tasks with sub-second temporal resolution. One method that might be used to measure acetylcholine release is the use of an enzyme-coupled amperometric sensor, which has been suggested to measure acetylcholine with high sensitivity, selectivity and relatively high temporal resolution (< 1 second). In the present study, we have tried to adapt the technique developed in the rodent model system by Parikh and colleagues1,2 for use in non-human primates. We aimed to measure in-vivo levels of acetylcholine in the macaque dorsolateral prefrontal cortex while the monkey performed an attention demanding curve-tracing task3,4. We report that our attempts to measure acetylcholine using amperometry in an awake behaving macaque monkey proved difficult and tedious and that our results are inconsistent and prone to noise. In the discussion, we will outline the challenges that will need to be addressed to use this technique in non-human primates and hope that our observations inspire solutions to help future research on the role of this important neurotransmitter.

neuroscience↗