Biology subjects

Roelfsema, P.

Publications and source records attributed to Roelfsema, P..

9 recordsLinked to original sources

bioRxiv2026

Friend, not Foe: Lowered Tissue Reactivity to Long-Term Polyimide Implants

One of the biggest challenges for neurotechnology is the design of devices that are tolerated well by brain tissue, without sacrificing functionality and implantability. This study examined which design choices mitigate tissue damage and improve longevity, by varying probe features implanted in the cerebral cortex of mice. We report on a systematic, quantitative analysis of neuronal and inflammation markers across cortical depth. We implanted a total of 103 stiff silicon or flexible polyimide probes in 32 mice, varying their thicknesses and widths, which were either attached to the skull or not. A new, automated workflow to quantify immunohistochemical data examines: 1) the tissue loss caused by the implant, 2) the cortical neuronal density, and 3) the immune response expressed by astrocytic and microglial reaction. Flexible polyimide probes exhibited a clear advantage, with fewer lesions and weaker immune responses than stiff silicon probes. Furthermore, we observed a weaker influence of the shank cross-section. A cortical depth profile of immune reactivity revealed focal reactions at the device entry points in the superficial cortex and at the cortex-white matter boundary. This study gives important insights on optimizing device design parameters as well as surgical insights for improved tissue integration of intracortical electrode arrays.

bioengineering↗

bioRxiv2025

A convergent cortical scheme for visual scene segmentation

Scene segmentation, a key step towards structured understanding of natural environments, entails a complementary set of computations. Instead of accessing these segmentation computations in isolation, we perceive the segregated object as a unified whole, but the neural basis supporting such a unitary percept remains unclear. We bridged this gap by showing a convergent coding scheme: an object-level additive code in the primary visual cortex, which serves as a spatial segmentation mask for the object. Across individual neurons, this code uniformly boosted their responses to the segmented object. In the population space, this code created an orthogonal representational geometry by perpendicularly displacing the neural manifold for image features, efficiently multiplexing both local and global information about the object. Computational modeling further revealed the utility and optimality of this convergent coding scheme, which effectively isolates objects in cluttered environments and in turn unclutters downstream object representations. Our findings reveal the computational logic of visual segmentation and call for a revision of the visual cortical hierarchy.

neuroscience↗

bioRxiv2024

Encoding of cerebellar dentate neuronal activity during visual attention in rhesus macaques

The role of cerebellum in controlling eye movements is well established, but its contribution to more complex forms of visual behavior has remained elusive. To study cerebellar activity during visual attention we recorded extracellular activity of dentate nucleus (DN) neurons in two non-human primates (NHPs). NHPs were trained to read the direction indicated by a peripheral visual stimulus while maintaining fixation at the center, and report the direction of the cue by performing a saccadic eye movement into the same direction following a delay. We found that single unit DN neurons modulated spiking activity over the entire time-course of the task, and that their activity often bridged temporally separated intra-trial events, yet in a heterogeneous manner. To better understand the heterogeneous relationship between task structure, behavioral performance and neural dynamics, we constructed a behavioral, an encoding and a decoding model. Both NHPs showed different behavioral strategies, which influenced the performance. Activity of the DN neurons reflected the unique strategies, with the direction of the visual stimulus frequently being encoded long before an upcoming saccade. Moreover, the latency of the ramping activity of DN neurons following presentation of the visual stimulus was shorter in the better performing NHP. Labeling with the retrograde tracer CTB in the recording location in the DN indicated that these neurons predominantly receive inputs from Purkinje cells in the D1 and D2 zones of the lateral cerebellum as well as neurons of the principal olive and medial pons, all regions known to connect with neurons in the prefrontal cortex contributing to planning of saccades. Together, our results highlight that DN neurons can dynamically modulate their activity during a visual attention task, comprising not only sensorimotor but also cognitive attentional components.

neuroscience↗

bioRxiv2024

How the brain learns to parse images using an attentional, incremental grouping process

Natural scenes usually contain many objects that need to be segregated from each other and the background. Object-based attention is the process that groups image fragments belonging to the same objects. Curve-tracing tasks provide a special case, testing our ability to group image elements of an elongated curve. In the brain, curve-tracing is associated with the gradual spread of enhanced neuronal activity over the representation of the traced curve. Previous studies demonstrated that the tracing speed is higher if curves are far apart than if they are nearby. One hypothesis is that a larger distance between curves permits activity propagation in higher visual cortex areas. In these higher areas receptive fields are larger and connections exist between neurons representing image regions that are farther apart (Pooresmaeili et al., 2014). We propose a recurrent architecture for the scale-invariant tracing of curves and objects. The architecture is composed of a feedforward pathway that dynamically selects the appropriate scale for tracing, and a recurrent pathway for propagating enhanced neuronal activity through horizontal and feedback connections, enabled by a disinhibitory loop involving VIP and SOM interneurons. We trained the network using a biologically plausible reinforcement learning scheme and observed that training on short curves allowed the networks to generalize to longer curves and 2D-objects. The network chose the scale based on the distance between curves and the width of objects, just as in human psychophysics and the visual cortex of monkeys. The results provide a mechanistic account of the learning and execution of multiscale perceptual grouping in the brain.

neuroscience↗

bioRxiv2023

Recurrent neural networks that learn multi-step visual routines with reinforcement learning

Many cognitive problems can be decomposed into series of subproblems that are solved sequentially by the brain. When subproblems are solved, relevant intermediate results need to be stored by neurons and propagated to the next subproblem, until the overarching goal has been completed. We will here consider visual tasks, which can be decomposed into sequences of elemental visual operations. Experimental evidence suggests that intermediate results of the elemental operations are stored in working memory as an enhancement of neural activity in the visual cortex. The focus of enhanced activity is then available for subsequent operations to act upon. The main question at stake is how the elemental operations and their sequencing can emerge in neural networks that are trained with only rewards, in a reinforcement learning setting. We here propose a new recurrent neural network architecture that can learn composite visual tasks that require the application of successive elemental operations. Specifically, we selected three tasks for which electrophysiological recordings of monkeys visual cortex are available. To train the networks, we used RELEARNN, a biologically plausible four-factor Hebbian learning rule, which is local both in time and space. We report that networks learn elemental operations, such as contour grouping and visual search, and execute sequences of operations, solely based on the characteristics of the visual stimuli and the reward structure of a task. After training was completed, the activity of the units of the neural network elicited by behaviorally relevant image items was stronger than that elicited by irrelevant ones, just as has been observed in the visual cortex of monkeys solving the same tasks. Relevant information that needed to be exchanged between subroutines was maintained as a focus of enhanced activity and passed on to the subsequent subroutines. Our results demonstrate how a biologically plausible learning rule can train a recurrent neural network on multistep visual tasks. Author SummaryMany visual problems, like finding your way on a map, are solved by decomposing them into a series of subproblems. For a successful decomposition, the neuronal processes that solve one subproblem must make their results available to the subsequent ones. Experiments in monkeys demonstrated that the outcomes of subproblems are represented as foci of enhanced activity in the visual cortex, which are related to attention shifts and can be used as inputs for the processes solving the next subproblems. To understand how subproblems and their sequencing are learned, we trained a recurrent artificial neural network on the same tasks that monkeys performed, using a biologically plausible reinforcement learning rule. The networks learned the tasks and, importantly, the activation of the networks units resembled the spatiotemporal patterns of activity observed in the visual cortex of monkeys. Our results shed light on how recurrent neural networks trained with a biologically plausible learning rule can learn to propagate enhanced activity between subroutines to solve complex visual tasks.

neuroscience↗

bioRxiv2023

Brain2GAN: Feature-disentangled neural coding of visual perception in the primate brain

A challenging goal of neural coding is to characterize the neural representations underlying visual perception. To this end, multi-unit activity (MUA) of macaque visual cortex was recorded in a passive fixation task upon presentation of faces and natural images. We analyzed the relationship between MUA and latent representations of state-of-the-art deep generative models, including the conventional and feature-disentangled representations of generative adversarial networks (GANs) (i.e., z- and w-latents of StyleGAN, respectively) and language-contrastive representations of latent diffusion networks (i.e., CLIP-latents of Stable Diffusion). A mass univariate neural encoding analysis of the latent representations showed that feature-disentangled w representations outperform both z and CLIP representations in explaining neural responses. Further, w-latent features were found to be positioned at the higher end of the complexity gradient which indicates that they capture visual information relevant to high-level neural activity. Subsequently, a multivariate neural decoding analysis of the feature-disentangled representations resulted in state-of-the-art spatiotemporal reconstructions of visual perception. Taken together, our results not only highlight the important role of feature-disentanglement in shaping high-level neural representations underlying visual perception but also serve as an important benchmark for the future of neural coding. Author summaryNeural coding seeks to understand how the brain represents the world by modeling the relationship between stimuli and internal neural representations thereof. This field focuses on predicting brain responses to stimuli (neural encoding) and deciphering information about stimuli from brain activity (neural decoding). Recent advances in generative adversarial networks (GANs; a type of machine learning model) have enabled the creation of photorealistic images. Like the brain, GANs also have internal representations of the images they create, referred to as "latents". More recently, a new type of feature-disentangled "w-latent" of GANs has been developed that more effectively separates different image features (e.g., color; shape; texture). In our study, we presented such GAN-generated pictures to a macaque with cortical implants and found that the underlying w-latents were accurate predictors of high-level brain activity. We then used these w-latents to reconstruct the perceived images with high fidelity. The remarkable similarities between our predictions and the actual targets indicate alignment in how w-latents and neural representations represent the same stimulus, even though GANs have never been optimized on neural data. This implies a general principle of shared encoding of visual phenomena, emphasizing the importance of feature disentanglement in deeper visual areas.

neuroscience↗

bioRxiv2022

Biologically plausible phosphene simulation forthe differentiable optimization of visual corticalprostheses

Blindness affects millions of people around the world, and is expected to become increasingly prevalent in the years to come. For some blind individuals, a promising solution to restore a form of vision are cortical visual prostheses, which convert camera input to electrical stimulation of the cortex to bypass part of the impaired visual system. Due to the constrained number of electrodes that can be implanted, the artificially induced visual percept (a pattern of localized light flashes, or phosphenes) is of limited resolution, and a great portion of the fields research attention is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is the non-invasive functional evaluation in sighted subjects or with computational models by making use of simulated prosthetic vision (SPV) pipelines. Although the SPV literature has provided us with some fundamental insights, an important drawback that researchers and clinicians may encounter is the lack of realism in the simulation of cortical prosthetic vision, which limits the validity for real-life applications. Moreover, none of the existing simulators address the specific practical requirements for the electrical stimulation parameters. In this study, we developed a PyTorch-based, fast and fully differentiable phosphene simulator. Our simulator transforms specific electrode stimulation patterns into biologically plausible representations of the artificial visual percepts that the prosthesis wearer is expected to see. The simulator integrates a wide range of both classical and recent clinical results with neurophysiological evidence in humans and non-human primates. The implemented pipeline includes a model of the retinotopic organisation and cortical magnification of the visual cortex. Moreover, the quantitative effect of stimulation strength, duration, and frequency on phosphene size and brightness as well as the temporal characteristics of phosphenes are incorporated in the simulator. Our results demonstrate the suitability of the simulator for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioural experiments. The modular approach of our work makes it ideal for further integrating new insights in artificial vision as well as for hypothesis testing. In summary, we present an open-source, fully differentiable, biologically plausible phosphene simulator as a tool for computational, clinical and behavioural neuroscientists working on visual neuroprosthetics.

neuroscience↗

bioRxiv2020

Direct comparison of population receptive fields from fMRI and large-scale neurophysiological recordings in awake non-human primates.

Population receptive field (pRF) modeling is a popular fMRI method to map the retinotopic organization of the human brain. While fMRI-based pRF-maps are qualitatively similar to invasively recorded single-cell receptive fields in animals, it remains unclear what neuronal signal they represent. We addressed this question in awake non-human primates comparing whole-brain fMRI and large-scale neurophysiological recordings in areas V1 and V4 of the visual cortex. We examined the fits of several pRF-models based on the fMRI BOLD-signal, multi-unit spiking activity (MUA) and local field potential (LFP) power in different frequency bands. We found that pRFs derived from BOLD-fMRI were most similar to MUA-pRFs in V1 and V4, while pRFs based on LFP gamma power also gave a good approximation. FMRI-based pRFs thus reliably reflect neuronal receptive field properties in the primate brain. In addition to our results in V1 and V4, the whole-brain fMRI measurements revealed retinotopic tuning in many other cortical and subcortical areas with a consistent increase in pRF-size with increasing eccentricity, as well as a retinotopically specific deactivation of default-mode network nodes similar to previous observations in humans.

neuroscience↗

bioRxiv2020

Object selection by automatic spreading of top-down attentional signals in V1

What is selected when attention is directed to a specific location of the visual field? Theories of object-based attention have suggested that when spatial attention is directed to part of an object, attention does not simply enhance the attended location but automatically spreads to enhance all locations that comprise the object. Here, we tested this hypothesis by reconstructing the distribution of attention from population neuronal activity patterns in V1 using functional magnetic resonance imaging (fMRI) and population-based receptive field mapping. We find that attention spreads from a spatially cued location to the underlying object - and enhances all spatial locations that comprise the object. Importantly, this spreading was also evident when the object was not task-relevant. These data suggest that attentional selection automatically operates at an object level, facilitating the reconstruction of coherent objects from fragmented representations in early visual cortex.

neuroscience↗