Biology subjects

Bosch, S. E.

Publications and source records attributed to Bosch, S. E..

7 recordsLinked to original sources

bioRxiv2020

Hyperrealistic neural decoding: Linear reconstruction of face stimuli from fMRI measurements via the GAN latent space

Neural decoding can be conceptualized as the problem of mapping brain responses back to sensory stimuli via a feature space. We introduce (i) a novel experimental paradigm which uses well-controlled yet highly naturalistic stimuli with a priori known feature representations and (ii) an implementation thereof for HYPerrealistic reconstruction of PERception (HYPER) of faces from brain recordings. To this end, we embrace the use of generative adversarial networks (GANs) at the earliest step of our neural decoding pipeline by acquiring fMRI data as subjects perceive face images synthesized by the generator network of a GAN. We show that the latent vectors used for generation effectively capture the same defining stimulus properties as the fMRI measurements. As such, GAN latent vectors can be used as features underlying the perceived images that can be predicted for (re-)generation, leading to the most accurate reconstructions of perception to date.

neuroscience↗

bioRxiv2019

DeepRF: Ultrafast population receptive field mapping with deep learning

Population receptive field (pRF) mapping is an important asset for cognitive neuroscience. The pRF model is used for estimating retinotopy, defining functional localizers and to study a vast amount of cognitive tasks. In a classic pRF, the cartesian location and receptive field size are modeled as a 2D Gaussian kernel in visual space and are estimated by optimizing the fit between observed responses and predicted responses. In the standard framework this is achieved using an iterative gradient descent algorithm. This optimization is time consuming because the number of pRFs to fit (e.g., fMRI voxels) is typically large. This computation time increases further with the complexity of the pRF model (e.g., adding HRF parameters, surround suppression and uncertainty measures). Here, we introduce DeepRF, which uses deep convolutional neural networks to estimate pRFs. We compare the performance of DeepRF with that of the conventional method using a synthetic dataset for which the ground truth is known and an empirical dataset. We show that DeepRF achieves state-of-the-art performance while being more than 3 orders of magnitude faster than the conventional method. This enables easier and faster modeling of more complex pRF models, resolving an important limitation of the conventional approach.

neuroscience↗

bioRxiv2019

A large single-participant fMRI dataset for probing brain responses to naturalistic stimuli in space and time

Visual and auditory representations in the human brain have been studied with encoding, decoding and reconstruction models. Representations from convolutional neural networks have been used as explanatory models for these stimulus-induced hierarchical brain activations. However, none of the fMRI datasets currently available has adequate amounts of data for sufficiently sampling their representations. We recorded a densely sampled large fMRI dataset (TR=700 ms) in a single individual exposed to spatiotemporal visual and auditory naturalistic stimuli (30 episodes of BBCs Doctor Who). The data consists of 120.830 whole-brain volumes (approx. 23 h) of single-presentation data (full episodes, training set) and 1.178 volumes (11 min) of repeated narrative short episodes (test set, 22 repetitions), recorded with fixation over a period of six months. This rich dataset can be used widely to study the way the brain represents audiovisual input across its sensory hierarchies.

neuroscience↗

bioRxiv2019

Individual differences in the influence of mental imagery on conscious perception

Mental imagery and visual perception rely on similar neural mechanisms, but the function of this overlap remains unclear. One idea is that imagery can influence perception. Previous research has shown that imagining a stimulus prior to binocular presentation of rivalling stimuli increases the chance of perceiving the imagined stimulus. In this study we investigated how this effect interacts with bottom-up sensory input by comparing psychometric response curves for congruent and incongruent imagery in humans. A Bayesian hierarchical model was used, allowing us to simultaneously study group-level effects as well as effects for individual participants. We found strong effects of both imagery as well as its interaction with sensory evidence within individual participants. However, the direction of these effects were highly variable between individuals, leading to weak effects at the group level. This highlights the heterogeneity of conscious perception and emphasizes the need for individualized investigation of such complex cognitive processes.

neuroscience↗

bioRxiv2017

Differential temporal dynamics during visual imagery and perception

Visual perception and imagery rely on similar representations in the visual cortex. During perception, visual activity is characterized by distinct processing stages, but the temporal dynamics underlying imagery remain unclear. Here, we investigated the dynamics of visual imagery in human participants using magnetoencephalography. We show that, contrary to perception, the onset of imagery is characterized by broad temporal generalization. Furthermore, there is consistent overlap between imagery and perceptual processing around 150 ms and from 300 ms after stimulus onset, presumably reflecting completion of the feedforward sweep and perceptual stabilization respectively. These results indicate that during imagery either the complete representation is activated at once and does not include low-level visual areas, or the order in which visual features are activated is less fixed and more flexible than during perception. These findings have important implications for our understanding of the neural mechanisms of visual imagery.

neuroscience↗

bioRxiv2017

CNN-based Encoding and Decoding of Visual Object Recognition in Space and Time

Deep convolutional neural networks (CNNs) have been put forward as neurobiologically plausible models of the visual hierarchy. Using functional magnetic resonance imaging, CNN representations of visual stimuli have previously been shown to correspond to processing stages in the ventral and dorsal streams of the visual system. Whether this correspondence between models and brain signals also holds for activity acquired at high temporal resolution has been explored less exhaustively. Here, we addressed this question by combining CNN-based encoding models with magnetoencephalography (MEG). Human participants passively viewed 1000 images of objects while MEG signals were acquired. We modelled their high temporal resolution source-reconstructed cortical activity with CNNs, and observed a feedforward sweep across the visual hierarchy between 75-200 ms after stimulus onset. This spatiotemporal cascade was captured by the network layer representations, where the increasingly abstract stimulus representation in the hierarchical network model was reflected in different parts of the visual cortex, following the visual ventral stream. We further validated the accuracy of our encoding model by decoding stimulus identity in a left-out validation set of viewed objects, achieving state-of-the-art decoding accuracy.

neuroscience↗

bioRxiv2016

Modeling Cognitive Processes with Neural Reinforcement Learning

Artificial neural networks (ANNs) have seen renewed interest in the fields of computer science, artificial intelligence and neuroscience. Recent advances in improving the performance of ANNs open up an exciting new avenue for cognitive neuroscience research. Here, we propose that ANNs that learn to solve complex tasks based on reinforcement learning, can serve as a universal computational framework for analyzing the neural and behavioural correlates of cognitive processing. We demonstrate this idea on a challenging probabilistic categorization task, where neural network dynamics are linked to human behavioural and neural data as identical tasks are solved.

neuroscience↗