Biology subjects

Muckli, L.

Publications and source records attributed to Muckli, L..

13 recordsLinked to original sources

bioRxiv2024

Experience-dependent predictions of feedforward and contextual information in mouse visual cortex

To navigate complex environments, the visual system must prioritize unexpected inputs or omissions over predictable sensory backgrounds. This saliency assignment is a hallmark of visual processing and depends on comparing visual feedforward inputs with learned contextual surrounds, a process often conceptualized as a difference computation. However, cortical circuitry poses fundamental challenges: visual contextual feedback is predominantly amplifying, and feedback-driven inhibitory neurons necessary for subtractive computations are relatively sparse. How then can complex contextual feedback be selectively compared with feedforward input at the level of individual neurons? Here, we show that divergent plasticity rules within single pyramidal neurons provide a solution to this problem. Using two-photon calcium imaging, we recorded layer 2/3 pyramidal cell responses in mouse primary visual cortex before and after visual familiarization. We dissociated feedforward and contextual signals by comparing responses to stimuli with and without an occluded sector. Repeated feedforward responses selectively weakened through adaptation and surround suppression, whereas contextual responses strengthened and generalized across scenes. These strengthened contextual inputs amplified non-adapted responses to novel images and drove neuronal activity when occlusion of feedforward input prevented surround suppression. This produced a conserved structure of feedforward and contextual responses across mice, monkeys, and humans. Ultimately, these results reveal a cellular mechanism for saliency assignment that does not require feedback-driven inhibition. Pyramidal neurons autonomously learn the generalized statistical structure of familiar contexts, allowing them to selectively amplify improbable inputs or omissions.

neuroscience↗

bioRxiv2024

Retinotopic biases in contextual feedback signals to V1 for object and scene processing

Identifying the objects embedded in natural scenes relies on recurrent processing between lower and higher visual areas. How is cortical feedback information related to objects and scenes organised in lower visual areas? The spatial organisation of cortical feedback converging in early visual cortex during object and scene processing could be retinotopically specific as it is coded in V1, or object centred as coded in higher areas, or both. Here, we characterise object and scene-related feedback information to V1. Participants identified foreground objects or background scenes in images with occluded central and peripheral subsections, allowing us to isolate feedback activity to foveal and peripheral regions of V1. Using fMRI and multivoxel pattern classification, we found that feedback of object information is projected to foveal V1 cortex with increased detail during an object identification task. Background scene information is projected to both foveal and peripheral V1 but can be disrupted by a sufficiently demanding object discrimination task. We suggest that the feedback connections during scene perception project back to earlier visual areas an automatic sketch of occluded information to the predicted retinotopic location. In the case of a cognitive task however, feedback pathways project content to foveal retinotopic space, potentially for introspection, functioning as a cognitive active blackboard and not necessarily predicting the objects location. This feedback architecture could reflect the internal mapping in V1 of the brains endogenous models of the visual environment that are used to predict perceptual inputs.

neuroscience↗

bioRxiv2023

Perceptual priors add sensory detail to contextual feedback processing in V1

How do we develop models of the world? Contextualising ambiguous information with previous experience allows us to form an enriched perception. Contextual information and prior knowledge facilitate perceptual processing, improving our recognition of even distorted or obstructed visual inputs. As a result, neuronal processing elicited by identical sensory inputs varies depending on the context in which we encounter those inputs. This modulation is in line with predictive processing accounts of vision which suggest that the brain uses internal models of the world to predict sensory inputs, with cortical feedback processing in sensory areas encoding beliefs about those inputs. As such, acquiring knowledge should enhance our internal models that we use to resolve sensory ambiguities, and feedback signals should encode more accurate estimates of sensory inputs. We used partially occluded Mooney images, ambiguous two-tone images which are difficult to recognise without prior knowledge of the image content, in behavioural and 3T fMRI experiments to measure if contextual feedback signals in early visual areas are modulated by learning. We show that perceptual priors add sensory detail to contextual feedback processing in early visual areas in response to subsequent presentations of previously ambiguous images.

neuroscience↗

bioRxiv2022

Feedback brings scene information to the representation of occluded image regions in area V1 of monkeys and humans

Neuronal activity in the primary visual cortex (V1) is driven by feedforward input from within the neurons receptive fields (RFs) and modulated by contextual information in regions surrounding the RF. The effect of contextual information on spiking activity occurs rapidly, and is therefore challenging to dissociate from feedforward input. To address this challenge, we recorded the spiking activity of V1 neurons in monkeys viewing either natural scenes or scenes where the information in the RF was occluded, effectively removing the feedforward input. We found that V1 neurons responded rapidly and selectively to occluded scenes. V1 responses elicited by occluded stimuli could be used to decode scene identity and could be predicted from those elicited by non-occluded images, indicating that there is overlap between visually-driven and contextual responses. We used representational similarity analysis to show that the structure of V1 representations of occluded scenes measured with electrophysiology in monkeys correlates strongly with the representations of the same scenes in humans measured with fMRI. Our results reveal that contextual influences alter the spiking activity of V1 in monkeys across large distances on a rapid time scale, carry information about scene identity and resemble those in human V1.

neuroscience↗

bioRxiv2021

Concurrent contextual and time-distant mnemonic information co-exist as feedback in human visual cortex.

Efficient processing of visual environment necessitates the integration of incoming sensory evidence with concurrent contextual inputs and mnemonic content from our past experiences. To delineate how this integration takes place in the brain, we studied modulations of feedback neural patterns in non-stimulated areas of the early visual cortex in humans (i.e., V1 and V2). Using functional magnetic resonance imaging and multivariate pattern analysis, we show that both, concurrent contextual and time-distant mnemonic information, coexist in V1/V2 as feedback signals. The extent to which mnemonic information is reinstated in V1/V2 depends on whether the information is retrieved episodically or semantically. These results demonstrate that our stream of visual experience contains not just information from the visual surrounding, but also memory-based predictions internally generated in the brain. One-Sentence SummaryFeedback activity in human early visual cortex contains concurrent contextual and time-distant mnemonic information.

neuroscience↗

bioRxiv2021

Increased region of surround stimulation enhances contextual feedback and feedforward processing in human V1

The majority of synaptic inputs to the primary visual cortex (V1) are non-feedforward, instead originating from local and anatomical feedback connections. Animal electrophysiology experiments show that feedback signals originating from higher visual areas with larger receptive fields modulate the surround receptive fields of V1 neurons. Theories of cortical processing propose various roles for feedback and feedforward processing, but systematically investigating their independent contributions to cortical processing is challenging because feedback and feedforward processes coexist even in single neurons. Capitalising on the larger receptive fields of higher visual areas compared to primary visual cortex (V1), we used an occlusion paradigm that isolates top-down influences from feedforward processing. We utilised functional magnetic resonance imaging (fMRI) and multi-voxel pattern analysis methods in humans viewing natural scene images. We parametrically measured how the availability of contextual information determines the presence of detectable feedback information in non-stimulated V1, and how feedback information interacts with feedforward processing. We show that increasing the visibility of the contextual surround increases scene-specific feedback information, and that this contextual feedback enhances feedforward information. Our findings are in line with theories that cortical feedback signals transmit internal models of predicted inputs. Significance StatementThe visual system has circuit mechanisms for processing scene context. These circuits involve lateral and feedback inputs to neurons. These inputs interact with feedforward inputs and modulate neuronal responses to visual stimuli presented outside their receptive fields. Systematically investigating independent contributions of feedback and feedforward processes is challenging because they coexist even in single neurons. Here we use an occlusion paradigm to isolate feedback and lateral signals in human participants viewing natural scene images in fMRI. We show that increasing the visibility of the contextual surround increases scene-specific feedback information, which also enhances feedforward signals. Our findings are in line with theories that cortical feedback signals carry abstract internal models that combine with more detailed representations in primary visual cortex.

neuroscience↗

bioRxiv2020

High-resolution line-scanning reveals distinct visual response properties across human cortical layers

Our understanding of the human brain relies on advancing noninvasive brain imaging approaches. Characterization of the function of brain circuitry depends on the spatiotemporal correspondence at which recorded signals can be mapped onto underlying neuronal structures and processes. Here we aimed to address key first-stage questions of feasibility, reliability, and utility of line-scanning fMRI as a next generation non-invasive imaging method for human neuroscience research at the mesoscopic scale. Line-scanning can achieve high spatial resolution by employing anisotropic voxels aligned to cortical layers. The method can simultaneously achieve high temporal resolution by limiting acquisition to a very small patch of cortex which is repeatedly acquired as a single frequency-encoded k-space line. We developed multi-echo line-scanning procedures to record cortical layers in humans at high spatial (200 m) and temporal resolution (100 ms) using ultra high-field 7T fMRI. Quantitative mapping allowed us to identify cortical layers in primary visual cortex (V1) and record functional signals from them while participants viewed movie clips. Analysis of these recordings revealed layer-specific V1 spatial and orientation tuning properties analogous to those previously observed in electrophysiological recordings of non-human primates. We have consequently demonstrated that line-scanning is a powerful non-invasive imaging technique for investigating mesoscopic functional circuits in human cortex. Graphical Abstract O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=82 SRC="FIGDIR/small/179762v1_ufig1.gif" ALT="Figure 1"> View larger version (28K): org.highwire.dtl.DTLVardef@c314f9org.highwire.dtl.DTLVardef@1960fd4org.highwire.dtl.DTLVardef@e95d73org.highwire.dtl.DTLVardef@f6680f_HPS_FORMAT_FIGEXP M_FIG C_FIG

neuroscience↗

bioRxiv2020

An Unsupervised Deep Neural Network for Image Completion Resembles Early Visual Cortex fMRI Activity Patterns for Occluded Scenes

AO_SCPLOWBSTRACTC_SCPLOWThe promise of artificial intelligence in understanding biological vision relies on the comparison of computational models with brain data with the goal of capturing functional principles of visual information processing. Convolutional neural networks (CNN) have successfully matched the transformations in hierarchical processing occurring along the brains feedforward visual pathway extending into ventral temporal cortex. However, we are still to learn if CNNs can successfully describe feedback processes in early visual cortex. Here, we investigated similarities between human early visual cortex and a CNN with encoder/decoder architecture, trained with self-supervised learning to fill occlusions and reconstruct an unseen image. Using Representational Similarity Analysis (RSA), we compared 3T fMRI data from a non-stimulated patch of early visual cortex in human participants viewing partially occluded images, with the different CNN layer activations from the same images. Results show that our self-supervised image-completion network outperforms a classical object-recognition supervised network (VGG16) in terms of similarity to fMRI data. This provides additional evidence that optimal models of the visual system might come from less feedforward architectures trained with less supervision. We also find that CNN decoder pathway activations are more similar to brain processing compared to encoder activations, suggesting an integration of mid- and low/middle-level features in early visual cortex. Challenging an AI model and the human brain to solve the same task offers a valuable way to compare CNNs with brain data and helps to constrain our understanding of information processing such as neuronal predictive coding.

neuroscience↗

bioRxiv2019

Multivoxel Pattern of Blood Oxygen Level Dependent Activity can be sensitive to stimulus specific fine scale responses

1.At ultra-high field, fMRI voxels can span the sub-millimeter range, allowing the recording of blood oxygenation level dependent (BOLD) responses at the level of fundamental units of neural computation, such as cortical columns and layers. This sub-millimeter resolution, however, is only nominal in nature as a number of factors limit the spatial acuity of functional voxels. Multivoxel Pattern Analysis (MVPA) may provide a means to detect information at finer spatial scales that may otherwise not be visible at the single voxel level due to limitations in sensitivity and specificity. Here, we evaluate the spatial scale of stimuli specific BOLD responses in multivoxel patterns exploited by linear Support Vector Machine, Linear Discriminant Analysis and Naive Bayesian classifiers across cortical depths in V1. To this end, we artificially misaligned the testing relative to the training portion of the data in increasing spatial steps, then investigated the breakdown of the classifiers performances. A one voxel shift led to a significant decrease in decoding accuracy (p<.05) across all cortical depths, indicating that stimulus specific responses in a multivoxel pattern of BOLD activity exploited by multivariate decoders can be as precise as the nominal resolution of single voxels (here .8 mm isotropic). Our results further indicate that large draining vessels, prominently residing in proximity of the pial surface, do not, in this case, hinder the ability of MVPA to exploit fine scale patterns of BOLD signals. We argue that tailored analytical approaches can help overcoming limitations in high-resolution fMRI and permit studying the mesoscale organization of the human brain with higher sensitivities.

neuroscience↗

bioRxiv2019

Two distinct feedback codes in V1 for \"real\" and \"imaginary\" internal experiences

Visual illusions and visual imagery are conscious sensory events that lack a corresponding physical input. But while everyday mental imagery feels distinct from incoming stimulus input, visual illusions, like hallucinations, are under limited volitional control and appear indistinguishable from physical reality. Illusions are thought to arise from lower-level processes within sensory cortices. In contrast, imagery involves a wide network of brain areas that recruit early visual cortices for the sensory representation of the imagined stimulus. Here, we combine laminar fMRI brain imaging with psychophysical methods and multivariate pattern analysis to investigate in human participants how seemingly real and imaginary non-physical experiences are processed in primary visual cortex (V1). We find that the content of mental imagery is only decodable in deep layers, whereas illusory content is only decodable at superficial depths. This suggests that feedback to the different layers may serve distinct functions: low-level feedback to superficial layers might be responsible for shaping perception-like experiences, while deep-layer feedback might serve the formation of a more malleable inner world, separate from ongoing perception.

neuroscience↗

bioRxiv2019

Transfer learning of deep neural network representations for fMRI decoding

BackgroundDeep neural networks have revolutionised machine learning, with unparalleled performance in object classification. However, in brain imaging (e.g. fMRI), the direct application of Convolutional Neural Networks (CNN) to decoding subject states or perception from imaging data seems impractical given the scarcity of available data. New methodIn this work we propose a robust method to transfer information from deep learning (DL) features to brain fMRI data with the goal of decoding. By adopting Reduced Rank Regression with Ridge Regularisation we establish a multivariate link between imaging data and the fully connected layer (fc7) of a CNN. We exploit the reconstructed fc7 features by performing an object image classification task on two datasets: one of the largest fMRI databases, taken from different scanners from more than two hundred subjects watching different movie clips, and another with fMRI data taken while watching static images, ResultsThe fc7 features could be significantly reconstructed from the imaging data, and led to significant decoding performance. Comparison with existing methodsThe decoding based on reconstructed fc7 outperformed the decoding based on imaging data alone. ConclusionIn this work we show how to improve fMRI-based decoding benefiting from the mapping between functional data and CNN features. The potential advantage of the proposed method is twofold: the extraction of stimuli representations by means of an automatic procedure (unsupervised) and the embedding of high-dimensional neuroimaging data onto a space designed for visual object discrimination, leading to a more manageable space from dimensionality point of view.

neuroscience↗

bioRxiv2018

Temporal Multivariate Pattern Analysis (tMVPA): a single trial approach exploring the temporal dynamics of the BOLD signal

BackgroundfMRI provides spatial resolution that is unmatched by any non-invasive neuroimaging technique. Its temporal dynamics however are typically neglected due to the sluggishness of the hemodynamic based fMRI signal.\n\nNew MethodsWe present temporal multivariate pattern analysis (tMVPA), a method for investigating the temporal evolution of neural representations in fMRI data, computed using pairs of single-trial BOLD time-courses, leveraging both spatial and temporal components of the fMRI signal. We implemented an expanding sliding window approach that allows identifying the time-window of an effect.\n\nResultsWe demonstrate that tMVPA can successfully detect condition-specific multivariate modulations over time, in the absence of univariate differences. Using Monte Carlo simulations and synthetic data, we quantified family-wise error rate (FWER) and statistical power. Both at the group and at the single subject level, FWER was either at or significantly below 5%. For the group level, we reached the desired power with 18 subjects and 12 trials; for the single subject scenario, 14 trials were required to achieve comparable power.\n\nComparison with existing methodstMVPA adds a temporal multivariate dimension to the tools available for fMRI analysis, enabling investigations of the evolution of neural representations over time. Moreover, tMVPA permits performing single subject inferential statistics by considering single-trial distribution.\n\nConclusionThe growing interest in fMRI temporal dynamics, motivated by recent evidence suggesting that the BOLD signal carries temporal information at a finer scale than previously thought, advocates the need for analytical tools, such as the tMVPA approach proposed here, tailored to investigating BOLD temporal information.

neuroscience↗

bioRxiv2017

Predictive feedback to V1 dynamically updates with sensory input

Predictive coding theories propose that the brain creates internal models of the environment to predict upcoming sensory input. Hierarchical predictive coding models of vision postulate that higher visual areas generate predictions of sensory inputs and feed them back to early visual cortex. In V1, sensory inputs that do not match the predictions lead to amplified brain activation, but does this amplification process dynamically update to new retinotopic locations with eye-movements? We investigated the effect of eye-movements in predictive feedback using functional brain imaging and eye-tracking whilst presenting an apparent motion illusion. Apparent motion induces an internal model of motion, during which sensory predictions of the illusory motion feed back to V1. We observed attenuated BOLD responses to predicted stimuli at the new post-saccadic location in V1. Therefore, pre-saccadic predictions update their retinotopic location in time for post-saccadic input, validating dynamic predictive coding theories in V1.

neuroscience↗