Biology subjects

Kietzmann, T. C.

Publications and source records attributed to Kietzmann, T. C..

15 recordsLinked to original sources

bioRxiv2026

Exploring Neural Signatures of Self-Initiated Saccadic and Blink Events in the Real World

During natural vision, the human brain constantly has to account for self-produced interruptions of visual input, through blinks and saccades, to maintain a stable perception of the world. It remains unclear whether these event-related neural responses are treated similarly due to a common underlying mechanism. To systematically investigate this functional connection, we recorded synchronized mobile EEG and eye-tracking data from freely moving participants as they (visually) explored a city center. Our results replicate prior findings showing that a substantial portion of the neural response is aligned to event onsets in comparison to offset, underscoring the relevance of investigating the beginning and end of these events. Using advanced deconvolution techniques, we disentangled overlapping, time-locked neural contributions associated with the onsets and offsets of saccades and blinks. The results demonstrate that the deconvoluted neural kernels are highly correlated when comparing saccade onset to blink onset, as well as saccade offset to blink offset, for both evoked and induced activity across electrodes. Together, our findings demonstrate that despite their distinct physical profiles, saccades and blinks share a common neural signature that governs both their onset and offset dynamics during unconstrained real-world behavior. HighlightsO_LICo-registration of portable EEG and Eye-Tracking yielded meaningful real world data. C_LIO_LIWe studied blinks and saccades as natural interruptions of visual processes. C_LIO_LIAligning EEG data to onset and offset underscored the relevance of both timepoints. C_LIO_LIWe deconvoluted overlapping neural activity for onsets/offsets of saccades/blinks. C_LIO_LIERPs and TFRs are highly correlated for onset/offset of blinks and saccades. C_LI

neuroscience↗

bioRxiv2025

Predictive remapping and allocentric coding as consequences of energy efficiency in recurrent neural network models of active vision

Despite moving our eyes from one location to another, our perception of the world is stable - an aspect thought to rely on predictive computations that use efference copies to predict the upcoming foveal input. Are these complex computations genetically hard-coded, or can they emerge from simpler principles? Here we consider the organisms limited energy budget as a potential origin. We expose a recurrent neural network to sequences of fixation patches and saccadic efference copies, training the model to minimise energy consumption (preactivation). We show that targeted inhibitory predictive remapping emerges from this energy efficiency optimization alone. As furthermore demonstrated, this computation relies on the models learned ability to re-code egocentric eye-coordinates into an allocentric (imagecentric) reference frame. Together, our findings suggest that both allocentric coding and predictive remapping can emerge from energy efficiency constraints during active vision, demonstrating how complex neural computations can arise from simple physical principles.

neuroscience↗

bioRxiv2024

Saccade onset, not fixation onset, best explains early sensory responses across the human visual cortex during naturalistic vision

Visual processing is traditionally studied using static viewing paradigms in which researchers analyse brain responses to the onsets of a sequence of randomly selected stimuli. Translating this "stimulus onset" approach to active vision paradigms that allow for free eye-movements, researchers often consider fixation onsets as the events that trigger visual activity across the visual system. Here, we test this assumption by analysing a large-scale magnetoencephalography (MEG) dataset with simultaneously recorded eye movements of 5 participants who freely explored thousands of natural images, yielding approximately 200,000 gaze events. We show that saccade-related events, particularly peak saccade curvature, rather than fixation onsets, explain most variance in latency of the early sensory component M100. Further, comparing the classic M100 elicited by stimulus onsets with the M100s elicited during active vision revealed stark differences, both in response to saccade-related and fixation-onset events. This indicates that phenomena discovered using gold standard stimulus onset paradigms do not necessarily translate to natural vision. Our findings challenge the prevailing approach to studying vision in static paradigms and highlight the importance of internally generated and action-driven signals in the dynamics of natural sensory processing.

neuroscience↗

bioRxiv2023

High-level prediction errors in low-level visual cortex

Perception and behaviour are significantly moulded by expectations derived from our prior knowledge. Hierarchical predictive processing theories provide a principled account of the neural mechanisms underpinning these processes, casting perception as a hierarchical inference process. While numerous studies have shown stronger neural activity for surprising inputs, in line with this account, it is unclear what predictions are made across the cortical hierarchy, and therefore what kind of surprise drives this upregulation of activity. Here we leveraged fMRI and visual dissimilarity metrics derived from a deep neural network to arbitrate between two hypotheses: prediction errors may signal a local mismatch between input and expectation at each level of the cortical hierarchy, or prediction errors may incorporate feedback signals and thereby inherit complex tuning properties from higher areas. Our results are in line with this second hypothesis. Prediction errors in both low- and high-level visual cortex primarily scaled with high-level, but not low-level, visual surprise. This scaling with high-level surprise in early visual cortex strongly diverges from feedforward tuning, indicating a shift induced by predictive contexts. Mechanistically, our results suggest that high-level predictions may help constrain perceptual interpretations in earlier areas thereby aiding perceptual inference. Combined, our results elucidate the feature tuning of visual prediction errors and bolster a core hypothesis of hierarchical predictive processing theories, that predictions are relayed top-down to facilitate perception.

neuroscience↗

bioRxiv2023

Empirically identifying and computationally modelling the brain-behaviour relationship for human scene categorization

Humans effortlessly make quick and accurate perceptual decisions about the nature of their immediate visual environment, such as the category of the scene they face. Previous research has revealed a rich set of cortical representations potentially underlying this feat. However, it remains unknown which of these representations are suitably formatted for decision-making. Here, we approached this question empirically and computationally, using neuroimaging and computational modelling. For the empirical part, we collected electroencephalography (EEG) data and reaction times from human participants during a scene categorization task (natural vs. man-made). We then related neural representations to behaviour using a multivariate extension of signal detection theory. We observed a correlation specifically between [~]100 ms and [~]200 ms after stimulus onset, suggesting that the neural scene representations in this time period are suitably formatted for decision-making. For the computational part, we evaluated a recurrent convolutional neural network (RCNN) as a model of brain and behaviour. Unifying our previous observations in an image-computable model, the RCNN predicted well the neural representations, the behavioural scene categorization data, as well as the relationship between them. Our results identify and computationally characterize the neural and behavioural correlates of scene categorization in humans. Significance statementCategorizing scene information is a ubiquitous and crucial task. Here we provide an empirical and computational account of scene categorization. Previous research has identified when scenes are represented in the visual processing hierarchy, but it remains unclear which of these representations are relevant for behaviour. We identified such representations between [~]100 ms and [~]200 ms after stimulus onset. We then showed that scene categorization in humans can be modelled via a recurrent convolutional neural network in a unified manner, i.e., in terms of neural and behavioural correlates, and their relationship. Together this reveals which representations underlie scene categorization behaviour and proposes a computational mechanism that implements such representations.

neuroscience↗

bioRxiv2021

Deep neural networks and visuo-semantic models explain complementary components of human ventral-stream representational dynamics

Deep neural networks (DNNs) are promising models of the cortical computations supporting human object recognition. However, despite their ability to explain a significant portion of variance in neural data, the agreement between models and brain representational dynamics is far from perfect. We address this issue by asking which representational features are currently unaccounted for in neural timeseries data, estimated for multiple areas of the ventral stream via source-reconstructed magnetoencephalography (MEG) data acquired in human participants (9 females, 6 males) during object viewing. We focus on the ability of visuo-semantic models, consisting of human-generated labels of object features and categories, to explain variance beyond the explanatory power of DNNs alone. We report a gradual reversal in the relative importance of DNN versus visuo-semantic features as ventral-stream object rep-resentations unfold over space and time. While lower-level visual areas are better explained by DNN features, especially during the early phase of the response (< 128 ms after stimulus onset), higher-level cortical dynamics are best accounted for by visuo-semantic features during a later time window (starting 146 ms after stimulus onset). Among the visuo-semantic features, object parts and basic categories drive the advantage over DNNs. These results show that a significant component of the variance unexplained by DNNs in higher-level cortical dynamics is structured, and can be explained by readily nameable aspects of the objects. We conclude that current DNNs fail to fully capture dynamic representations in higher-level human visual cortex and suggest a path toward more accurate models of ventral stream computations. SIGNIFICANCE STATEMENTWhen we view objects such as faces and cars in our visual environment, their neural representations dynamically unfold over time at a millisecond scale. These dynamics reflect the cortical computations that support fast and robust object recognition. Deep neural networks (DNNs) have emerged as a promising framework for modeling these computations but cannot yet fully account for the neural dynamics. Using magnetoencephalography data acquired in human observers during object viewing, we show that readily nameable aspects of objects, such as "eye", "wheel", and "face", can account for variance in the neural dynamics over and above DNNs. These findings suggest that DNNs and humans may in part rely on different object features for visual recognition and provide guidelines for model improvement.

neuroscience↗

bioRxiv2021

A morphometric double dissociation: cortical thickness is more related to aging; surface area is more related to cognition

The thickness and surface area of cortex are genetically distinct aspects of brain structure, and may be affected differently by age. However, their potential to differentially predict age and cognitive abilities has been largely overlooked, likely because they are typically aggregated into the commonly used measure of volume. In a large sample of healthy adults (N=647, aged 18-88), we investigated the brain-age and brain-cognition relationships of thickness, surface area, and volume, plus five additional morphological shape metrics. Cortical thickness was the metric most strongly associated with age cross-sectionally, as well as exhibiting the steepest longitudinal change over time (subsample N=261, aged 25-84). In contrast, surface area was the best single predictor of age-residualized cognitive abilities (fluid intelligence), and changes in surface area were most strongly associated with cognitive change over time. These findings were replicated in an independent dataset (N=1345, aged 18-93). Our results suggest that cortical thickness and surface area make complementary contributions the age-brain-cognition triangle, and highlight the importance of considering these volumetric components separately.

neuroscience↗

bioRxiv2021

Wild lab: A naturalistic free viewing experiment reveals previously unknown EEG signatures of face processing.

Neural mechanisms of face perception are predominantly studied in well-controlled experimental settings that involve random stimulus sequences and fixed eye positions. While powerful, the employed paradigms are far from what constitutes natural vision. Here, we demonstrate the feasibility of ecologically more valid experimental paradigms using natural viewing behavior, by combining a free viewing paradigm on natural scenes, free of photographer bias, with advanced data processing techniques that correct for overlap effects and co-varying nonlinear dependencies of multiple eye movement parameters. We validate this approach by replicating classic N170 effects in neural responses, triggered by fixation onsets (fERPs). Importantly, our more natural stimulus paradigm yielded smaller variability between subjects than the classic setup. Moving beyond classic temporal and spatial effect locations, our experiment furthermore revealed previously unknown signatures of face processing. This includes modulation of early fERP components, as well as category-specific adaptation effects across subsequent fixations that emerge even before fixation onset.

neuroscience↗

bioRxiv2021

Predictive coding is a consequence of energy efficiency in recurrent neural networks

Predictive coding represents a promising framework for understanding brain function. It postulates that the brain continuously inhibits predictable sensory input, ensuring a preferential processing of surprising elements. A central aspect of this view is its hierarchical connectivity, involving recurrent message passing between excitatory bottom-up signals and inhibitory top-down feedback. Here we use computational modelling to demonstrate that such architectural hard-wiring is not necessary. Rather, predictive coding is shown to emerge as a consequence of energy efficiency. When training recurrent neural networks to minimise their energy consumption while operating in predictive environments, the networks self-organise into prediction and error units with appropriate inhibitory and excitatory interconnections, and learn to inhibit predictable sensory input. Moving beyond the view of purely top-down driven predictions, we furthermore demonstrate, via virtual lesioning experiments, that networks perform predictions on two timescales: fast lateral predictions among sensory units, and slower prediction cycles that integrate evidence over time.

neuroscience↗

bioRxiv2020

The genetic organization of subcortical volumetric change is stable throughout the lifespan

While development and aging of the cerebral cortex show a similar topographic organization and are mainly governed by the same genes, it is unclear whether the same is true for subcortical structures, which follow fundamentally different ontogenetic and phylogenetic principles than the cerebral cortex. To test the hypothesis that genetically governed neurodevelopmental processes can be traced in subcortical structures throughout life, we analyzed a longitudinal magnetic resonance imaging dataset (n = 974, age 4-89 years), identifying five clusters of longitudinal change in development. With some exceptions, these clusters followed placement along the cranial axis in embryonic brain development, suggesting continuity in the pattern of change from prenatal stages. Developmental change patterns were conserved through the lifespan and predicted general cognitive function in an age-invariant manner. The results were replicated in longitudinal data from the Lifebrain consortium (n = 756, age 19-83 years). Genetic contributions to longitudinal brain changes were calculated from the Vietnam Era Twin Study of Aging (n = 331 male twins, age 51-60 years), revealing that distinct sets of genes tended to govern change for each developmental cluster. This finding was confirmed with single nucleotide polymorphisms and cross-sectional MRI data from the UK Biobank (n = 20,588, age 40-69), demonstrating significantly higher co-heritability among structures belonging to the same developmental clusters. Together, these results suggest that coordination of subcortical change adheres to fundamental principles of lifespan continuity, genetic organization and age-invariant relationships to cognitive function. Significance statementHere we show that subcortical change during childhood development is organized in clusters. These clusters tend to follow the main gradient of embryonic brain development, and are stable across life. This means that subcortical regions changing together in childhood also change together throughout the rest of life, in accordance with a lifespan perspective on brain development and aging. Twin and single nucleotide polymorphism-based heritability analyses in middle-aged and older adults showed that volume and volume change of regions within each developmental cluster tended to be governed by the same sets of genes. Thus, volumetric changes across subcortical regions are tightly organized, and the coordinated change can be described in a lifespan perspective according to ontogenetic and genetic influences.

neuroscience↗

bioRxiv2020

Diverse deep neural networks all predict human IT well, after training and fitting

Deep neural networks (DNNs) trained on object recognition provide the best current models of high-level visual areas in the brain. What remains unclear is how strongly network design choices, such as architecture, task training, and subsequent fitting to brain data contribute to the observed similarities. Here we compare a diverse set of nine DNN architectures on their ability to explain the representational geometry of 62 isolated object images in human inferior temporal (hIT) cortex, as measured with functional magnetic resonance imaging. We compare untrained networks to their task-trained counterparts, and assess the effect of fitting them to hIT using a cross-validation procedure. To best explain hIT, we fit a weighted combination of the principal components of the features within each layer, and subsequently a weighted combination of layers. We test all models across all stages of training and fitting for their correlation with the hIT representational dissimilarity matrix (RDM) using an independent set of images and subjects. We find that trained models significantly outperform untrained models (accounting for 57% more of the explainable variance), suggesting that features representing natural images are important for explaining hIT. Model fitting further improves the alignment of DNN and hIT representations (by 124%), suggesting that the relative prevalence of different features in hIT does not readily emerge from the particular ImageNet object-recognition task used to train the networks. Finally, all DNN architectures tested achieved equivalent high performance once trained and fitted. Similar ability to explain hIT representations appears to be shared among deep feedforward hierarchies of nonlinear features with spatially restricted receptive fields.

neuroscience↗

bioRxiv2020

Individual differences among deep neural network models

Deep neural networks (DNNs) excel at visual recognition tasks and are increasingly used as a modelling framework for neural computations in the primate brain. However, each DNN instance, just like each individual brain, has a unique connectivity and representational profile. Here, we investigate individual differences among DNN instances that arise from varying only the random initialization of the network weights. Using representational similarity analysis, we demonstrate that this minimal change in initial conditions prior to training leads to substantial differences in intermediate and higher-level network representations, despite achieving indistinguishable network-level classification performance. We locate the origins of the effects in an under-constrained alignment of category exemplars, rather than a misalignment of category centroids. Furthermore, while network regularization can increase the consistency of learned representations, considerable differences remain. These results suggest that computational neuroscientists working with DNNs should base their inferences on multiple networks instances instead of single off-the-shelf networks.

neuroscience↗

bioRxiv2019

Self-reported sleep relates to hippocampal atrophy across the adult lifespan - results from the Lifebrain consortium

BackgroundPoor sleep is associated with multiple age-related neurodegenerative and neuropsychiatric conditions. The hippocampus plays a special role in sleep and sleep-dependent cognition, and accelerated hippocampal atrophy is typically seen with higher age. Hence, it is critical to establish how the relationship between sleep and hippocampal volume loss unfolds across the adult lifespan.\n\nMethodsSelf-reported sleep measures and MRI-derived hippocampal volumes were obtained from 3105 cognitively normal participants (18-90 years) from major European brain studies in the Lifebrain consortium. Hippocampal volume change was estimated from 5116 MRIs from 1299 participants, covering up to 11 years. Cross-sectional analyses were repeated in a sample of 21390 participants from the UK Biobank.\n\nResultsThe relationship between self-reported sleep and age differed across sleep items. Sleep duration, efficiency, problems, and use of medication worsened monotonously with age, whereas subjective sleep quality, sleep latency, and daytime tiredness improved. Women reported worse sleep in general than men, but the relationship to age was similar. No cross-sectional sleep - hippocampal volume relationships was found. However, worse sleep quality, efficiency, problems, and daytime tiredness were related to greater hippocampal volume loss over time, with high scorers showing on average 0.22% greater annual loss than low scorers. Simulations showed that longitudinal effects were too small to be detected as age-interactions in cross-sectional analyses.\n\nConclusionsWorse self-reported sleep is associated with higher rates of hippocampal decline across the adult lifespan. This suggests that sleep is relevant to understand individual differences in hippocampal atrophy, but limited effect sizes call for cautious interpretation.

neuroscience↗

bioRxiv2019

Recurrent networks can recycle neural resources to flexibly trade speed for accuracy in visual recognition

Deep feedforward neural network models of vision dominate in both computational neuroscience and engineering. The primate visual system, by contrast, contains abundant recurrent connections. Recurrent signal flow enables recycling of limited computational resources over time, and so might boost the performance of a physically finite brain or model. Here we show: (1) Recurrent convolutional neural network models outperform feedforward convolutional models matched in their number of parameters in large-scale visual recognition tasks on natural images. (2) Setting a confidence threshold, at which recurrent computations terminate and a decision is made, enables flexible trading of speed for accuracy. At a given confidence threshold, the model expends more time and energy on images that are harder to recognise, without requiring additional parameters for deeper computations. (3) The recurrent models reaction time for an image predicts the human reaction time for the same image better than several parameter-matched and state-of-the-art feedforward models. (4) Across confidence thresholds, the recurrent model emulates the behaviour of feedforward control models in that it achieves the same accuracy at approximately the same computational cost (mean number of floating-point operations). However, the recurrent model can be run longer (higher confidence threshold) and then outperforms parameter-matched feedforward comparison models. These results suggest that recurrent connectivity, a hallmark of biological visual systems, may be essential for understanding the accuracy, flexibility, and dynamics of human visual recognition. Author summaryDeep neural networks provide the best current models of biological vision and achieve the highest performance in computer vision. Inspired by the primate brain, these models transform the image signals through a sequence of stages, leading to recognition. Unlike brains in which outputs of a given computation are fed back into the same computation, these models do not process signals recurrently. The ability to recycle limited neural resources by processing information recurrently could explain the accuracy and flexibility of biological visual systems, which computer vision systems cannot yet match. Here we report that recurrent processing can improve recognition performance compared to similarly complex feedforward networks. Recurrent processing also enabled models to behave more flexibly and trade off speed for accuracy. Like humans, the recurrent network models can compute longer when an object is hard to recognise, which boosts their accuracy. The models recognition times predicted human recognition times for the same images. The performance and flexibility of recurrent neural network models illustrates that modeling biological vision can help us improve computer vision.

neuroscience↗

bioRxiv2017

Deep Neural Networks In Computational Neuroscience

The goal of computational neuroscience is to find mechanistic explanations of how the nervous system processes information to give rise to cognitive function and behaviour. At the heart of the field are its models, i.e. mathematical and computational descriptions of the system being studied, which map sensory stimuli to neural responses and/or neural to behavioural responses. These models range from simple to complex. Recently, deep neural networks (DNNs) have come to dominate several domains of artificial intelligence (AI). As the term \"neural network\" suggests, these models are inspired by biological brains. However, current DNNs neglect many details of biological neural networks. These simplifications contribute to their computational efficiency, enabling them to perform complex feats of intelligence, ranging from perceptual (e.g. visual object and auditory speech recognition) to cognitive tasks (e.g. machine translation), and on to motor control (e.g. playing computer games or controlling a robot arm). In addition to their ability to model complex intelligent behaviours, DNNs excel at predicting neural responses to novel sensory stimuli with accuracies well beyond any other currently available model type. DNNs can have millions of parameters, which are required to capture the domain knowledge needed for successful task performance. Contrary to the intuition that this renders them into impenetrable black boxes, the computational properties of the network units are the result of four directly manipulable elements: input statistics, network structure, functional objective, and learning algorithm. With full access to the activity and connectivity of all units, advanced visualization techniques, and analytic tools to map network representations to neural data, DNNs represent a powerful framework for building task-performing models and will drive substantial insights in computational neuroscience.

neuroscience↗