Biology subjects

Kay, K.

Publications and source records attributed to Kay, K..

At least 37 records · Page 2Linked to original sources

bioRxiv2024

Disentangling signal and noise in neural responses through generative modeling

Measurements of neural responses to identically repeated experimental events often exhibit large amounts of variability. This noise is distinct from signal, operationally defined as the average expected response across repeated trials for each given event. Accurately distinguishing signal from noise is important, as each is a target that is worthy of study (many believe noise reflects important aspects of brain function) and it is important not to confuse one for the other. Here, we describe a principled modeling approach in which response measurements are explicitly modeled as the sum of samples from multivariate signal and noise distributions. In our proposed method--termed Generative Modeling of Signal and Noise (GSN)--the signal distribution is estimated by subtracting the estimated noise distribution from the estimated data distribution. Importantly, GSN improves estimates of the signal distribution, but does not provide improved estimates of responses to individual events. We validate GSN using ground-truth simulations and show that it compares favorably with related methods. We also demonstrate the application of GSN to empirical fMRI data to illustrate a simple consequence of GSN: by disentangling signal and noise components in neural responses, GSN denoises principal components analysis and improves estimates of dimensionality. We end by discussing other situations that may benefit from GSNs characterization of signal and noise, such as estimation of noise ceilings for computational models of neural activity. A code toolbox for GSN is provided with both MATLAB and Python implementations.

neuroscience↗

bioRxiv2024

Recent statistics shift object representations in parahippocampal cortex

Our representations of the world need to be stable enough to support general knowledge but flexible enough to incorporate new information as our environment changes. How does the human brain manage this stability-plasticity trade-off? We analyzed a large dataset in which participants viewed objects embedded in thousands of natural scenes across many fMRI sessions. Semantic item representations were located by jointly leveraging a voxelwise encoding model to find reliable item representations and a word-embedding model to evaluate semantic content. Within the medial temporal lobe, semantic item representations in hippocampal subfield CA1, parahippocampal cortex, and perirhinal cortex gradually drifted across a period of multiple months. Whole-brain analyses revealed a gradient of plasticity in the temporal lobe, with drift more evident in anterior than posterior areas. On short timescales, rapid plasticity was observed only in parahippocampal cortex, such that item co-occurrence statistics warped item representations within a single session. Together, the results suggest that the brain solves the stability-plasticity trade-off through a gradient of plasticity across semantic regions.

neuroscience↗

bioRxiv2023

A single computational objective drives specialization of streams in visual cortex

Human visual cortex is organized into dorsal, lateral, and ventral streams. A long-standing hypothesis is that the functional organization into streams emerged to support distinct visual behaviors. Here, we compare neural network-based computational models against a massive fMRI dataset to investigate why visual streams emerge. We find that a self-supervised Topographic Deep Artificial Neural Network (TDANN), which encourages nearby units to respond similarly, better captures brain responses, as well as spatial segregation and functional differentiation across streams, than DANN models trained for stream-specific visual behaviors. These findings challenge the prevailing view that streams evolved to separately support different behaviors and suggest instead the possibility that functional organization can arise from a single principle: learning generally useful visual representations subject to local spatial constraints.

neuroscience↗

bioRxiv2023

Social cognitive regions of human association cortex are selectively connected to the amygdala

Reasoning about someones thoughts and intentions - i.e., forming a theory of mind - is an important aspect of social cognition that relies on association areas of the brain that have expanded disproportionately in the human lineage. We recently showed that these association zones comprise parallel distributed networks that, despite occupying adjacent and interdigitated regions, serve dissociable functions. One network is selectively recruited by theory of mind processes. What circuit properties differentiate these parallel networks? Here, we show that social cognitive association areas are intrinsically and selectively connected to regions of the anterior medial temporal lobe that are implicated in emotional learning and social behaviors, including the amygdala at or near the basolateral complex and medial nucleus. The results suggest that social cognitive functions emerge through coordinated activity between amygdala circuits and a distributed association network, and indicate the medial nucleus may play an important role in social cognition in humans.

neuroscience↗

bioRxiv2023

Tumor cell-derived spermidine promotes a pro-tumorigenic immune microenvironment in glioblastoma via CD8+ T cell inhibition

The glioblastoma microenvironment is enriched in immunosuppressive factors that potently interfere with the function of cytotoxic T lymphocytes. Cancer cells can directly impact the immune system, but the mechanisms driving these interactions are not completely clear. Here we demonstrate that the polyamine metabolite spermidine is elevated in the glioblastoma tumor microenvironment. Exogenous administration of spermidine drives tumor aggressiveness in an immune-dependent manner in pre-clinical mouse models via reduction of CD8+ T cell frequency and phenotype. Knockdown of ornithine decarboxylase, the rate-limiting enzyme in spermidine synthesis, did not impact cancer cell growth in vitro but did result in extended survival. Furthermore, glioblastoma patients with a more favorable outcome had a significant reduction in spermidine compared to patients with a poor prognosis. Our results demonstrate that spermidine functions as a cancer cell-derived metabolite that drives tumor progression by reducing CD8+T cell number and function.

cancer biology↗

bioRxiv2023

A mathematical theory of relational generalization in transitive inference

Humans and animals routinely infer relations between different items or events and generalize these relations to novel combinations of items. This allows them to respond appropriately to radically novel circumstances and is fundamental to advanced cognition. However, how learning systems (including the brain) can implement the necessary inductive biases has been unclear. Here we investigated transitive inference (TI), a classic relational task paradigm in which subjects must learn a relation (A > B and B > C) and generalize it to new combinations of items (A > C). Through mathematical analysis, we found that a broad range of biologically relevant learning models (e.g. gradient flow or ridge regression) perform TI successfully and recapitulate signature behavioral patterns long observed in living subjects. First, we found that models with item-wise additive representations automatically encode transitive relations. Second, for more general representations, a single scalar "conjunctivity factor" determines model behavior on TI and, further, the principle of norm minimization (a standard statistical inductive bias) enables models with fixed, partly conjunctive representations to generalize transitively. Finally, neural networks in the "rich regime," which enables representation learning and has been found to improve generalization, unexpectedly show poor generalization and anomalous behavior. We find that such networks implement a form of norm minimization (over hidden weights) that yields a local encoding mechanism lacking transitivity. Our findings show how minimal statistical learning principles give rise to a classical relational inductive bias (transitivity), explain empirically observed behaviors, and establish a formal approach to understanding the neural basis of relational abstraction.

neuroscience↗

bioRxiv2023

Situating the parietal memory network in the context of multiple parallel distributed networks using high-resolution functional connectivity

A principle of brain organization is that networks serving higher cognitive functions are widely distributed across the brain. One exception has been the parietal memory network (PMN), which plays a role in recognition memory but is often defined as being restricted to posteromedial association cortex. We hypothesized that high-resolution estimates of the PMN would reveal small regions that had been missed by prior approaches. High-field 7T functional magnetic resonance imaging (fMRI) data from extensively sampled participants was used to define the PMN within individuals. The PMN consistently extended beyond the core posteromedial set to include regions in the inferior parietal lobule; rostral, dorsal, medial, and ventromedial prefrontal cortex; the anterior insula; and ramus marginalis of the cingulate sulcus. The results suggest that, when fine-scale anatomy is considered, the PMN matches the expected distributed architecture of other association networks, reinforcing that parallel distributed networks are an organizing principle of association cortex.

neuroscience↗

bioRxiv2023

Expansion of a frontostriatal salience network in individuals with depression

Hundreds of neuroimaging studies spanning two decades have revealed differences in brain structure and functional connectivity in depression, but with modest effect sizes, complicating efforts to derive mechanistic pathophysiologic insights or develop biomarkers.1 Furthermore, although depression is a fundamentally episodic condition, few neuroimaging studies have taken a longitudinal approach, which is critical for understanding cause and effect and delineating mechanisms that drive mood state transitions over time. The emerging field of precision functional mapping using densely-sampled longitudinal neuroimaging data has revealed unexpected, functionally meaningful individual differences in brain network topology in healthy individuals,2-5 but these approaches have never been applied to individuals with depression. Here, using precision functional mapping techniques and 11 datasets comprising n=187 repeatedly sampled individuals and >21,000 minutes of fMRI data, we show that the frontostriatal salience network is expanded two-fold in most individuals with depression. This effect was replicable in multiple samples, including large-scale, group-average data (N=1,231 subjects), and caused primarily by network border shifts affecting specific functional systems, with three distinct modes of encroachment occurring in different individuals. Salience network expansion was unexpectedly stable over time, unaffected by changes in mood state, and detectable in children before the subsequent onset of depressive symptoms in adolescence. Longitudinal analyses of individuals scanned up to 62 times over 1.5 years identified connectivity changes in specific frontostriatal circuits that tracked fluctuations in specific symptom domains and predicted future anhedonia symptoms before they emerged. Together, these findings identify a stable trait-like brain network topology that may confer risk for depression and mood-state dependent connectivity changes in frontostriatal circuits that predict the emergence and remission of depressive symptoms over time.

neuroscience↗

bioRxiv2023

Neural mechanisms of transitive inference and fast knowledge reassembly in learning neural networks

How do we gain general insights from limited novel experiences? Humans and animals have a striking ability to learn relationships between experienced items, enabling efficient generalization and rapid assimilation of new information. One fundamental instance of such relational learning is transitive inference (learn A>B and B>C, infer A>C), which can be quickly and globally reorganized upon learning a new item (learn A>B>C and D>E>F, then C>D, and infer B>E). Despite considerable study, neural mechanisms of transitive inference and fast reassembly of existing knowledge remain elusive. Here we adopt a meta-learning ("learning-to-learn") approach. We train artificial neural networks, endowed with synaptic plasticity and neuromodulation, to be able to learn novel orderings of arbitrary stimuli from repeated presentation of stimulus pairs. We then obtain a complete mechanistic understanding of this discovered neural learning algorithm. Remarkably, this learning involves active cognition: items from previous trials are selectively reinstated in working memory, enabling delayed, self-generated learning and knowledge reassembly. These findings identify a new mechanism for relational learning and insight, suggest new interpretations of neural activity in cognitive tasks, and highlight a novel approach to discovering neural mechanisms capable of supporting cognitive behaviors.

neuroscience↗

bioRxiv2023

Stacked regressions and structured variance partitioning for interpretable brain maps

Relating brain activity associated with a complex stimulus to different properties of that stimulus is a powerful approach for constructing functional brain maps. However, when stimuli are naturalistic, their properties are often correlated (e.g., visual and semantic features of natural images, or different layers of a convolutional neural network that are used as features of images). Correlated properties can act as confounders for each other and complicate the interpretability of brain maps, and can impact the robustness of statistical estimators. Here, we present an approach for brain mapping based on two proposed methods: stacking different encoding models and structured variance partitioning. Our stacking algorithm combines encoding models that each use as input a feature space that describes a different stimulus attribute. The algorithm learns to predict the activity of a voxel as a linear combination of the outputs of different encoding models. We show that the resulting combined model can predict held-out brain activity better or at least as well as the individual encoding models. Further, the weights of the linear combination are readily interpretable; they show the importance of each feature space for predicting a voxel. We then build on our stacking models to introduce structured variance partitioning, a new type of variance partitioning that takes into account the known relationships between features. Our approach constrains the size of the hypothesis space and allows us to ask targeted questions about the similarity between feature spaces and brain regions even in the presence of correlations between the feature spaces. We validate our approach in simulation, showcase its brain mapping potential on fMRI data, and release a Python package. Our methods can be useful for researchers interested in aligning brain activity with different layers of a neural network, or with other types of correlated feature spaces.

neuroscience↗

bioRxiv2023

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are highly predictive of human brain responses to language. Here, using fMRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of brain response associated with each sentence. Then, we use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also noninvasively control neural activity in higher-level cortical areas, like the language network.

neuroscience↗

bioRxiv2023

BOLD Moments: modeling short visual events through a video fMRI dataset and metadata

Studying the neural basis of human dynamic visual perception requires extensive experimental data to evaluate the large swathes of functionally diverse brain neural networks driven by perceiving visual events. Here, we introduce the BOLD Moments Dataset (BMD), a repository of whole-brain fMRI responses to over 1,000 short (3s) naturalistic video clips of visual events across ten human subjects. We use the videos extensive metadata to show how the brain represents word- and sentence-level descriptions of visual events and identify correlates of video memorability scores extending into the parietal cortex. Furthermore, we reveal a match in hierarchical processing between cortical regions of interest and video-computable deep neural networks, and we showcase that BMD successfully captures temporal dynamics of visual events at second resolution. With its rich metadata, BMD offers new perspectives and accelerates research on the human brain basis of visual event perception.

neuroscience↗

bioRxiv2022

Engaging in word recognition elicits highly specific modulations in visual cortex

A persons cognitive state determines how their brain responds to visual stimuli. The most common such effect is a response enhancement when stimuli are task-relevant and attended rather than ignored. In this fMRI study, we report a surprising twist on such attention effects in the visual word form area (VWFA), a region that plays a key role in reading. We presented participants with strings of letters and visually similar shapes which were either relevant for a specific task (lexical decision or gap localization) or ignored (during a fixation dot color task). In the VWFA, the enhancement of responses to attended stimuli occurred only for letter strings, whereas the shapes evoked smaller responses when attended than when ignored. The enhancement of VWFA activity was accompanied by strengthened functional connectivity with higher-level language regions. These task-dependent modulations of response magnitude and functional connectivity were specific to the VWFA and absent in the rest of visual cortex. We suggest that language regions send targeted excitatory feedback into the VWFA only when the observer is trying to read. This feedback enables the discrimination of familiar and nonsense words, and is distinct from generic effects of visual attention.

neuroscience↗

bioRxiv2022

Neural dynamics and geometry for transitive inference

Relational cognition -- the ability to infer relationships that generalize to novel combinations of objects -- is fundamental to human and animal intelligence. Despite this importance, it remains unclear how relational cognition is implemented in the brain due in part to a lack of hypotheses and predictions at the levels of collective neural activity and behavior. Here we discovered, analyzed, and experimentally tested neural networks (NNs) that perform transitive inference (TI), a classic relational task (if A > B and B > C, then A > C). We found NNs that (i) generalized perfectly, despite lacking overt transitive structure prior to training, (ii) generalized when the task required working memory (WM), a capacity thought essential to inference in the brain, (iii) emergently expressed behaviors long observed in living subjects, in addition to a novel order-dependent behavior, and (iv) adopted different task solutions yielding alternative behavioral and neural predictions. Further, in a large-scale experiment, we found that human subjects performing WM-based TI showed behavior inconsistent with a class of NNs expressing an intuitive task solution. These findings provide neural insights into a classical relational ability, with wider implications for how the brain realizes relational cognition.

neuroscience↗

bioRxiv2022

Incorporating natural language into vision models improves prediction and understanding of higher visual cortex

Advances in neural networks have been catalyzed by joint training on images and natural language, increased dataset sizes, and data diversity. We explored whether the same factors support similar improvements in predicting visual responses in the human brain. We used models pre-trained with Contrastive Language-Image Pre-training (CLIP) - which learns image embeddings that best match text embeddings of image captions from diverse, large-scale datasets - to study visual representations. We built voxelwise encoding models based on CLIP image features to predict brain responses to real-world images. ResNet50 with CLIP explained up to R2 = 79% of variance in individual voxel responses in held-out test data, a significant increase from models trained only with image/label pairs (ImageNet trained ResNet) or text (BERT). Comparisons across different model backbones ruled out network architecture as a factor in performance improvements. Comparisons across models that controlled for dataset size and data diversity demonstrated that language feedback along with data diversity in larger datasets are important factors in explaining neural responses in high-level visual brain regions. Visualizations of model embeddings and Principal Component Analysis (PCA) revealed that our models capture both global and fine-grained semantic dimensions represented within human visual cortex.

neuroscience↗

bioRxiv2022

Re-expression of CA1 and entorhinal activity patterns preserves temporal context memory at long timescales

Converging, cross-species evidence indicates that memory for time is supported by hippocampal area CA1 and entorhinal cortex. However, limited evidence characterizes how these regions preserve temporal memories over long timescales (e.g., months). At long timescales, memoranda may be encountered in multiple temporal contexts, potentially creating interference. Here, using 7T fMRI, we measured CA1 and entorhinal activity patterns as human participants viewed thousands of natural scene images distributed, and repeated, across many months. We show that memory for an images original temporal context was predicted by the degree to which CA1/entorhinal activity patterns from the first encounter with an image were re-expressed during re-encounters occurring minutes to months later. Critically, temporal memory signals were dissociable from predictors of recognition confidence, which were carried by distinct medial temporal lobe expressions. These findings suggest that CA1 and entorhinal cortex preserve temporal memories across long timescales by coding for and reinstating temporal context information.

neuroscience↗

bioRxiv2022

Color-biased regions in the ventral visual pathway are food-selective

The ventral visual pathway is well known to be involved in recognizing and categorizing objects (Kanwisher and Dilks, 2013). Three color-biased areas have also been found between face and place selective areas in the ventral visual pathway (Lafer-Sousa et al., 2016). To understand the function of these color-biased areas in a region known for object recognition, we analyzed the Natural Scenes Dataset (NSD; Allen et al., 2022), a large 7T fMRI dataset from 8 participants who viewed up to 30,000 trials of images of colored natural scenes. In a whole-brain analysis, we correlated the average color saturation of the images and the voxel responses, revealing color-biased areas that diverge into two streams in the ventral visual pathway, beginning in V4 and extending medially and laterally of the Fusiform Face Area in both hemispheres. We drew regions of interest (ROIs) for the two streams and found that the images for each ROI that evoked the largest responses had certain characteristics: They contained food, contained circular objects, had higher color saturation, contained warmer hues, and had more luminance entropy. A multiple linear regression showed that the presence of food in images was the strongest predictor of voxel responses in the medial and lateral color-biased regions for all eight participants, but that color saturation also contributed independently to voxel responses. Our results show that these areas are food-selective and color biased. We suggest that these streams might be involved in using color to recognize and judge the properties of food.

neuroscience↗

bioRxiv2022

Brain-optimized neural networks learn non-hierarchical models of representation in human visual cortex

Deep neural networks (DNNs) trained to perform visual tasks learn representations that align with the hierarchy of visual areas in the primate brain. This finding has been taken to imply that the primate visual system forms representations by passing them through a hierarchical sequence of brain areas, just as DNNs form representations by passing them through a hierarchical sequence of layers. To test the validity of this assumption, we optimized DNNs not to perform visual tasks but to directly predict brain activity in human visual areas V1-V4. Using a massive sampling of human brain activity, we constructed brain-optimized networks that predict brain activity even more accurately than task-optimized networks. We show that brain-optimized networks can learn representations that diverge from those formed in a strict hierarchy. Brain-optimized networks do not need to align representations in V1-V4 with layer depth; moreover, they are able to accurately model anterior brain areas (e.g., V4) without computing intermediary representations associated with posterior brain areas (e.g., V1). Our results challenge the view that human visual areas V1-V4 act--like the early layers of a DNN--as a serial pre-processing sequence for higher areas, and suggest they may subserve their own independent functions.

neuroscience↗